Add sysctl cpu info task
[ikiwiki.git] / docs / developer / ProjectsPage.mdwn
CommitLineData
8c0cdf29 1# DragonFly Projects
b790f880 2
8c0cdf29 3This page shall serve as a common place to look if you're in search of a DragonFly related project. It's also the place to check if someone else is already working on it (to prevent project collision) or should be contacted.
b790f880 4
b1b401e0 5Projects that can be clearly used for Google Code-In are marked with their category, where applicable. Some unmarked items may be eligible; it depends on how the student wants to tackle the project.
6
8c0cdf29 7[[!toc levels=3 ]]
b790f880 8
8c0cdf29 9## Website projects
b790f880 10
06d07101 11### One-liners
b1b401e0 12* Add traffic report, especially to lists requests that cause 404s (GCI:Research)
13* Add the mail archive to the search index (GCI:Documentation)
14* Download link right on the main page.
8753e704 15* Fix RSS feed to have correct links
b1b401e0 16* Create layout for http://bugs.dragonflybsd.org that matches the main site. (GCI:User Interface)
8753e704 17* Create short list of tasks for a new DragonFly user
b1b401e0 18* How to upgrade the operating system (GCI:Training)
19* How to get to a working desktop (GCI:Training)
20* and where and how to report issues.(GCI:Documentation)
06d07101 21
22### Post papers in the proper locations on the website
8753e704 23* Format conversion may be necessary
24* Aggelos's papers from [http://leaf.dragonflybsd.org/~aggelos/] (http://leaf.dragonflybsd.org/~aggelos/) (netmp-paper.pdf and netmp.pdf) to Presentations
25* ["A Peek at the vKernel" article](http://cvsweb.dragonflybsd.org/cvsweb/site/data/docs/articles/vkernel/vkernel.shtml?rev=1.3&content-type=text/x-cvsweb-markup) from old site
b1b401e0 26(GCI:Documentation)
b790f880 27
8c0cdf29 28## Documentation projects
b790f880 29
06d07101 30### One-liners
8753e704 31* Reorder `/usr/src/UPDATING` to put more relevant information at top; remove data no longer relevant.
32* Help out in [http://bugs.dragonflybsd.org](http://bugs.dragonflybsd.org) (try to reproduce, diagnose, propose fixes ...)
33* Organize, contribute to, and finish the C book project.
06d07101 34
35### Handbook maintenance
8753e704 36* Check if the content applies to DragonFly. If not, change it.
37* Add new content and enhance the previous chapters
b1b401e0 38* Check for syntax error, typos and wiki errors.
39* Add prev/next buttons to all pages. (GCI:User Interface)
40
193c08af 41
87bf445d 42### Write manpages
b1b401e0 43* sysref (GCI:Documentation or Research)
87bf445d 44
8c0cdf29 45## Userland projects
b790f880 46
06d07101 47### One-liners
8753e704 48* Update the [[contributed software|docs/user/ContribSoftware]] which is out-of-date.
49* Remove `NOINET6` build option
b1b401e0 50* `WARN` corrections to utilities (GCI:Code)
8753e704 51* Bring in code from other *BSDs:
b1b401e0 52* smbfs changes from FreeBSD (GCI:Code)
8753e704 53* Add extended slice support to `fdisk`
54* Install Coverity and fix the FreeBSD bugs that were uncovered by Coverity, but do not just blindly pull over the FreeBSD patches. Make sure that you first understand what the patch does.
b1b401e0 55* C99 Standards Conformance. The todo list is on [[/docs/developer/StandardsConformanceProject]] (GCI:Code)
56* Setup a regression testing machine/system to register and find problems and new improvements. (GCI:Quality Assurance)
57* Networking performance / scalability ((GCI:Research)
58* [[RegressionTest|/docs/developer/RegressionTest]] (GCI:Research)
59* also check [[HowToStressTest|/docs/developer/HowToStressTest]] (GCI:Research)
8753e704 60* Add lwp support to ptrace/gdb/core dumps.
8753e704 61* Bringing in version 2.0 of the BSD Installer
62* UTF8 support in the console
06d07101 63
b1b401e0 64### Scalability (algorithmic performance) (GCI:Research)
65* [http://bulk.fefe.de/scalability/](http://bulk.fefe.de/scalability/)
8753e704 66* [http://bulk.fefe.de/lk2006/talk.pdf](http://bulk.fefe.de/lk2006/talk.pdf)
06d07101 67
4e927740 68### Clean our code to make it [style(9)](http://leaf.dragonflybsd.org/cgi/web-man?command=style&section=ANY#style&sectionANY) compatible. (GCI:Code)
8753e704 69* Compile and test your changes.
b1b401e0 70* Verify that the checksum (sha(1)) of the unmodified object matches the checksum of the cleaned object. Check also with strip(1)+sha(1)
06d07101 71
b1b401e0 72### Port BSD-licensed tools (ex: `grep`, `diff` and `sort`) (GCI:Code)
8753e704 73* The OpenBSD guys already did some work related to that.
74* If you manage to bring the tools to DragonFly, check if everything works as expected (e.g. rc.d scripts, make world runs, ...).
75* The new tools need to have at least all the features of the old GNU tools.
06d07101 76
77### Randomize mmap() offsets
8753e704 78* [http://www.openbsd.org/papers/ven05-deraadt/index.html](http://www.openbsd.org/papers/ven05-deraadt/index.html)
06d07101 79
99e24f21 80### GDB
8753e704 81* ptrace/gdb follow-fork-mode support and more (peek at linux)
82* Change the build to create one libbfd for gdb and binutils
83* Separate RPC code from NFS into separate library.
84
85### I/O diagnostic utilities
86* A utility similar to top which displays I/O usage on a per-process basis
87* Network, Disk
8c0cdf29 88
05f26292 89### SMART capabilities
05f26292 90* Add to camcontrol
8e7be9e6 91* Add to natacontrol (see NetBSD's atactl?)
05f26292 92
b1b401e0 93### Disk scheduling rc scripts (GCI:Code)
05f26292 94* Create a rc script to manage the disk/io scheduling system
95* Perhaps 1 entry to "enable" it, and all disks will have mode set to "auto", in auto mode smart inquiries and other heuristics could attempt to determine the best i/o scheduler
96* Per-device and device class or similar defaults should be definable in rc.conf also
dcf86f60 97* The bulk of this functionality could be implemented in a resurrected "dschedctl" utility and exposed through the rc interface using just a thin wrapper, allowing hotplug scripts and etc. an easier option to use the same facilities.
05f26292 98
b1b401e0 99### libHAMMER
3eda57ef 100* HAMMER has the capability to expose very rich information to userland through ioctl's.
101* Currently the hammer(8) utility makes use of this information in an ad-hoc manner.
102* Port this core functionality into a public libhammer library so that other base and third party utilities may take advantage of it.
8e7be9e6 103* See commit: cb7575e6a89409a2041a37fcfc22ce9e41297ab8 -- libHAMMER already exists, port functionality into it!
3eda57ef 104
5dd9f19a 105### Clean up crypt(3) consumers
106* crypt(3) can legitimately return NULL on error, but many consumers do not check for this, instead passing the return directly to str[n]cmp. Fix these cases.
107* Fix these cases in pkgsrc as well.
108
8c0cdf29 109## Kernel projects
110
06d07101 111### One-liners
8753e704 112* Port the BSDL OSS code to DragonFly
113* Complete Path MTU Discovery by adding a host route to remember the Path MTU and setting a timer to expire old host routes. See netinet/if_ether.c for an example of this mechanism as used by ARP. Periodically increase MTU of hosts that have had its MTU decreased.
114* Look for places in the kernel that can benefit from Solaris-style caching of preconstructed slab allocator objects. If we can find enough of these uses, we can add this functionality to the kernel memory allocator.
b1b401e0 115* Implement [`sem_open()`](http://www.opengroup.org/onlinepubs/000095399/functions/sem_open.html), [`sem_close()`](http://www.opengroup.org/onlinepubs/000095399/functions/sem_close.html), and [`sem_unlink()`](http://www.opengroup.org/onlinepubs/000095399/functions/sem_unlink.html). (GCI:Code)
8753e704 116* Clean our code to make it [style(9)](http://leaf.dragonflybsd.org/cgi/web-man?command#style&sectionANY) compatible. Compile and test your changes. Verify that the checksum (sha(1)) of the unmodified object matches the checksum of the cleaned object. Check also with strip(1)+sha(1)
b1b401e0 117* Setup a regression testing machine/system to register and find problems and new improvements. (GCI:Research)
8753e704 118* Port or update drivers from other systems.
119* Port NFSv4. [This mail](http://leaf.dragonflybsd.org/mailarchive/kernel/2008-01/msg00065.html) is a good starting point.
8753e704 120* clean up buildworld/kernel compilation warnings
8753e704 121* Port DTrace
52dfca9b 122* Port Linux emulation to x86-64
06d07101 123
124### Code to port/sync from FreeBSD
8753e704 125* PCI code (to take advantage of power saving features)
b1b401e0 126* Bring in support for UFS2, just the changes to extend the width of some fields from 32 bits to 64 bits. (GCI:Code)
127* hardware drivers.
99e24f21 128
06d07101 129### Code to port/sync from OpenBSD
8753e704 130* Add support for the NoExecute bit as described in [http://www.openbsd.org/papers/auug04/index.html](http://www.openbsd.org/papers/auug04/index.html).
131* After that, make user stacks and data heaps non-executable. (W^X)
132* hardware drivers, specially wireless.
06d07101 133
df817d42 134### CPU scheduler
8753e704 135* A scheduler API supporting multiple scheduler implementations already exists
136* Add a Solaris-like dispatcher framework that can handle more than one installed scheduling policy
06d07101 137
138### I/O scheduler
df817d42 139* A pluggable kernel I/O scheduler already exists
140* Implement additional disk scheduling policies
8753e704 141
43da2530 142### Implement boot cache
143* Effectively a smart readahead.
947d3e3a 144* Store the pattern of incoming read requests of the boot disk. (Likely into an array structure for better linear readback, We mmap() the whole thing into RAM on boot.)
43da2530 145* Sort pattern into a playback list. (Burstsort or a simple quick sort to get things going?)
146* Provide feedback of boot cache playback list hit rate for better adaptiveness.
147
c1941460 148### Improve kernel boot speed
149* Research source of delays in boot process, keyboard init, scsi?
150* Better thread some hardware init, for example USB?
151* Perhaps look to see how Linux can boot in one second, better pci scan code?
ab4ba9d2 152* "Some kernel work made it possible to do asynchronous initialization of some subsystems. For example, the modified kernel starts the Advanced Host Controller Interface (AHCI) initialization, to handle storage, at the same time as the Universal Host Controller Interface (UHCI), in order to handle USB" - http://lwn.net/Articles/299483/
c1941460 153
b1b401e0 154### Modify firmware framework
1ac22aa1 155* We currently use the firmware(9) FreeBSD also uses
156* It would be more appropriate to avoid future problems with redistribution problems, etc, to adapt the wifi firmware stuff to use the firmware(9) we used to have before, which was able to load firmware files from userland (/etc/firmware).
157* See "firmware discussion" thread on kernel@ mailing list, March-May 2010.
ca622843 158* Since it's not desired to just add another way of doing it, this project should include getting rid of loading firmwares as modules
1ac22aa1 159
8753e704 160### Work relating to LWKT (LightWeightKernelThreading)
161* Implement lazy IPI cross-processor lwkt message passing.
162* Add timeout functionality to lwkt_waitmsg().
8753e704 163
164### Filesystem extended attributes
165* Generic VFS attributes layer
166* Emulate attributes ala Darwin
167* Allow filesystems to define their own attribute vop ops
168* QUESTIONS: Attributes or subfiles? The consensus is that subfiles are better?
169
b6a0e48f 170### Capabilities
171* Implement something resembling or inspired by POSIX.1e
172* This implementation could possibly exist in userland and interlock with the kernel via a VFS Journal or HAMMER mirroring ioctl's.
173* These should be capable of supporting NFSv4 capabilities.
174* References: [1](http://leaf.dragonflybsd.org/mailarchive/kernel/2006-07/msg00012.html)
175
8753e704 176### Hardware virtualization extensions
177* Increase performance of virtual kernels
178* Make use of hardware virtualization extensions, if supported, to manage vmspaces
0b48e63b 179* Implement a KVM-compatible virtualization device to support qemu
8898930f 180* Hardware IOMMU support is not a priority and if implemented must be strictly optional.
8753e704 181
f1dc70a1 182### Remove zalloc
183* zalloc is a deprecated kernel interface and all current consumers can be ported to objcache, with varying levels of difficulty.
48b27198 184* zlib, pv entries
f1dc70a1 185* Once all consumers are ported, zalloc can be removed.
97f327a5 186
b6a0e48f 187### Document all sysctl's
188* The description of sysctl's can be provided when the sysctl is declared and displayed in userland by passing the -d flag to the sysctl utility.
189* Document all undocumented sysctl's
190* Verify that the description of documented sysctl's is correct.
191
192### Convert kprintf-enabling sysctl's to KTR's
193* Many sysctl's simply enable one or more kernel kprintf's
194* This can be very easy, or can be very unwieldy
195* Convert all of these cases to ktr's, while slightly less easy they are far easier to wield in all cases.
196
b1b401e0 197### Tear out C/H/S disk reporting (GCI:Research)
bd0ac588 198* Cylinders/Heads/Sectors are an outdated concept and the system doesn't rely on them anymore.
199* Verify the assumption that we don't rely on them in any way, shape or form.
200* Tear the reporting out of the kernel/installer/etc.
201
e2952a1f 202### Change vm_map lookup algorithm
203* The vm_map lookups currently use a Red-Black tree, since 2005.
204* It has been decided that using an array'ized bucket'ized hash table is probably a better approach.
205* References: [1](http://kerneltrap.org/mailarchive/freebsd-current/2010/10/1/6260944) [2](http://leaf.dragonflybsd.org/mailarchive/kernel/2005-01/msg00122.html)
97f327a5 206
0422aa9f
M
207### Trampoline Code Page instead of direct syscalls
208* Map a read/execute trampoline page into every process, which has a syscall table, i.e. is used instead of "int" to enter the kernel. This way we can easily change the kernel enter method from "int" to e.g. "sysenter" without having to recompile userland applications, or even implement some syscalls in userspace.
209
210### Compressed in-memory swap device
211* A device that uses physical memory as swap space, but compresses it.
212* Do we support stacking of swap space? For example, one would have this compressed in-memory swap device with highest priority. Replaced objects will be put into the next priority swap device (e.g. a SSD), and so on.
213
fa076070 214### tmpfs allocations from swap
215* Currently, tmpfs nodes and stuff are allocated from KVA are the size limiter for a tmpfs filesystem
216* Instead allocate them from swappable memory; this will allow larger tmpfses up to swap limits
217
3114c38f 218### mmap MAP_ALIGN
219* Solaris's mmap support a flag, MAP_ALIGN, where the address to mmap acts as an alignment hint
220* Our backing VM calls support an alignment parameter, but our public mmap does not
221* This would allow nmalloc to allocate slabs (64k, 64k-aligned) without wastage
222
fb27736e 223### vnode dumps
224* It would be beneficial to be able to have crash dumps written to an ordinary file in configurations where swap is not configured.
225
af80ec85 226### Kernel allocator feature enhancements
227* Enhance objcache in a manner that will allow the initialization to specify allocation functions which will allocate and free entire slabs of memory, not just single objects.
60f380f8 228* Rework the kmalloc allocator to sit on top of objcache (there are various dependencies to making this work).
229* Create or sort out a metric for vnode/other cache object cycling rate that can indicate realistic memory pressure.
230* Clear out free objcache slabs under memory pressure.
231* Add reclaim functionality to objcache, such that it may ask consumers to free objects back into it under memory pressure.
af80ec85 232
8e7be9e6 233### Tear out serializers
234* Serializers could be _carefully_ replaced with MTX locks
235
236### Tear out condvars
237* Conditional vars -- condvar(9), could be replaced with other locking primitives and our tsleep/wakeup interlock.
238
8ac2913d 239### Make karc4random in libkern per-cpu
240* Verify that it is possible and safe to do this, what care would need to be taken, especially with respect to the random seeding?
241* Pull out locks around calls to karc4rand*
242
d882bf8a 243### Improve kernel spinlock debug-ability
244* Add a const char *descr field to the spinlock structure.
245* Adjust spin_init() and all use cases, SPINLOCK_INITIALIZER() could juse use macro string extensions (# head) to install the name there.
246* Adjust error messages to print the spinlock desc field.
247* This change will require a full world & kernel recompile.
248
ff694554 249### Add informational hardware-related sysctl's
250* MacOS X exports a bunch of cpu-specific sysctl's detailing sizes, layout, features, etc. Most/all of this could be detected by a userspace program, but exporting these makes the barrier of entry to cpu-conditional code lower.
251* Research which of these sysctl's is the most useful and add them, using the same node names as OSX.
2521. hw.vectorunit = 1
2531. hw.busfrequency = 100000000
2541. hw.cpufrequency = 3062000000
2551. hw.cachelinesize = 64
2561. hw.l1icachesize = 32768
2571. hw.l1dcachesize = 32768
2581. hw.l2settings = 1
2591. hw.l2cachesize = 262144
2601. hw.l3settings = 1
2611. hw.l3cachesize = 6291456
2621. hw.tbfrequency = 1000000000
2631. hw.activecpu: 2
2641. hw.physicalcpu: 2
2651. hw.physicalcpu_max: 2
2661. hw.logicalcpu: 2
2671. hw.logicalcpu_max: 2
2681. hw.cputype: 7
2691. hw.cpusubtype: 4
2701. hw.cpu64bit_capable: 1
2711. hw.cpufamily: 1418770316
2721. hw.cacheconfig: 2 1 1 1 0 0 0 0 0 0
2731. hw.cachesize: 2147483648 32768 262144 6291456 0 0 0 0 0 0
2741. hw.busfrequency: 100000000
2751. hw.busfrequency_min: 100000000
2761. hw.busfrequency_max: 100000000
2771. hw.cpufrequency: 3062000000
2781. hw.cpufrequency_min: 3062000000
2791. hw.cpufrequency_max: 3062000000
2801. hw.cachelinesize: 64
2811. hw.l1icachesize: 32768
2821. hw.l1dcachesize: 32768
2831. hw.l2cachesize: 262144
2841. hw.l3cachesize: 6291456
2851. hw.tbfrequency: 1000000000
2861. hw.packages: 2
2871. hw.optional.floatingpoint: 1
2881. hw.optional.mmx: 1
2891. hw.optional.sse: 1
2901. hw.optional.sse2: 1
2911. hw.optional.sse3: 1
2921. hw.optional.supplementalsse3: 1
2931. hw.optional.sse4_1: 1
2941. hw.optional.sse4_2: 1
2951. hw.optional.x86_64: 1
2961. hw.optional.aes: 1
2971. hw.optional.avx1_0: 1
2981. hw.optional.rdrand: 0
2991. hw.optional.f16c: 0
3001. hw.optional.enfstrg: 0
3011. machdep.cpu.max_basic: 13
3021. machdep.cpu.max_ext: 2147483656
3031. machdep.cpu.vendor: GenuineIntel
3041. machdep.cpu.brand_string: Intel(R) Core(TM) i5-3470 CPU @ 3.20GHz
3051. machdep.cpu.family: 6
3061. machdep.cpu.model: 42
3071. machdep.cpu.extmodel: 2
3081. machdep.cpu.extfamily: 0
3091. machdep.cpu.stepping: 7
3101. machdep.cpu.feature_bits: 262929407 2660770315
3111. machdep.cpu.extfeature_bits: 672139520 1
3121. machdep.cpu.signature: 132775
3131. machdep.cpu.brand: 0
3141. machdep.cpu.features: FPU VME DE PSE TSC MSR PAE MCE CX8 APIC SEP MTRR PGE MCA CMOV PAT PSE36 CLFSH DS MMX FXSR SSE SSE2 SS SSE3 PCLMULQDQ MON SSSE3 CX16 SSE4.1 SSE4.2 POPCNT AES VMM XSAVE OSXSAVE AVX1.0
3151. machdep.cpu.extfeatures: SYSCALL XD EM64T LAHF RDTSCP TSCI
3161. machdep.cpu.cores_per_package: 1
3171. machdep.cpu.microcode_version: 16
3181. machdep.cpu.processor_flag: 0
3191. machdep.cpu.mwait.linesize_min: 4096
3201. machdep.cpu.mwait.linesize_max: 4096
3211. machdep.cpu.mwait.extensions: 3
3221. machdep.cpu.mwait.sub_Cstates: 4384
3231. machdep.cpu.xsave.extended_state: 7 832 832 0
3241. machdep.cpu.arch_perf.version: 1
3251. machdep.cpu.arch_perf.number: 8
3261. machdep.cpu.arch_perf.width: 48
3271. machdep.cpu.arch_perf.events_number: 7
3281. machdep.cpu.arch_perf.events: 127
3291. machdep.cpu.arch_perf.fixed_number: 0
3301. machdep.cpu.arch_perf.fixed_width: 0
3311. machdep.cpu.cache.linesize: 64
3321. machdep.cpu.cache.L2_associativity: 8
3331. machdep.cpu.cache.size: 256
3341. machdep.cpu.tlb.inst.small: 128
3351. machdep.cpu.tlb.data.small: 64
3361. machdep.cpu.tlb.data.large: 32
3371. machdep.cpu.tlb.shared: 512
3381. machdep.cpu.address_bits.physical: 40
3391. machdep.cpu.address_bits.virtual: 48
3401. machdep.cpu.core_count: 1
3411. machdep.cpu.thread_count: 1
342
b10d5b08 343## Live images enhancement
54596ebd 344
d0512cee 345The live images could be made easier to use by not assuming a QWERTY keyboard layout.
0db55667 346
d0512cee 347Not beeing able to choose a localized keyboard layout before having to type 'installer' or 'root' may be a deal breaker for some users.
094b2c92 348A simple menu replacing login(1) and displaying the following three choices could be a good solution:
d0512cee 349
3501. Set keyboard layout
3512. Launch the installer
3523. Login as root
094b2c92 353
97f327a5 354For more theoretical projects and project concepts see [[ResearchProjects|/docs/developer/ResearchProjectsPage]]