(no commit message)
[ikiwiki.git] / docs / developer / ProjectsPage.mdwn
8c0cdf29 1# DragonFly Projects
b790f880 2
8c0cdf29 3This page shall serve as a common place to look if you're in search of a DragonFly related project. It's also the place to check if someone else is already working on it (to prevent project collision) or should be contacted.
b790f880 4
b1b401e0 5Projects that can be clearly used for Google Code-In are marked with their category, where applicable. Some unmarked items may be eligible; it depends on how the student wants to tackle the project.
8c0cdf29 7[[!toc levels=3 ]]
b790f880 8
8c0cdf29 9## Documentation projects
b790f880 10
87bf445d 11### Write manpages
b1b401e0 12* sysref (GCI:Documentation or Research)
87bf445d 13
8c0cdf29 14## Userland projects
b790f880 15
06d07101 16### One-liners
8116f646 17* Bring in smbfs changes from FreeBSD (GCI:Code)
1571f8e5 18* rpc.lockd and rpc.statd sync with FreeBSD (GCI:Code)
8753e704 19* Add extended slice support to `fdisk`
b1b401e0 20* C99 Standards Conformance. The todo list is on [[/docs/developer/StandardsConformanceProject]] (GCI:Code)
8753e704 21* Add lwp support to ptrace/gdb/core dumps.
8753e704 22* UTF8 support in the console
06d07101 23
8116f646 24### dma
25* Make dma(8) our default MTA
26* Remove sendmail (optional)
b1b401e0 28### Scalability (algorithmic performance) (GCI:Research)
29* [http://bulk.fefe.de/scalability/](http://bulk.fefe.de/scalability/)
8753e704 30* [http://bulk.fefe.de/lk2006/talk.pdf](http://bulk.fefe.de/lk2006/talk.pdf)
06d07101 31
4e927740 32### Clean our code to make it [style(9)](http://leaf.dragonflybsd.org/cgi/web-man?command=style&section=ANY#style&sectionANY) compatible. (GCI:Code)
8753e704 33* Compile and test your changes.
b1b401e0 34* Verify that the checksum (sha(1)) of the unmodified object matches the checksum of the cleaned object. Check also with strip(1)+sha(1)
06d07101 35
b1b401e0 36### Port BSD-licensed tools (ex: `grep`, `diff` and `sort`) (GCI:Code)
8753e704 37* The OpenBSD guys already did some work related to that.
38* If you manage to bring the tools to DragonFly, check if everything works as expected (e.g. rc.d scripts, make world runs, ...).
06d07101 39
99e24f21 40### GDB
8753e704 41* ptrace/gdb follow-fork-mode support and more (peek at linux)
42* Change the build to create one libbfd for gdb and binutils
43* Separate RPC code from NFS into separate library.
45### I/O diagnostic utilities
46* A utility similar to top which displays I/O usage on a per-process basis
47* Network, Disk
8c0cdf29 48
05f26292 49### SMART capabilities
05f26292 50* Add to camcontrol
8e7be9e6 51* Add to natacontrol (see NetBSD's atactl?)
05f26292 52
b1b401e0 53### Disk scheduling rc scripts (GCI:Code)
05f26292 54* Create a rc script to manage the disk/io scheduling system
55* Perhaps 1 entry to "enable" it, and all disks will have mode set to "auto", in auto mode smart inquiries and other heuristics could attempt to determine the best i/o scheduler
56* Per-device and device class or similar defaults should be definable in rc.conf also
dcf86f60 57* The bulk of this functionality could be implemented in a resurrected "dschedctl" utility and exposed through the rc interface using just a thin wrapper, allowing hotplug scripts and etc. an easier option to use the same facilities.
05f26292 58
b1b401e0 59### libHAMMER
3eda57ef 60* HAMMER has the capability to expose very rich information to userland through ioctl's.
61* Currently the hammer(8) utility makes use of this information in an ad-hoc manner.
62* Port this core functionality into a public libhammer library so that other base and third party utilities may take advantage of it.
8e7be9e6 63* See commit: cb7575e6a89409a2041a37fcfc22ce9e41297ab8 -- libHAMMER already exists, port functionality into it!
3eda57ef 64
5dd9f19a 65### Clean up crypt(3) consumers
66* crypt(3) can legitimately return NULL on error, but many consumers do not check for this, instead passing the return directly to str[n]cmp. Fix these cases.
67* Fix these cases in pkgsrc as well.
8c0cdf29 69## Kernel projects
06d07101 71### One-liners
8753e704 72* Port the BSDL OSS code to DragonFly
73* Complete Path MTU Discovery by adding a host route to remember the Path MTU and setting a timer to expire old host routes. See netinet/if_ether.c for an example of this mechanism as used by ARP. Periodically increase MTU of hosts that have had its MTU decreased.
b1b401e0 74* Implement [`sem_open()`](http://www.opengroup.org/onlinepubs/000095399/functions/sem_open.html), [`sem_close()`](http://www.opengroup.org/onlinepubs/000095399/functions/sem_close.html), and [`sem_unlink()`](http://www.opengroup.org/onlinepubs/000095399/functions/sem_unlink.html). (GCI:Code)
8753e704 75* Port or update drivers from other systems.
52dfca9b 76* Port Linux emulation to x86-64
06d07101 77
78### Code to port/sync from FreeBSD
8753e704 79* PCI code (to take advantage of power saving features)
b1b401e0 80* Bring in support for UFS2, just the changes to extend the width of some fields from 32 bits to 64 bits. (GCI:Code)
81* hardware drivers.
8116f646 82* ext2fs
83* DTrace
99e24f21 84
06d07101 85### Code to port/sync from OpenBSD
8753e704 86* Add support for the NoExecute bit as described in [http://www.openbsd.org/papers/auug04/index.html](http://www.openbsd.org/papers/auug04/index.html).
87* After that, make user stacks and data heaps non-executable. (W^X)
88* hardware drivers, specially wireless.
06d07101 89
df817d42 90### CPU scheduler
8753e704 91* A scheduler API supporting multiple scheduler implementations already exists
92* Add a Solaris-like dispatcher framework that can handle more than one installed scheduling policy
06d07101 93
94### I/O scheduler
df817d42 95* A pluggable kernel I/O scheduler already exists
96* Implement additional disk scheduling policies
8753e704 97
43da2530 98### Implement boot cache
99* Effectively a smart readahead.
7957e767 100* Store the pattern of incoming read requests of the boot disk.
43da2530 101
c1941460 102### Improve kernel boot speed
103* Research source of delays in boot process, keyboard init, scsi?
104* Better thread some hardware init, for example USB?
105* Perhaps look to see how Linux can boot in one second, better pci scan code?
ab4ba9d2 106* "Some kernel work made it possible to do asynchronous initialization of some subsystems. For example, the modified kernel starts the Advanced Host Controller Interface (AHCI) initialization, to handle storage, at the same time as the Universal Host Controller Interface (UHCI), in order to handle USB" - http://lwn.net/Articles/299483/
c1941460 107
b1b401e0 108### Modify firmware framework
1ac22aa1 109* We currently use the firmware(9) FreeBSD also uses
110* It would be more appropriate to avoid future problems with redistribution problems, etc, to adapt the wifi firmware stuff to use the firmware(9) we used to have before, which was able to load firmware files from userland (/etc/firmware).
111* See "firmware discussion" thread on kernel@ mailing list, March-May 2010.
ca622843 112* Since it's not desired to just add another way of doing it, this project should include getting rid of loading firmwares as modules
1ac22aa1 113
8753e704 114### Work relating to LWKT (LightWeightKernelThreading)
115* Implement lazy IPI cross-processor lwkt message passing.
116* Add timeout functionality to lwkt_waitmsg().
8753e704 117
118### Filesystem extended attributes
119* Generic VFS attributes layer
120* Emulate attributes ala Darwin
121* Allow filesystems to define their own attribute vop ops
122* QUESTIONS: Attributes or subfiles? The consensus is that subfiles are better?
b6a0e48f 124### Capabilities
125* Implement something resembling or inspired by POSIX.1e
126* This implementation could possibly exist in userland and interlock with the kernel via a VFS Journal or HAMMER mirroring ioctl's.
127* These should be capable of supporting NFSv4 capabilities.
128* References: [1](http://leaf.dragonflybsd.org/mailarchive/kernel/2006-07/msg00012.html)
8753e704 130### Hardware virtualization extensions
131* Increase performance of virtual kernels
132* Make use of hardware virtualization extensions, if supported, to manage vmspaces
0b48e63b 133* Implement a KVM-compatible virtualization device to support qemu
8898930f 134* Hardware IOMMU support is not a priority and if implemented must be strictly optional.
8753e704 135
f1dc70a1 136### Remove zalloc
137* zalloc is a deprecated kernel interface and all current consumers can be ported to objcache, with varying levels of difficulty.
48b27198 138* zlib, pv entries
f1dc70a1 139* Once all consumers are ported, zalloc can be removed.
97f327a5 140
b6a0e48f 141### Convert kprintf-enabling sysctl's to KTR's
142* Many sysctl's simply enable one or more kernel kprintf's
143* This can be very easy, or can be very unwieldy
144* Convert all of these cases to ktr's, while slightly less easy they are far easier to wield in all cases.
8116f646 146### Tear out C/H/S disk reporting
bd0ac588 147* Cylinders/Heads/Sectors are an outdated concept and the system doesn't rely on them anymore.
148* Verify the assumption that we don't rely on them in any way, shape or form.
149* Tear the reporting out of the kernel/installer/etc.
e2952a1f 151### Change vm_map lookup algorithm
152* The vm_map lookups currently use a Red-Black tree, since 2005.
153* It has been decided that using an array'ized bucket'ized hash table is probably a better approach.
154* References: [1](http://kerneltrap.org/mailarchive/freebsd-current/2010/10/1/6260944) [2](http://leaf.dragonflybsd.org/mailarchive/kernel/2005-01/msg00122.html)
97f327a5 155
156### Trampoline Code Page instead of direct syscalls
157* Map a read/execute trampoline page into every process, which has a syscall table, i.e. is used instead of "int" to enter the kernel. This way we can easily change the kernel enter method from "int" to e.g. "sysenter" without having to recompile userland applications, or even implement some syscalls in userspace.
159### Compressed in-memory swap device
160* A device that uses physical memory as swap space, but compresses it.
161* Do we support stacking of swap space? For example, one would have this compressed in-memory swap device with highest priority. Replaced objects will be put into the next priority swap device (e.g. a SSD), and so on.
3114c38f 163### mmap MAP_ALIGN
164* Solaris's mmap support a flag, MAP_ALIGN, where the address to mmap acts as an alignment hint
165* Our backing VM calls support an alignment parameter, but our public mmap does not
166* This would allow nmalloc to allocate slabs (64k, 64k-aligned) without wastage
fb27736e 168### vnode dumps
8116f646 169* It may be beneficial to be able to have crash dumps written to an ordinary file in configurations where swap is not configured.
171### vnode swap
172* Add a vnode-backed swap pager that respects a file size limit, to allow paging to a "swap file" on a filesystem.
fb27736e 173
af80ec85 174### Kernel allocator feature enhancements
175* Enhance objcache in a manner that will allow the initialization to specify allocation functions which will allocate and free entire slabs of memory, not just single objects.
60f380f8 176* Rework the kmalloc allocator to sit on top of objcache (there are various dependencies to making this work).
177* Create or sort out a metric for vnode/other cache object cycling rate that can indicate realistic memory pressure.
178* Clear out free objcache slabs under memory pressure.
179* Add reclaim functionality to objcache, such that it may ask consumers to free objects back into it under memory pressure.
af80ec85 180
8e7be9e6 181### Tear out serializers
8116f646 182* Serializers could be _carefully_ replaced with MTX locks?
8e7be9e6 183
184### Tear out condvars
185* Conditional vars -- condvar(9), could be replaced with other locking primitives and our tsleep/wakeup interlock.
c52e2798 187### Make karc4random in libkern per-cpu (or at least wrap its own token around it)
8ac2913d 188* Verify that it is possible and safe to do this, what care would need to be taken, especially with respect to the random seeding?
189* Pull out locks around calls to karc4rand*
d882bf8a 191### Improve kernel spinlock debug-ability
192* Add a const char *descr field to the spinlock structure.
193* Adjust spin_init() and all use cases, SPINLOCK_INITIALIZER() could juse use macro string extensions (# head) to install the name there.
194* Adjust error messages to print the spinlock desc field.
195* This change will require a full world & kernel recompile.
d866bea9 197### Partially rewrite buffer cache
198* Buffer cache buffers map VM pages from the filesystems VM-backed vnodes, the "buffer space" is the maximum amount of virtual space to allocate to these buffers.
199* When the buffer_map KVA space gets fragmented it caused very expensive defrag operations, the buffer_map KVA was recently increased to double the actual buffer space to allow full space utilization in the face of fragmentation and reduce the frequency of defrag operations.
200* Rewrite the buffer cache to have separate spaces or separate buffer pools for different sizes (16, 32, 64, 128, 256... up to max).
201* The vm_map API is used to allocate kvm out of the buffer_map. If the buffer's KVAs are preallocated then things can basically just be setup linearly at boot time.
ff694554 203### Add informational hardware-related sysctl's
204* MacOS X exports a bunch of cpu-specific sysctl's detailing sizes, layout, features, etc. Most/all of this could be detected by a userspace program, but exporting these makes the barrier of entry to cpu-conditional code lower.
205* Research which of these sysctl's is the most useful and add them, using the same node names as OSX.
2061. hw.vectorunit = 1
2071. hw.busfrequency = 100000000
2081. hw.cpufrequency = 3062000000
2091. hw.cachelinesize = 64
2101. hw.l1icachesize = 32768
2111. hw.l1dcachesize = 32768
2121. hw.l2settings = 1
2131. hw.l2cachesize = 262144
2141. hw.l3settings = 1
2151. hw.l3cachesize = 6291456
2161. hw.tbfrequency = 1000000000
2171. hw.activecpu: 2
2181. hw.physicalcpu: 2
2191. hw.physicalcpu_max: 2
2201. hw.logicalcpu: 2
2211. hw.logicalcpu_max: 2
2221. hw.cputype: 7
2231. hw.cpusubtype: 4
2241. hw.cpu64bit_capable: 1
2251. hw.cpufamily: 1418770316
2261. hw.cacheconfig: 2 1 1 1 0 0 0 0 0 0
2271. hw.cachesize: 2147483648 32768 262144 6291456 0 0 0 0 0 0
2281. hw.busfrequency: 100000000
2291. hw.busfrequency_min: 100000000
2301. hw.busfrequency_max: 100000000
2311. hw.cpufrequency: 3062000000
2321. hw.cpufrequency_min: 3062000000
2331. hw.cpufrequency_max: 3062000000
2341. hw.cachelinesize: 64
2351. hw.l1icachesize: 32768
2361. hw.l1dcachesize: 32768
2371. hw.l2cachesize: 262144
2381. hw.l3cachesize: 6291456
2391. hw.tbfrequency: 1000000000
2401. hw.packages: 2
2411. hw.optional.floatingpoint: 1
2421. hw.optional.mmx: 1
2431. hw.optional.sse: 1
2441. hw.optional.sse2: 1
2451. hw.optional.sse3: 1
2461. hw.optional.supplementalsse3: 1
2471. hw.optional.sse4_1: 1
2481. hw.optional.sse4_2: 1
2491. hw.optional.x86_64: 1
2501. hw.optional.aes: 1
2511. hw.optional.avx1_0: 1
2521. hw.optional.rdrand: 0
2531. hw.optional.f16c: 0
2541. hw.optional.enfstrg: 0
2551. machdep.cpu.max_basic: 13
2561. machdep.cpu.max_ext: 2147483656
2571. machdep.cpu.vendor: GenuineIntel
2581. machdep.cpu.brand_string: Intel(R) Core(TM) i5-3470 CPU @ 3.20GHz
2591. machdep.cpu.family: 6
2601. machdep.cpu.model: 42
2611. machdep.cpu.extmodel: 2
2621. machdep.cpu.extfamily: 0
2631. machdep.cpu.stepping: 7
2641. machdep.cpu.feature_bits: 262929407 2660770315
2651. machdep.cpu.extfeature_bits: 672139520 1
2661. machdep.cpu.signature: 132775
2671. machdep.cpu.brand: 0
2691. machdep.cpu.extfeatures: SYSCALL XD EM64T LAHF RDTSCP TSCI
2701. machdep.cpu.cores_per_package: 1
2711. machdep.cpu.microcode_version: 16
2721. machdep.cpu.processor_flag: 0
2731. machdep.cpu.mwait.linesize_min: 4096
2741. machdep.cpu.mwait.linesize_max: 4096
2751. machdep.cpu.mwait.extensions: 3
2761. machdep.cpu.mwait.sub_Cstates: 4384
2771. machdep.cpu.xsave.extended_state: 7 832 832 0
2781. machdep.cpu.arch_perf.version: 1
2791. machdep.cpu.arch_perf.number: 8
2801. machdep.cpu.arch_perf.width: 48
2811. machdep.cpu.arch_perf.events_number: 7
2821. machdep.cpu.arch_perf.events: 127
2831. machdep.cpu.arch_perf.fixed_number: 0
2841. machdep.cpu.arch_perf.fixed_width: 0
2851. machdep.cpu.cache.linesize: 64
2861. machdep.cpu.cache.L2_associativity: 8
2871. machdep.cpu.cache.size: 256
2881. machdep.cpu.tlb.inst.small: 128
2891. machdep.cpu.tlb.data.small: 64
2901. machdep.cpu.tlb.data.large: 32
2911. machdep.cpu.tlb.shared: 512
2921. machdep.cpu.address_bits.physical: 40
2931. machdep.cpu.address_bits.virtual: 48
2941. machdep.cpu.core_count: 1
2951. machdep.cpu.thread_count: 1
b10d5b08 297## Live images enhancement
54596ebd 298
d0512cee 299The live images could be made easier to use by not assuming a QWERTY keyboard layout.
0db55667 300
d0512cee 301Not beeing able to choose a localized keyboard layout before having to type 'installer' or 'root' may be a deal breaker for some users.
094b2c92 302A simple menu replacing login(1) and displaying the following three choices could be a good solution:
d0512cee 303
3041. Set keyboard layout
3052. Launch the installer
3063. Login as root
094b2c92 307
97f327a5 308For more theoretical projects and project concepts see [[ResearchProjects|/docs/developer/ResearchProjectsPage]]