X-Git-Url: https://gitweb.dragonflybsd.org/ikiwiki.git/blobdiff_plain/d0512cee8f15c18e750841d1ad18173f9f3e404e..HEAD:/docs/developer/ProjectsPage.mdwn diff --git a/docs/developer/ProjectsPage.mdwn b/docs/developer/ProjectsPage.mdwn index 81dc33e7..b29d72b6 100644 --- a/docs/developer/ProjectsPage.mdwn +++ b/docs/developer/ProjectsPage.mdwn @@ -1,64 +1,24 @@ -# DragonFly Projects +[[!meta title="DragonFly Projects"]] This page shall serve as a common place to look if you're in search of a DragonFly related project. It's also the place to check if someone else is already working on it (to prevent project collision) or should be contacted. Projects that can be clearly used for Google Code-In are marked with their category, where applicable. Some unmarked items may be eligible; it depends on how the student wants to tackle the project. -[[!toc levels=3 ]] - -## Website projects - -### One-liners -* Add traffic report, especially to lists requests that cause 404s (GCI:Research) -* Add the mail archive to the search index (GCI:Documentation) -* Download link right on the main page. -* Fix RSS feed to have correct links -* Create layout for http://bugs.dragonflybsd.org that matches the main site. (GCI:User Interface) -* Create short list of tasks for a new DragonFly user -* How to upgrade the operating system (GCI:Training) -* How to get to a working desktop (GCI:Training) -* and where and how to report issues.(GCI:Documentation) - -### Post papers in the proper locations on the website -* Format conversion may be necessary -* Aggelos's papers from [http://leaf.dragonflybsd.org/~aggelos/] (http://leaf.dragonflybsd.org/~aggelos/) (netmp-paper.pdf and netmp.pdf) to Presentations -* ["A Peek at the vKernel" article](http://cvsweb.dragonflybsd.org/cvsweb/site/data/docs/articles/vkernel/vkernel.shtml?rev=1.3&content-type=text/x-cvsweb-markup) from old site -(GCI:Documentation) +[[!toc levels=2 startlevel=2]] ## Documentation projects -### One-liners -* Reorder `/usr/src/UPDATING` to put more relevant information at top; remove data no longer relevant. -* Help out in [http://bugs.dragonflybsd.org](http://bugs.dragonflybsd.org) (try to reproduce, diagnose, propose fixes ...) -* Organize, contribute to, and finish the C book project. - -### Handbook maintenance -* Check if the content applies to DragonFly. If not, change it. -* Add new content and enhance the previous chapters -* Check for syntax error, typos and wiki errors. -* Add prev/next buttons to all pages. (GCI:User Interface) - - ### Write manpages * sysref (GCI:Documentation or Research) ## Userland projects ### One-liners -* Update the [[contributed software|docs/user/ContribSoftware]] which is out-of-date. -* Remove `NOINET6` build option -* `WARN` corrections to utilities (GCI:Code) -* Bring in code from other *BSDs: -* smbfs changes from FreeBSD (GCI:Code) +* Bring in smbfs changes from FreeBSD (GCI:Code) +* rpc.lockd and rpc.statd sync with FreeBSD (GCI:Code) * Add extended slice support to `fdisk` -* Install Coverity and fix the FreeBSD bugs that were uncovered by Coverity, but do not just blindly pull over the FreeBSD patches. Make sure that you first understand what the patch does. * C99 Standards Conformance. The todo list is on [[/docs/developer/StandardsConformanceProject]] (GCI:Code) -* Setup a regression testing machine/system to register and find problems and new improvements. (GCI:Quality Assurance) -* Networking performance / scalability ((GCI:Research) -* [[RegressionTest|/docs/developer/RegressionTest]] (GCI:Research) -* also check [[HowToStressTest|/docs/developer/HowToStressTest]] (GCI:Research) * Add lwp support to ptrace/gdb/core dumps. -* Bringing in version 2.0 of the BSD Installer * UTF8 support in the console ### Scalability (algorithmic performance) (GCI:Research) @@ -69,13 +29,9 @@ Projects that can be clearly used for Google Code-In are marked with their categ * Compile and test your changes. * Verify that the checksum (sha(1)) of the unmodified object matches the checksum of the cleaned object. Check also with strip(1)+sha(1) -### Port BSD-licensed tools (ex: `grep`, `diff` and `sort`) (GCI:Code) +### Port BSD-licensed tools (ex: `diff` and `sort`) (GCI:Code) * The OpenBSD guys already did some work related to that. * If you manage to bring the tools to DragonFly, check if everything works as expected (e.g. rc.d scripts, make world runs, ...). -* The new tools need to have at least all the features of the old GNU tools. - -### Randomize mmap() offsets -* [http://www.openbsd.org/papers/ven05-deraadt/index.html](http://www.openbsd.org/papers/ven05-deraadt/index.html) ### GDB * ptrace/gdb follow-fork-mode support and more (peek at linux) @@ -102,29 +58,30 @@ Projects that can be clearly used for Google Code-In are marked with their categ * Port this core functionality into a public libhammer library so that other base and third party utilities may take advantage of it. * See commit: cb7575e6a89409a2041a37fcfc22ce9e41297ab8 -- libHAMMER already exists, port functionality into it! -### Clean up crypt(3) consumers -* crypt(3) can legitimately return NULL on error, but many consumers do not check for this, instead passing the return directly to str[n]cmp. Fix these cases. -* Fix these cases in pkgsrc as well. +### Live images enhancement + +The live images could be made easier to use by not assuming a QWERTY keyboard layout. + +Not beeing able to choose a localized keyboard layout before having to type 'installer' or 'root' may be a deal breaker for some users. +A simple menu replacing login(1) and displaying the following three choices could be a good solution: + +1. Set keyboard layout +2. Launch the installer +3. Login as root ## Kernel projects ### One-liners * Port the BSDL OSS code to DragonFly * Complete Path MTU Discovery by adding a host route to remember the Path MTU and setting a timer to expire old host routes. See netinet/if_ether.c for an example of this mechanism as used by ARP. Periodically increase MTU of hosts that have had its MTU decreased. -* Look for places in the kernel that can benefit from Solaris-style caching of preconstructed slab allocator objects. If we can find enough of these uses, we can add this functionality to the kernel memory allocator. -* Implement [`sem_open()`](http://www.opengroup.org/onlinepubs/000095399/functions/sem_open.html), [`sem_close()`](http://www.opengroup.org/onlinepubs/000095399/functions/sem_close.html), and [`sem_unlink()`](http://www.opengroup.org/onlinepubs/000095399/functions/sem_unlink.html). (GCI:Code) -* Clean our code to make it [style(9)](http://leaf.dragonflybsd.org/cgi/web-man?command#style§ionANY) compatible. Compile and test your changes. Verify that the checksum (sha(1)) of the unmodified object matches the checksum of the cleaned object. Check also with strip(1)+sha(1) -* Setup a regression testing machine/system to register and find problems and new improvements. (GCI:Research) * Port or update drivers from other systems. -* Port NFSv4. [This mail](http://leaf.dragonflybsd.org/mailarchive/kernel/2008-01/msg00065.html) is a good starting point. -* clean up buildworld/kernel compilation warnings -* Port DTrace * Port Linux emulation to x86-64 ### Code to port/sync from FreeBSD * PCI code (to take advantage of power saving features) * Bring in support for UFS2, just the changes to extend the width of some fields from 32 bits to 64 bits. (GCI:Code) * hardware drivers. +* ext2fs ### Code to port/sync from OpenBSD * Add support for the NoExecute bit as described in [http://www.openbsd.org/papers/auug04/index.html](http://www.openbsd.org/papers/auug04/index.html). @@ -135,15 +92,9 @@ Projects that can be clearly used for Google Code-In are marked with their categ * A scheduler API supporting multiple scheduler implementations already exists * Add a Solaris-like dispatcher framework that can handle more than one installed scheduling policy -### I/O scheduler -* A pluggable kernel I/O scheduler already exists -* Implement additional disk scheduling policies - ### Implement boot cache * Effectively a smart readahead. -* Store the pattern of incoming read requests of the boot disk. (Likely into an array structure for better linear readback, We mmap() the whole thing into RAM on boot.) -* Sort pattern into a playback list. (Burstsort or a simple quick sort to get things going?) -* Provide feedback of boot cache playback list hit rate for better adaptiveness. +* Store the pattern of incoming read requests of the boot disk. ### Improve kernel boot speed * Research source of delays in boot process, keyboard init, scsi? @@ -160,7 +111,6 @@ Projects that can be clearly used for Google Code-In are marked with their categ ### Work relating to LWKT (LightWeightKernelThreading) * Implement lazy IPI cross-processor lwkt message passing. * Add timeout functionality to lwkt_waitmsg(). -* Write man pages for the lwkt message passing API. ### Filesystem extended attributes * Generic VFS attributes layer @@ -185,17 +135,12 @@ Projects that can be clearly used for Google Code-In are marked with their categ * zlib, pv entries * Once all consumers are ported, zalloc can be removed. -### Document all sysctl's -* The description of sysctl's can be provided when the sysctl is declared and displayed in userland by passing the -d flag to the sysctl utility. -* Document all undocumented sysctl's -* Verify that the description of documented sysctl's is correct. - ### Convert kprintf-enabling sysctl's to KTR's * Many sysctl's simply enable one or more kernel kprintf's * This can be very easy, or can be very unwieldy * Convert all of these cases to ktr's, while slightly less easy they are far easier to wield in all cases. -### Tear out C/H/S disk reporting (GCI:Research) +### Tear out C/H/S disk reporting * Cylinders/Heads/Sectors are an outdated concept and the system doesn't rely on them anymore. * Verify the assumption that we don't rely on them in any way, shape or form. * Tear the reporting out of the kernel/installer/etc. @@ -212,17 +157,16 @@ Projects that can be clearly used for Google Code-In are marked with their categ * A device that uses physical memory as swap space, but compresses it. * Do we support stacking of swap space? For example, one would have this compressed in-memory swap device with highest priority. Replaced objects will be put into the next priority swap device (e.g. a SSD), and so on. -### tmpfs allocations from swap -* Currently, tmpfs nodes and stuff are allocated from KVA are the size limiter for a tmpfs filesystem -* Instead allocate them from swappable memory; this will allow larger tmpfses up to swap limits - ### mmap MAP_ALIGN * Solaris's mmap support a flag, MAP_ALIGN, where the address to mmap acts as an alignment hint * Our backing VM calls support an alignment parameter, but our public mmap does not * This would allow nmalloc to allocate slabs (64k, 64k-aligned) without wastage ### vnode dumps -* It would be beneficial to be able to have crash dumps written to an ordinary file in configurations where swap is not configured. +* It may be beneficial to be able to have crash dumps written to an ordinary file in configurations where swap is not configured. + +### vnode swap +* Add a vnode-backed swap pager that respects a file size limit, to allow paging to a "swap file" on a filesystem. ### Kernel allocator feature enhancements * Enhance objcache in a manner that will allow the initialization to specify allocation functions which will allocate and free entire slabs of memory, not just single objects. @@ -232,24 +176,113 @@ Projects that can be clearly used for Google Code-In are marked with their categ * Add reclaim functionality to objcache, such that it may ask consumers to free objects back into it under memory pressure. ### Tear out serializers -* Serializers could be _carefully_ replaced with MTX locks +* Serializers could be _carefully_ replaced with MTX locks? ### Tear out condvars * Conditional vars -- condvar(9), could be replaced with other locking primitives and our tsleep/wakeup interlock. -### Make karc4random in libkern per-cpu -* Verify that it is possible and safe to do this, what care would need to be taken, especially with respect to the random seeding? -* Pull out locks around calls to karc4rand* +### Partially rewrite buffer cache +* Buffer cache buffers map VM pages from the filesystems VM-backed vnodes, the "buffer space" is the maximum amount of virtual space to allocate to these buffers. +* When the buffer_map KVA space gets fragmented it caused very expensive defrag operations, the buffer_map KVA was recently increased to double the actual buffer space to allow full space utilization in the face of fragmentation and reduce the frequency of defrag operations. +* Rewrite the buffer cache to have separate spaces or separate buffer pools for different sizes (16, 32, 64, 128, 256... up to max). +* The vm_map API is used to allocate kvm out of the buffer_map. If the buffer's KVAs are preallocated then things can basically just be setup linearly at boot time. + +### Sync wireless infrastructure / drivers from FreeBSD +* Rui Paulo ported the wireless infrastructure and ath drivers to DragonFly from FreeBSD in recent history. A lot has changed in the intervening time, the infrastructure should be synchronized/updated to match the state-of-the-art in FreeBSD. At a minimum the drivers that currently work in DragonFly should be tested to ensure functionality and updates as well or improved. + +### Add informational hardware-related sysctl's +* MacOS X exports a bunch of cpu-specific sysctl's detailing sizes, layout, features, etc. Most/all of this could be detected by a userspace program, but exporting these makes the barrier of entry to cpu-conditional code lower. +* Research which of these sysctl's is the most useful and add them, using the same node names as OSX. +1. hw.vectorunit = 1 +1. hw.busfrequency = 100000000 +1. hw.cpufrequency = 3062000000 +1. hw.cachelinesize = 64 +1. hw.l1icachesize = 32768 +1. hw.l1dcachesize = 32768 +1. hw.l2settings = 1 +1. hw.l2cachesize = 262144 +1. hw.l3settings = 1 +1. hw.l3cachesize = 6291456 +1. hw.tbfrequency = 1000000000 +1. hw.activecpu: 2 +1. hw.physicalcpu: 2 +1. hw.physicalcpu_max: 2 +1. hw.logicalcpu: 2 +1. hw.logicalcpu_max: 2 +1. hw.cputype: 7 +1. hw.cpusubtype: 4 +1. hw.cpu64bit_capable: 1 +1. hw.cpufamily: 1418770316 +1. hw.cacheconfig: 2 1 1 1 0 0 0 0 0 0 +1. hw.cachesize: 2147483648 32768 262144 6291456 0 0 0 0 0 0 +1. hw.busfrequency: 100000000 +1. hw.busfrequency_min: 100000000 +1. hw.busfrequency_max: 100000000 +1. hw.cpufrequency: 3062000000 +1. hw.cpufrequency_min: 3062000000 +1. hw.cpufrequency_max: 3062000000 +1. hw.cachelinesize: 64 +1. hw.l1icachesize: 32768 +1. hw.l1dcachesize: 32768 +1. hw.l2cachesize: 262144 +1. hw.l3cachesize: 6291456 +1. hw.tbfrequency: 1000000000 +1. hw.packages: 2 +1. hw.optional.floatingpoint: 1 +1. hw.optional.mmx: 1 +1. hw.optional.sse: 1 +1. hw.optional.sse2: 1 +1. hw.optional.sse3: 1 +1. hw.optional.supplementalsse3: 1 +1. hw.optional.sse4_1: 1 +1. hw.optional.sse4_2: 1 +1. hw.optional.x86_64: 1 +1. hw.optional.aes: 1 +1. hw.optional.avx1_0: 1 +1. hw.optional.rdrand: 0 +1. hw.optional.f16c: 0 +1. hw.optional.enfstrg: 0 +1. machdep.cpu.max_basic: 13 +1. machdep.cpu.max_ext: 2147483656 +1. machdep.cpu.vendor: GenuineIntel +1. machdep.cpu.brand_string: Intel(R) Core(TM) i5-3470 CPU @ 3.20GHz +1. machdep.cpu.family: 6 +1. machdep.cpu.model: 42 +1. machdep.cpu.extmodel: 2 +1. machdep.cpu.extfamily: 0 +1. machdep.cpu.stepping: 7 +1. machdep.cpu.feature_bits: 262929407 2660770315 +1. machdep.cpu.extfeature_bits: 672139520 1 +1. machdep.cpu.signature: 132775 +1. machdep.cpu.brand: 0 +1. machdep.cpu.features: FPU VME DE PSE TSC MSR PAE MCE CX8 APIC SEP MTRR PGE MCA CMOV PAT PSE36 CLFSH DS MMX FXSR SSE SSE2 SS SSE3 PCLMULQDQ MON SSSE3 CX16 SSE4.1 SSE4.2 POPCNT AES VMM XSAVE OSXSAVE AVX1.0 +1. machdep.cpu.extfeatures: SYSCALL XD EM64T LAHF RDTSCP TSCI +1. machdep.cpu.cores_per_package: 1 +1. machdep.cpu.microcode_version: 16 +1. machdep.cpu.processor_flag: 0 +1. machdep.cpu.mwait.linesize_min: 4096 +1. machdep.cpu.mwait.linesize_max: 4096 +1. machdep.cpu.mwait.extensions: 3 +1. machdep.cpu.mwait.sub_Cstates: 4384 +1. machdep.cpu.xsave.extended_state: 7 832 832 0 +1. machdep.cpu.arch_perf.version: 1 +1. machdep.cpu.arch_perf.number: 8 +1. machdep.cpu.arch_perf.width: 48 +1. machdep.cpu.arch_perf.events_number: 7 +1. machdep.cpu.arch_perf.events: 127 +1. machdep.cpu.arch_perf.fixed_number: 0 +1. machdep.cpu.arch_perf.fixed_width: 0 +1. machdep.cpu.cache.linesize: 64 +1. machdep.cpu.cache.L2_associativity: 8 +1. machdep.cpu.cache.size: 256 +1. machdep.cpu.tlb.inst.small: 128 +1. machdep.cpu.tlb.data.small: 64 +1. machdep.cpu.tlb.data.large: 32 +1. machdep.cpu.tlb.shared: 512 +1. machdep.cpu.address_bits.physical: 40 +1. machdep.cpu.address_bits.virtual: 48 +1. machdep.cpu.core_count: 1 +1. machdep.cpu.thread_count: 1 -## Live images enhancement - -The live images could be made easier to use by not assuming a QWERTY keyboard layout. - -Not beeing able to choose a localized keyboard layout before having to type 'installer' or 'root' may be a deal breaker for some users. -A simple menu replacing login(1) and displaying the following three choices could be a good solution: - -1. Set keyboard layout -2. Launch the installer -3. Login as root For more theoretical projects and project concepts see [[ResearchProjects|/docs/developer/ResearchProjectsPage]]