kernel - rewrite the LWKT scheduler's priority mechanism
authorMatthew Dillon <dillon@apollo.backplane.com>
Tue, 24 Aug 2010 01:39:45 +0000 (18:39 -0700)
committerMatthew Dillon <dillon@apollo.backplane.com>
Tue, 24 Aug 2010 01:49:53 +0000 (18:49 -0700)
commitf9235b6d9cd4b6ef2a6f977a1e659de0ac635e32
treee2d2484c3b9627403e1bad83b60734c32abc017a
parentb28ad496a7a00d768ba61347d2878fe7c92e5d58
kernel - rewrite the LWKT scheduler's priority mechanism

The purpose of these changes is to begin to address the issue of cpu-bound
kernel threads.  For example, the crypto threads, or a HAMMER prune cycle
that operates entirely out of the buffer cache.  These threads tend to hicup
the system, creating temporary lockups because they never switch away due
to their nature as kernel threads.

* Change the LWKT scheduler from a strict hard priority model to
  a fair-share with hard priority queueing model.

  A kernel thread will be queued with a hard priority, giving it dibs on
  the cpu earlier if it has a higher priority.  However, if the thread
  runs past its fair-share quantum it will then become limited by that
  quantum and other lower-priority threads will be allowed to run.

* Rewrite lwkt_yield() and lwkt_user_yield(), remove uio_yield().
  Both yield functions are now very fast and can be called without
  further timing conditionals, simplifying numerous callers.

  lwkt_user_yield() now uses the fair-share quantum to determine when
  to yield the cpu for a cpu-bound kernel thread.

* Implement the new yield in the crypto kernel threads, HAMMER, and
  other places (many of which already used the old yield functions
  which didn't work very well).

* lwkt_switch() now only round-robins after the fair share
  quantum is exhausted.  It does not necessarily always round robin.

* Separate the critical section count from td_pri.  Add td_critcount.
53 files changed:
sys/ddb/db_ps.c
sys/kern/kern_clock.c
sys/kern/kern_intr.c
sys/kern/kern_kinfo.c
sys/kern/kern_subr.c
sys/kern/kern_synch.c
sys/kern/kern_threads.c
sys/kern/kern_time.c
sys/kern/lwkt_ipiq.c
sys/kern/lwkt_thread.c
sys/kern/lwkt_token.c
sys/kern/usched_bsd4.c
sys/kern/vfs_vnops.c
sys/opencrypto/crypto.c
sys/platform/pc32/apic/apic_vector.s
sys/platform/pc32/i386/bcopy.s
sys/platform/pc32/i386/exception.s
sys/platform/pc32/i386/genassym.c
sys/platform/pc32/i386/machdep.c
sys/platform/pc32/i386/swtch.s
sys/platform/pc32/i386/trap.c
sys/platform/pc32/icu/icu_vector.s
sys/platform/pc32/isa/ipl.s
sys/platform/pc64/apic/apic_vector.s
sys/platform/pc64/icu/icu_vector.s
sys/platform/pc64/x86_64/exception.S
sys/platform/pc64/x86_64/genassym.c
sys/platform/pc64/x86_64/ipl.s
sys/platform/pc64/x86_64/machdep.c
sys/platform/pc64/x86_64/swtch.s
sys/platform/pc64/x86_64/trap.c
sys/platform/vkernel/i386/cpu_regs.c
sys/platform/vkernel/i386/exception.c
sys/platform/vkernel/i386/fork_tramp.s
sys/platform/vkernel/i386/genassym.c
sys/platform/vkernel/i386/swtch.s
sys/platform/vkernel/i386/trap.c
sys/platform/vkernel/platform/machintr.c
sys/platform/vkernel64/platform/machintr.c
sys/platform/vkernel64/x86_64/cpu_regs.c
sys/platform/vkernel64/x86_64/exception.c
sys/platform/vkernel64/x86_64/fork_tramp.s
sys/platform/vkernel64/x86_64/genassym.c
sys/platform/vkernel64/x86_64/swtch.s
sys/platform/vkernel64/x86_64/trap.c
sys/sys/globaldata.h
sys/sys/thread.h
sys/sys/thread2.h
sys/sys/uio.h
sys/sys/upcall.h
sys/vfs/hammer/hammer_flusher.c
sys/vfs/ufs/ffs_rawread.c
sys/vm/vm_zeroidle.c