kernel - Break up scheduler and loadavg callout
* Change the scheduler and loadavg callouts from cpu 0 to all cpus, and
adjust the allproc_scan() and alllwp_scan() to segment the hash table
when asked.
Every cpu is now tasked with handling the nominal scheduler recalc and
nominal load calculation for a portion of the process list. The portion
is unrelated to which cpu(s) the processes are actually scheduled on,
it is strictly a way to spread the work around, split up by hash range.
* Significantly reduces cpu 0 stalls when a large number of user processes
or threads are present (that is, in the tens of thousands or more). In
the test below, before this change, cpu 0 was straining under 40%+
interupt load (from the callout). After this change the load is spread
across all cpus, approximately 1.5% per cpu.
* Tested with 400,000 running user processes on a 32-thread dual-socket
xeon (yes, these numbers are real):
12:27PM up 8 mins, 3 users, load avg: 395143.28, 270541.13, 132638.33
12:33PM up 14 mins, 3 users, load avg: 399496.57, 361405.54, 225669.14
* NOTE: There are still a number of other non-segmented allproc scans in
the system, particularly related to paging and swapping.
* NOTE: Further spreading-out of the work may be needed, by using a more
frequent callout and smaller hash index range for each.
16 files changed: