kernel - Add MONITOR/MWAIT support to the LWKT scheduler
* Adjust the FIFO contention resequencer (which deals with spinning
on tokens) to use MONITOR/MWAIT when available instead of DELAY(1)
when waiting to become the head of the queue.
* Adjust the x86-64 idle loop to use MONITOR/MWAIT when available when
the idle halt mode (machdep.cpu_idle_hlt) is set to 1. This
significantly improves performance for event-oriented programs, including
compile pipelines.
NOTE: On the 48-core monster setting machdep.cpu_idle_hlt to 1 improves
performance but at the cost of an additional 200W of power at idle vs
the default value of 2 (ACPI idle halt). Look for a hybrid approach in
a future commit.