x86_64: Implement x2apic support.
Now LAPIC registers are accessed through MSR at fixed location, instead
of going through MMIO region.
Most noticeable is that ICR operation is greatly simplified, i.e. IPI
sending operation:
- Reserved bits are read as 0; there is no need to read ICR first for
OR with the new values.
- No more pending bit, i.e. ICR write is synchronized; there is no need
to read ICR to test pending bit.
- ICR is 64 bits in x2apic mode, i.e. two 32 bits writes to ICR-low and
ICR-high become one write to ICR.
NOTE:
Though Intel SDM says that wrmsr to LAPIC registers are relaxed, we
don't need to put mfence or sfence before them, especially for sending
IPIs, since the generic IPIQ and the machdep code already uses atomic
operation before doing ICR operation. For the rest of the code, there
really are no needs to add mfence/sfence before rdmsr/wrmsr to LAPIC
registers.
As of this commit, x2apic mode is _not_ enabled by default. It can be
enabled through hw.x2apic_enable tuneable, and a read-only sysctl node
with the same name is available for debugging purpose.
Based on work by ivadasz@.