x86_64: Implement x2apic support. Now LAPIC registers are accessed through MSR at fixed location, instead of going through MMIO region. Most noticeable is that ICR operation is greatly simplified, i.e. IPI sending operation: - Reserved bits are read as 0; there is no need to read ICR first for OR with the new values. - No more pending bit, i.e. ICR write is synchronized; there is no need to read ICR to test pending bit. - ICR is 64 bits in x2apic mode, i.e. two 32 bits writes to ICR-low and ICR-high become one write to ICR. NOTE: Though Intel SDM says that wrmsr to LAPIC registers are relaxed, we don't need to put mfence or sfence before them, especially for sending IPIs, since the generic IPIQ and the machdep code already uses atomic operation before doing ICR operation. For the rest of the code, there really are no needs to add mfence/sfence before rdmsr/wrmsr to LAPIC registers. As of this commit, x2apic mode is _not_ enabled by default. It can be enabled through hw.x2apic_enable tuneable, and a read-only sysctl node with the same name is available for debugging purpose. Based on work by ivadasz@.
kernel - Enhance the sniff code, refactor interrupt disablement for IPIs * Add kern.sniff_enable, default to 1. Allows the sysop to disable the feature if desired. * Add kern.sniff_target, allows sniff IPIs to be targetted to all cpus (-1), or to a particular cpu (0...N). This feature allows the sysop to test IPI delivery to particular CPUs (typically monitoring with systat -pv 0.1) to determine that delivery is working properly. * Bring in some additional AMD-specific setup from FreeBSD, beginnings of support for the APIC Extended space. For now just make sure the extended entries are masked. * Change interrupt disablement expectations. The caller of apic_ipi(), selected_apic_ipi(), and related macros is now required to hard-disable interrupts rather than these functions doing so. This allows the caller to run certain operational sequences atomically. * Use the TSC to detect IPI send stalls instead of a hard-coded loop count. * Also set the APIC_LEVEL_ASSERT bit when issuing a directed IPI, though the spec says this is unnecessary. Do it anyway. * Remove unnecessary critical section in selected_apic_ipi(). We are in a hard-disablement and in particular we do not want to accidently trigger a splz() due to the crit_exit() while in the hard-disablement. * Enhance the IPI stall detection and recovery code. Provide more inforamtion. Also enable the LOOPMASK_IN debugging tracker by default. * Add a testing feature to machdep.all_but_self_ipi_enable. By setting this to 2, we force the smp_invltlb() to always use the ALL_BUT_SELF IPI. For testing only.
kernel64 - Add some lapic timer based debugging (disabled by default) * Adjust the Xtimer interrupt to unconditionally call lapic_timer_always() regardless of the critical section state. * Create a procedure lapic_timer_always() on x86-64 which is #if 0'd out by default with some on-screen debugging that can be used to help find system lockups.
Bring in sephe's recent intr/apic work to x86_64. The following commits: 6ab7c3a intr: Clean up comment of Local APIC TPR 84bf7d5 intr: Remove unused TRP macros e4a4c49 intr: Remove unused typedef 507bf37 intr: Make sure that changing IPI offset will also update related TPR 072d9b3 intr: Move IO/APIC IDT vector offset into isa/intr_machdep.h 7b32a0a apic: Remove unused macros and duplicated comment 2ce3ae8 apic: Remove unused macros 26408d7 intr: Remove call_fast_unpend() declaration 87e3ca2 intr: Start hardware interrupt at IDT_OFFSET 90a567f apic: Clear all entries in int table
kernel - SMP - "Fix AP #%d (PHY# %d) failed" issues Ok, here's what is going on. If an SMI interrupt occurs while an AP is going through the INIT/STARTUP IPI sequence the AP will brick, and nothing you do will resurrect it. BIOSes typically set up SMI interrupts when emulating (for example) a PS/2 keyboard with a USB keyboard, or even if just implementing BIOS support for a USB keyboard. Even worse, the BIOS may set up the interrupt to poll at 1000hz. And, EVEN WORSE, it can totally depend on which USB port you've plugged your keyboard in. And, on top of all of that, the SMI interrupt is not consistent. The INIT/STARTUP code contains a 10ms delay (as per Intel spec) between the INIT IPI and the STARTUP IPI. Well, you can do the math. In order to reliably boot a SMP system where the BIOS has set up SMI interrupts this patch uses a nifty bit of code to detect when the SMI interrupt has occurred and tries to shift the INIT/STARTUP sequence into a gap between SMI interrupts. If it has to it will reduce the 10ms spec delay all the way down to 150us. In many cases we really have no choice for reliable operation. Even a 300uS delay is too much in the tests I performed on a Shuttle Phenom and Phenom II cube. I don't honestly know if this will break other SMP configurations, we'll have to see. On the particular shuttle I tested on, one of the four USB connections on the backpanel (the upper left when looking at it from the back) seemed to cause the BIOS to set up SMI interrupts at a high rate and caused kernel boots to fail. With this commit those boots now succeed.
AMD64 - Sync AMD64 support from Jordan Gordeev's svn repository and Google SOC project. This work is still continuing but represents substantial progress in the effort. With this commit the world builds and installs, the loader is able to boot the kernel, and the kernel is able to initialize, probe devices, and exec the init program. The init program is then able to run until it hits its first fork(). For the purposes of the GSOC the project is being considered a big success! The code has been adapted from multiple sources, most notably Peter Wemm and other peoples work from FreeBSD, with many modifications to make it work with DragonFly. Also thanks go to Simon Schubert for working on gdb and compiler issues, and to Noah Yan for a good chunk of precursor work in 2007. While Jordan wishes to be modest on his contribution, frankly we would not have been able to make this much progress without the large number of man-hours Jordan has dedicated to his GSOC project painstakingly gluing code together, tracking down issues, and progressing the boot sequence. Submitted-by: Jordan Gordeev <jgordeev@dir.bg>