kernel - Refactor smp collision statistics * Add an indefinite wait timing API (sys/indefinite.h, sys/indefinite2.h). This interface uses the TSC and will record lock latencies to our pcpu stats in microseconds. The systat -pv 1 display shows this under smpcoll. Note that latencies generated by tokens, lockmgr, and mutex locks do not necessarily reflect actual lost cpu time as the kernel will schedule other threads while those are blocked, if other threads are available. * Formalize TSC operations more, supply a type (tsc_uclock_t and tsc_sclock_t). * Reinstrument lockmgr, mutex, token, and spinlocks to use the new indefinite timing interface.
kernel - Refactor kern_mutex (mtx* functions) * Refactor kern_mutex in order to support asynchronous lock requests, which hammer2 is going to need. kern_mutex already supports abortable locks. * Add callback fields to the mtx_link structure. * Use the mtx_link structure for shared locks in addition to exclusive locks, allowing asynchronous callbacks for shared locks and exclusive locks. * Make the locking flags more deterministic. * Redo the typedefs to be more like hammer2. Typedef the structures rather than pointers so the typedef names can be used for structural embedding.
Remove useless belt and suspenders include guards in some of our headers. For these headers: /usr/include/machine/atomic.h /usr/include/machine/bus_dma.h /usr/include/machine/coredump.h /usr/include/machine/cpufunc.h /usr/include/machine/db_machdep.h /usr/include/machine/elf.h /usr/include/machine/endian.h /usr/include/machine/frame.h /usr/include/machine/limits.h /usr/include/machine/npx.h /usr/include/machine/profile.h /usr/include/machine/segments.h /usr/include/machine/stdarg.h /usr/include/machine/stdint.h /usr/include/machine/trap.h /usr/include/machine/tss.h /usr/include/machine/ucontext.h /usr/include/machine/vframe.h /usr/include/machine/vm86.h All these headers #define _CPU_... and not _MACHINE_... even though they are in /usr/include/machine. And the headers themselves have include guards already. So there's little point in having them around the actual #include additionally.
kernel - revamp mtx_spinlock() * Revamp mtx_spinlock(), add mtx_spinlock_try(), and add mtx_spinunlock(). * Enter a proper hard critical section when using mtx_spinlock*(), just like the normal spinlock() code does. The difference is that mtx spinlocks have a ref count and thus are reentrant. * mtx_spinlock*() is not used in the code yet. A followup commit will begin using it for the syscons lock.
MPSAFE - mutex - better exclusive lock sequencer, bug fixes, abort * Redo the exclusive lock chaining algorithm. Use an explicit link structure and directly pass ownership to the next thread waiting on an exclusive lock. * Exclusive locks can be aborted via mtx_lock_ex_link() and mtx_abort_ex_link(). * Lots of misc bug fixes.
MPSAFE - Add a set of general blocking/spinnable mutex functions. These locks are intended to eventually replace lockmgr locks for most use cases. * Optimized based on in-line atomic_cmpset_int() calls with fallback call. * Recursive shared and exclusive locks. Downgrading, and non-blocking upgrading. * Interlocked wakeup flags. * Serial wakeup for exclusive waiters (i.e. optimal in the face of a large number of waiting threads). Mass-wakeup for shared waiters. * Additional entry points for spinning. * Ref-count support, separate from lock count.