kernel -- ffs: Replace softupdates critical section locks with lockmgr.
ffs softupdates was using a combination of critical sections and the mplock
to construct its acquire_lock/free_lock/interlocked_sleep primitives.
The softupdates I/O completion callback had a few points where it could block,
causing it to lose the mplock and any critical sections it held. When it did
so, front-end softupdates code would try to grab the lock but panic on seeing
the I/O completion callback in progress. This was not a problem in older
systems, as splx() would prevent the I/O callback from starting while
softdep code was executing.
This patch converts to using hard locks (lockmgr), which are held even while
a thread is blocked.
getdirtybuf(), locking a dirty buffer via BUF_LOCK, was changed to not
deadlock against the I/O completion path; specifically it drops the softdep
lock when its non-blocking attempt to lock a dirty buffer fails. One caller
of getdirtybuf() also required changes to retry locking buffers.
Closes-bug: 2291
Partially-from: FreeBSD r140709
Reported-by: tuxillo, marino, Rumko, vsrinivas
Discussed-with: dillon