Make all network interrupt service routines MPSAFE part 1/3.
Replace the critical section that was previously used to serialize access
with the LWKT serializer. Integrate the serializer into the IFNET structure.
Note that kern.intr_mpsafe must be set to 1 for network interrupts to actually
run MPSAFE. Also note that any interrupts shared with othre non-MP drivers
will cause all drivers on that interrupt to run with the Big Giant Lock.
Network interrupt - Each network driver then simply passes that serializer
to bus_setup_intr() so only a single serializer is required to process the
entire interrupt path. LWKT serialization support is already 100% integrated
into the interrupt subsystem so it will already be held as of when the
registered interrupt procedure is called.
Ioctl and if_* functions - All callers of if_* functions (such as if_start,
if_ioctl, etc) now obtain the IFNET serializer before making the call. Thus
all of these entry points into the driver will now be serialized.
if_input - All code that calls if_input now ensures that the serializer
is held. It will either already be held (when called from a driver), or
the serializer will be wrapped around the call. When packets are forwarded
or bridged between interfaces, the target interface serializer will be
dropped temporarily to avoid a deadlock.
Device Driver access - dev_* entry points into certain pseudo-network
devices now obtain and release the serializer. This had to be done on
a device-by-device basis (but there are only a few such devices).
Thanks to several people for helping test the patch, in particular
Sepherosa Ziehau.