MPSAFE - tsleep_interlock, BUF/BIO, cluster, swap_pager.
authorMatthew Dillon <dillon@apollo.backplane.com>
Wed, 15 Jul 2009 02:31:18 +0000 (19:31 -0700)
committerMatthew Dillon <dillon@apollo.backplane.com>
Wed, 15 Jul 2009 02:31:18 +0000 (19:31 -0700)
commitae8e83e6e95ab527d7528e1f38a1f95aa251908e
treeb9164ba6d2e757b7e37c9335d657478a9c1081bc
parent2e7d1bab46da7843e5cdfae644b9a44fde77a8a4
MPSAFE - tsleep_interlock, BUF/BIO, cluster, swap_pager.

* tsleep_interlock()/tsleep() could miss wakeups during periods of
  heavy cpu activity.  What would happen is code inbetween the two
  calls would try to send an IPI (say, issue a wakeup()), but while
  sending the IPI the kernel would be forced to process incoming IPIs
  synchronous to avoid a deadlock.

  The new tsleep_interlock()/tsleep() code adds another TAILQ_ENTRY to
  the thread structure allowing tsleep_interlock() to formally place
  the thread on the appropriate sleep queue without having to deschedule
  the thread.  Any wakeup which occurs between the interlock and the
  real tsleep() call will remove the thread from the queue and the
  later tsleep() call will recognize this and simply return without sleeping.

  The new tsleep() call requires PINTERLOCKED to be passed to tsleep
  so tsleep() knows that the thread has already been placed on a sleep
  queue.

* Continue making BUF/BIO MPSAFE.  Remove B_ASYNC and B_WANT from buf->b_flag
  and add a new bio->bio_flags field to the bio.  Add BIO_SYNC, BIO_WANT,
  and BIO_DONE.  Use atomic_cmpset_int() (aka cmpxchg) to interlock
  biodone() against biowait().

  vn_strategy() and dev_dstrategy() call semantics now require that
  synchronous BIO's install a bio_done function and set BIO_SYNC in
  the bio.

* Clean up the cluster code a bit.

* Redo the swap_pager code.  Instead of issuing I/O during the collection,
  which depended on critical sections to avoid races in the cluster append,
  we now build the entire collection first and then dispatch the I/O.
  This allows us to use only async completion for the BIOs, instead of
  a hybrid sync-or-async completion.
65 files changed:
sys/bus/cam/cam_sim.c
sys/bus/cam/cam_xpt.c
sys/dev/disk/ahci/ahci_dragonfly.c
sys/dev/disk/aic7xxx/aic_osm_lib.c
sys/dev/disk/ata/ata-raid.c
sys/dev/disk/fd/fd.c
sys/dev/disk/nata/ata-raid.c
sys/dev/disk/sili/sili_dragonfly.c
sys/dev/disk/vn/vn.c
sys/dev/drm/drmP.h
sys/dev/drm/drm_drv.c
sys/dev/drm/drm_lock.c
sys/dev/drm/radeon_cp.c
sys/dev/netif/iwi/if_iwi.c
sys/dev/raid/aac/aac.c
sys/dev/raid/vinum/vinuminterrupt.c
sys/dev/raid/vinum/vinumio.c
sys/dev/raid/vinum/vinumrequest.c
sys/dev/raid/vinum/vinumrevive.c
sys/dev/sound/pcm/sound.c
sys/kern/kern_device.c
sys/kern/kern_lock.c
sys/kern/kern_physio.c
sys/kern/kern_synch.c
sys/kern/kern_umtx.c
sys/kern/lwkt_ipiq.c
sys/kern/lwkt_serialize.c
sys/kern/lwkt_thread.c
sys/kern/subr_bus.c
sys/kern/subr_diskgpt.c
sys/kern/subr_disklabel32.c
sys/kern/subr_disklabel64.c
sys/kern/subr_diskmbr.c
sys/kern/sys_pipe.c
sys/kern/vfs_aio.c
sys/kern/vfs_bio.c
sys/kern/vfs_cluster.c
sys/kern/vfs_subr.c
sys/kern/vfs_vnops.c
sys/net/tap/if_tap.c
sys/netproto/smb/smb_subr.c
sys/sys/bio.h
sys/sys/buf.h
sys/sys/buf2.h
sys/sys/systm.h
sys/sys/thread.h
sys/vfs/gnu/ext2fs/ext2_bmap.c
sys/vfs/gnu/ext2fs/ext2_inode.c
sys/vfs/hammer/hammer_io.c
sys/vfs/hammer/hammer_subs.c
sys/vfs/mfs/mfs_vnops.c
sys/vfs/nfs/nfs_bio.c
sys/vfs/nfs/nfs_serv.c
sys/vfs/nfs/nfs_syscalls.c
sys/vfs/nfs/nfs_vnops.c
sys/vfs/nwfs/nwfs_io.c
sys/vfs/nwfs/nwfs_vnops.c
sys/vfs/smbfs/smbfs_io.c
sys/vfs/smbfs/smbfs_vnops.c
sys/vfs/specfs/spec_vnops.c
sys/vfs/ufs/ffs_balloc.c
sys/vfs/ufs/ffs_inode.c
sys/vfs/ufs/ffs_rawread.c
sys/vfs/ufs/ufs_bmap.c
sys/vm/swap_pager.c