kernel - Improve physio performance
authorMatthew Dillon <dillon@apollo.backplane.com>
Sun, 17 Jul 2016 06:15:19 +0000 (23:15 -0700)
committerMatthew Dillon <dillon@apollo.backplane.com>
Sun, 17 Jul 2016 06:27:24 +0000 (23:27 -0700)
commit2f0acc22b5ce1dd94f2d01c149a7b1bf0d8eb707
tree8b97226b3876a7ccb1868e1dd04ce90d04fdad2f
parentccd67bf614ec9cd72ba3afc5d73e88ba8be3bc70
kernel - Improve physio performance

* See http://apollo.backplane.com/DFlyMisc/nvme_sys03.txt

* Hash the pbuf system.  This chops down spin-lock collisions
  at high transaction rates (>150K IOPS) by 1000x.

* Implement a pbuf with pre-allocated kernel memory that we
  copy into, avoiding page table manipulations and thus
  avoiding system-wide invltlb/invlpg IPIs.

* This increases NVMe IOPS tests with three cards from
  150K-200K IOPS to 950K IOPS using physio (random read,
  4K blocks, from urandom-filled partition, with many
  process threads, from 3 NVMe cards in parallel).

* Further adjustments to the vkernel build.
14 files changed:
sys/dev/raid/vinum/vinum.c
sys/kern/kern_physio.c
sys/kern/subr_param.c
sys/platform/pc64/x86_64/machdep.c
sys/platform/vkernel64/include/pmap.h
sys/platform/vkernel64/platform/pmap.c
sys/platform/vkernel64/x86_64/autoconf.c
sys/sys/buf.h
sys/vfs/nfs/nfs_subs.c
sys/vfs/smbfs/smbfs_vfsops.c
sys/vfs/ufs/ffs_rawread.c
sys/vm/swap_pager.c
sys/vm/vm_pager.c
sys/vm/vnode_pager.c