hammer2 - live dedup, cleanup
authorMatthew Dillon <dillon@apollo.backplane.com>
Sun, 30 Aug 2015 03:02:14 +0000 (20:02 -0700)
committerMatthew Dillon <dillon@apollo.backplane.com>
Sun, 30 Aug 2015 06:39:37 +0000 (23:39 -0700)
commit3f01ebaaf512c09685f0f7b8bfec3b47bbd1e72a
treeaad2874d9410c6c0e878a94e1f23c215537e21ad
parentaa6ac96e01825b3efcab953441f85adbf9815e0f
hammer2 - live dedup, cleanup

* First attempt at a live dedup.  The H2 strategy code now caches
  {data_off, crc} info to track recently accessed data blocks.  The
  cache is checked in the strategy_write code after device-level
  block encoding.  If we get a cache hit, the disk block is compared
  against the write data and reused if it matches.

* This 'live' dedup should catch most typical 'cp' or 'cpdup' style
  commands.  There will also be a bulk dedup capable of catching
  everything.

* Note that 'df' output might be a bit confusing because the 'Used'
  field represents the topology and does not take into account dedups.
  'Avail' is calculated from the actual freemap.  To make things look
  right the total disk size is adjusted upward so it matches
  Used+Avail.  This mechanism will likely change.

  Here is an example with one copy of /usr/src and 13 copies of /usr/src.
  The first copy eats around 872MB, and a 'du' will show each copy eating
  about the same.  But because of dedup each subsequent copy actually only
  eats around 160MB as you can see from the 'Avail' field:

    test40# df -h /mnt
    Filesystem                             Size   Used  Avail Capacity
    /dev/serno/WD-WX51A82J2299.s1f@LOCAL    99G   934M    99G     1%
    Filesystem                             Size   Used  Avail Capacity
    /dev/serno/WD-WX51A82J2299.s1f@LOCAL   106G   8.5G    97G     8%

* Rename hammer2_bulkscan.c to hammer2_bulkfree.c since that is
  basically all it does.

* Move the synchronization code to its own file, hammer2_synchro.c.
  (note: This code is currently in rip-up mode and will not operate
  properly).
15 files changed:
sys/vfs/hammer2/Makefile
sys/vfs/hammer2/hammer2.h
sys/vfs/hammer2/hammer2_bulkfree.c [moved from sys/vfs/hammer2/hammer2_bulkscan.c with 99% similarity]
sys/vfs/hammer2/hammer2_chain.c
sys/vfs/hammer2/hammer2_flush.c
sys/vfs/hammer2/hammer2_freemap.c
sys/vfs/hammer2/hammer2_inode.c
sys/vfs/hammer2/hammer2_io.c
sys/vfs/hammer2/hammer2_ioctl.c
sys/vfs/hammer2/hammer2_strategy.c
sys/vfs/hammer2/hammer2_synchro.c [copied from sys/vfs/hammer2/hammer2_thread.c with 50% similarity]
sys/vfs/hammer2/hammer2_thread.c
sys/vfs/hammer2/hammer2_vfsops.c
sys/vfs/hammer2/hammer2_vnops.c
sys/vfs/hammer2/hammer2_xops.c