1 .\" Copyright (c) 2007 The DragonFly Project. All rights reserved.
3 .\" This code is derived from software contributed to The DragonFly Project
4 .\" by Matthew Dillon <dillon@backplane.com>
6 .\" Redistribution and use in source and binary forms, with or without
7 .\" modification, are permitted provided that the following conditions
10 .\" 1. Redistributions of source code must retain the above copyright
11 .\" notice, this list of conditions and the following disclaimer.
12 .\" 2. Redistributions in binary form must reproduce the above copyright
13 .\" notice, this list of conditions and the following disclaimer in
14 .\" the documentation and/or other materials provided with the
16 .\" 3. Neither the name of The DragonFly Project nor the names of its
17 .\" contributors may be used to endorse or promote products derived
18 .\" from this software without specific, prior written permission.
20 .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
21 .\" ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
22 .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
23 .\" FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
24 .\" COPYRIGHT HOLDERS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
25 .\" INCIDENTAL, SPECIAL, EXEMPLARY OR CONSEQUENTIAL DAMAGES (INCLUDING,
26 .\" BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
27 .\" LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED
28 .\" AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
29 .\" OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
30 .\" OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
33 .\" $DragonFly: src/sbin/hammer/hammer.8,v 1.58 2008/11/13 02:04:27 dillon Exp $
40 .Nd HAMMER file system utility
46 .Op Fl f Ar blkdev[:blkdev]*
47 .\" .Op Fl s Ar linkpath
50 .Op Fl C Ar cachesize[:readahead]
54 This manual page documents the
56 utility which provides miscellaneous functions related to managing a
59 For a general introduction to the
61 file system, its features, and
62 examples on how to set up and maintain one, see
65 The options are as follows:
66 .Bl -tag -width indent
70 Tell the mirror commands to use a 2-way protocol, which allows
71 automatic negotiation of transaction id ranges.
72 This option is automatically enabled by the
76 Specify a bandwidth limit in bytes per second for mirroring streams.
77 This option is typically used to prevent batch mirroring operations from
78 loading down the machine.
79 The bandwidth may be suffixed with
84 to specify values in kilobytes, megabytes, and gigabytes per second.
85 If no suffix is specified, bytes per second is assumed.
87 When pruning and reblocking you can instruction
89 to start at the object id stored in the specified file.
90 If the file does not exist
92 will start at the beginning.
96 specific period of time and is unable to complete the operation it will
97 write out the current object id so the next run can pick up where it left off.
100 runs to completion it will delete
102 .It Fl f Ar blkdev[:blkdev]*
103 Specify the volumes making up a
107 When maintaining a streaming mirroring this option specifies the
108 minimum delay after a batch ends before the next batch is allowed
110 The default is five seconds.
112 Decrease verboseness.
113 May be specified multiple times.
115 Specify recursion for those commands which support it.
117 When pruning and reblocking you can tell the utility to stop after a
118 certain period of time.
119 This option is used along with the
121 option to prune or reblock a portion of the file system incrementally.
123 Increase verboseness.
124 May be specified multiple times.
126 Force "yes" for any interactive question.
127 .It Fl C Ar cachesize[:readahead]
128 Set the memory cache size for any raw I/O. The default is 16m.
129 A suffix of 'k' for kilobytes and 'm' for megabytes is allowed,
130 else the cache size is specified in bytes.
132 The read-behind/read-ahead defaults to 4 hammer blocks.
134 This option is typically only used with diagnostic commands
135 as kernel-supported commands will use the kernel's buffer cache.
138 The commands are as follows:
139 .Bl -tag -width indent
140 .\" ==== synctid ====
141 .It Ar synctid Ar filesystem Op quick
142 Generates a guaranteed, formal 64 bit transaction id representing the
143 current state of the specified
146 The file system will be synced to the media.
150 keyword is specified the file system will be soft-synced, meaning that a
151 crash might still undo the state of the file system as of the transaction
152 id returned but any new modifications will occur after the returned
153 transaction id as expected.
155 .It Ar bstats Op interval
158 B-tree statistics until interrupted.
161 seconds between each display.
162 The default interval is one second.
163 .\" ==== iostats ====
164 .It Ar iostats Op interval
167 I/O statistics until interrupted.
170 seconds between each display.
171 The default interval is one second.
172 .\" ==== history ====
173 .It Ar history Ar path ...
174 Show the modification history for
176 file's inode and data.
177 .\" ==== blockmap ====
179 Dump the blockmap for the filesystem. The HAMMER blockmap is two-layer
180 blockmap representing the maximum possible filesystem size of 1 Exabyte.
181 Needless to say the second layer is only present for blocks which exist.
182 HAMMER's blockmap represents 8-Megabyte blocks. Each block has an append
183 point, a free byte count, and a typed zone id which allows content to be
184 reverse engineered to some degree.
186 In HAMMER allocations essentially appended to a selected big-block using
187 the append offset and deducted from the free byte count. When space is
188 freed the free byte count is adjusted but HAMMER does not track holes in
189 big-blocks for reallocation. A big-block must be completely freed, either
190 through normal filesystem operations or through reblocking, before
193 Data blocks can be shared by deducting the space used from the free byte
194 count for each shared references, though HAMMER does not yet make use of
195 this feature. This means the free byte count can legally go negative.
197 This command needs the
202 Dump the B-tree. By default this command will validate all B-Tree
203 linkages and CRCs, including data CRCs, and will report the most verbose
204 information it can dig up.
205 Any errors will show up with a 'B' in column 1 along with various
210 the command will report less information about the inode contents.
214 the command will not report the content of the inode or other typed
219 the command will not report volume header information, big-block fill
220 ratios, mirror TIDs, or report or check data CRCs. B-Tree CRCs and
221 linkages are still checked.
223 This command needs the
227 .\" Dump the B-tree, record, large-data, and small-data blockmaps, showing
228 .\" physical block assignments and free space percentages.
229 .\" ==== namekey1 ====
230 .It Ar namekey1 Ar filename
233 64 bit directory hash for the specified file name, using
234 the original directory hash algorithm in version 1 of the filesystem.
235 The low 32 bits are used as an iterator for hash collisions and will be
237 .\" ==== namekey2 ====
238 .It Ar namekey2 Ar filename
241 64 bit directory hash for the specified file name, using
242 the new directory hash algorithm in version 2 of the filesystem.
243 The low 32 bits are still used as an iterator but will start out containing
244 part of the hash key.
245 .\" ==== namekey32 ====
246 .It Ar namekey32 Ar filename
247 Generate the top 32 bits of a
249 64 bit directory hash for the specified file name.
252 Shows extended information about all the mounted HAMMER filesystems.
253 At the moment volume identification, big blocks information and space details are shown.
254 .\" ==== cleanup ====
255 .It Ar cleanup Op Ar filesystem ...
256 This is a meta-command which executes snapshot, pruning, and reblocking
257 commands on the specified
262 is specified this command will clean-up all
264 file systems in use, including PFS's.
265 To do this it will scan all
269 mounts, extract PFS id's, and clean-up each PFS found.
271 This command will by default access a
277 creating them if necessary.
278 The format of the configuration file is:
279 .Bd -literal -offset indent
280 snapshots <period> <retention-time> [any]
281 prune <period> <max-runtime>
282 .\"rebalance <period> <max-runtime>
283 reblock <period> <1/3 max-runtime>
284 recopy <period> <1/3 max-runtime>
287 snapshots 1d 60d # 0d 60d for PFS /tmp, /var/tmp, /usr/obj
294 Time is given with a suffix of
300 meaning day, hour, minute and second.
302 If the snapshots directive has a period of 0 and a retention time of 0
303 then snapshot generation is disabled, removal of old snapshots are
304 disabled, and prunes will use
305 .Ar prune-everything .
306 If the snapshots directive has a period of 0 but a non-zero retention time
307 then this command will not create any new snapshots but will remove old
308 snapshots it finds based on the retention time.
310 By default only snapshots in the form: snap-yyyymmdd[-hhmm] are processed.
313 directive is specified as a third argument on the snapshots config line
314 then any softlink of the form *[- or .]yyyymmdd[-hhmm] will be processed.
316 A prune max-runtime of 0 means unlimited.
318 If period hasn't passed since the previous
321 For example a day has passed when midnight is passed (localtime).
329 The default configuration file will create a daily snapshot, do a daily
330 pruning and reblocking run and a monthly recopy run.
331 Reblocking is defragmentation with a level of 95%,
332 and recopy is full defragmentation.
334 By default prune and reblock operations are limited to 5 minutes per function,
335 and recopy operations are limited to 10 minutes per function.
336 Reblocking and recopy runs are each broken down into three separate functions
337 (btree, inodes and data)
338 and are thus by default limited to 15 and 30 minutes respectively.
339 Also note that this directive will by default disable snapshots on
346 The defaults may be adjusted by modifying the
349 The pruning and reblocking commands automatically maintain a cyclefile
350 for incremental operation.
351 If you interrupt (^C) the program the cyclefile will be updated, but a sub-command
352 may continue to run in the background for a few seconds until the
354 ioctl detects the interrupt.
357 PFS option can be set to use another location for the snapshots directory.
359 Work on this command is still in progress.
360 Expected additions: An ability to remove snapshots dynamically as the
361 file system becomes full.
363 .It Ar expand Ar filesystem Ar device
364 This command will format
366 and add all of its space to
369 NOTE! All existing data contained on
371 will be destroyed by this operation! If
375 filesystem, formatting will be denied. You can overcome this sanity check
378 to erase the beginning sectors of the device.
379 Also remember that you have to specify
381 together with any other device that make the filesystem, colon-separated to
383 .\" ==== snapshot ====
384 .It Ar snapshot Oo Ar filesystem Oc Ar snapshot-dir
385 Takes a snapshot of the file system either explicitly given by
387 or implicitly derived from the
389 argument and creates a symlink in the directory provided by
391 pointing to the snapshot.
394 is not a directory, it is assumed to be a format string passed to
396 with the current time as parameter.
399 refers to an existing directory, a default format string of "snap-%Y%d%m-%H%M"
400 is assumed and used as name for the newly created symlink.
402 Snapshot is a per PFS operation, so a
404 file system and each PFS in it have to be snapshot separately.
406 Example, assuming that
412 is a file system on its own, the following invocations:
413 .Bd -literal -offset indent
414 hammer snapshot /mysnapshots
416 hammer snapshot /mysnapshots/%Y-%m-%d
418 hammer snapshot /obj /mysnapshots/obj-%Y-%m-%d
421 would create symlinks similar to:
422 .Bd -literal -offset indent
423 /mysnapshots/snap-20080627-1210 -> /@@0x10d2cd05b7270d16
425 /mysnapshots/2008-06-27 -> /@@0x10d2cd05b7270d16
427 /mysnapshots/obj-2008-06-27 -> /obj@@0x10d2cd05b7270d16
430 .It Ar prune Ar softlink-dir
431 Prune the file system based on previously created snapshot softlinks.
432 Pruning is the act of deleting file system history.
436 will delete file system history such that
437 the file system state is retained for the given snapshots,
438 and all history after the latest snapshot,
439 but all other history is deleted.
441 The target directory is expected to contain softlinks pointing to
442 snapshots of the file systems you wish to retain.
443 The directory is scanned non-recursively and the mount points and
444 transaction ids stored in the softlinks are extracted and sorted.
445 The file system is then explicitly pruned according to what is found.
446 Cleaning out portions of the file system is as simple as removing a softlink
451 As a safety measure pruning only occurs if one or more softlinks are found
452 containing the @@ snapshot id extension.
453 Currently the scanned softlink directory must contain softlinks pointing
457 The softlinks may specify absolute or relative paths.
458 Softlinks must use 20-character (@@0x%016llx) transaction ids,
459 as might be returned from
460 .Dq Nm Ar synctid filesystem .
462 Pruning is a per PFS operation, so a
464 file system and each PFS in it have to be pruned separately.
466 Note that pruning a file system may not immediately free-up space,
467 though typically some space will be freed if a large number of records are
469 The file system must be reblocked to completely recover all available space.
471 Example, lets say your snapshot directory contains the following links:
472 .Bd -literal -offset indent
473 lrwxr-xr-x 1 root wheel 29 May 31 17:57 snap1 ->
474 /usr/obj/@@0x10d2cd05b7270d16
476 lrwxr-xr-x 1 root wheel 29 May 31 17:58 snap2 ->
477 /usr/obj/@@0x10d2cd13f3fde98f
479 lrwxr-xr-x 1 root wheel 29 May 31 17:59 snap3 ->
480 /usr/obj/@@0x10d2cd222adee364
483 If you were to run the
485 command on this directory, then the
488 mount will be pruned to retain the above three snapshots.
489 In addition, history for modifications made to the file system older than
490 the oldest snapshot will be destroyed and history for potentially fine-grained
491 modifications made to the file system more recently than the most recent
492 snapshot will be retained.
494 If you then delete the
496 softlink and rerun the
499 history for modifications pertaining to that snapshot would be destroyed.
500 .\" ==== prune-everything ====
501 .It Ar prune-everything Ar filesystem
502 This command will remove all historical records from the file system.
503 This directive is not normally used on a production system.
504 .\" ==== rebalance ====
505 .It Ar rebalance Ar filesystem Op Ar saturation_level
506 This command will rebalance the B-Tree, nodes with small numbers of
507 elements will be combined and element counts will be smoothed out
510 The saturation level is a percentage between 50 and 100. The default
512 .\" ==== reblock ====
513 .It Ar reblock Ar filesystem Op Ar fill_percentage
514 .It Ar reblock-btree Ar filesystem Op Ar fill_percentage
515 .It Ar reblock-inodes Ar filesystem Op Ar fill_percentage
516 .It Ar reblock-dirs Ar filesystem Op Ar fill_percentage
517 .It Ar reblock-data Ar filesystem Op Ar fill_percentage
518 Attempt to defragment and free space for reuse by reblocking a live
521 Big blocks cannot be reused by
523 until they are completely free.
524 This command also has the effect of reordering all elements, effectively
525 defragmenting the file system.
527 The default fill percentage is 100% and will cause the file system to be
528 completely defragmented.
529 All specified element types will be reallocated and rewritten.
530 If you wish to quickly free up space instead try specifying
531 a smaller fill percentage, such as 90% or 80% (the
533 suffix is not needed).
535 Since this command may rewrite the entire contents of the disk it is
536 best to do it incrementally from a
542 options to limit the run time.
543 The file system would thus be defragmented over long period of time.
545 It is recommended that separate invocations be used for each data type.
546 B-tree nodes, inodes, and directories are typically the most important
547 elements needing defragmentation.
548 Data can be defragmented over a longer period of time.
550 Reblocking is a per PFS operation, so a
552 file system and each PFS in it have to be reblocked separately.
553 .\" ==== pfs-status ====
554 .It Ar pfs-status Ar dirpath ...
555 Retrieve the mirroring configuration parameters for the specified
557 file systems or pseudo-filesystems (PFS's).
558 .\" ==== pfs-master ====
559 .It Ar pfs-master Ar dirpath Op options
560 Create a pseudo-filesystem (PFS) inside a
563 Up to 65535 such file systems can be created.
564 Each PFS uses an independent inode numbering space making it suitable
565 for use as a replication source or target.
569 directive creates a PFS that you can read, write, and use as a mirroring
572 It is recommended to use a
574 mount to access a PFS, for more information see
576 .\" ==== pfs-slave ====
577 .It Ar pfs-slave Ar dirpath Op options
578 Create a pseudo-filesystem (PFS) inside a
581 Up to 65535 such file systems can be created.
582 Each PFS uses an independent inode numbering space making it suitable
583 for use as a replication source or target.
587 directive creates a PFS that you can use as a mirroring target.
588 You will not be able to access a slave PFS until you have completed the
589 first mirroring operation with it as the target (its root directory will
590 not exist until then).
592 Access to the pfs-slave via the special softlink,
598 dynamically modify the snapshot transaction id by returning a dynamic result
603 A PFS can only be truly destroyed with the
606 Removing the softlink will not destroy the underlying PFS.
608 It is recommended to use a
610 mount to access a PFS, for more information see
612 .\" ==== pfs-update ====
613 .It Ar pfs-update Ar dirpath Op options
614 Update the configuration parameters for an existing
617 or pseudo-filesystem.
618 Options that may be specified:
619 .Bl -tag -width indent
620 .It sync-beg-tid=0x16llx
621 This is the automatic snapshot access starting transaction id for
623 This parameter is normally updated automatically by the
627 It is important to note that accessing a mirroring slave
628 with a transaction id greater than the last fully synchronized transaction
629 id can result in an unreliable snapshot since you will be accessing
630 data that is still undergoing synchronization.
632 Manually modifying this field is dangerous and can result in a broken
634 .It sync-end-tid=0x16llx
635 This is the current synchronization point for mirroring slaves.
636 This parameter is normally updated automatically by the
640 Manually modifying this field is dangerous and can result in a broken mirror.
641 .It shared-uuid=<uuid>
642 Set the shared UUID for this file system.
643 All mirrors must have the same shared UUID.
644 For safety purposes the
646 directives will refuse to operate on a target with a different shared UUID.
648 Changing the shared UUID on an existing, non-empty mirroring target,
649 including an empty but not completely pruned target,
650 can lead to corruption of the mirroring target.
651 .It unique-uuid=<uuid>
652 Set the unique UUID for this file system.
653 This UUID should not be used anywhere else,
654 even on exact copies of the file system.
656 Set a descriptive label for this file system.
657 .It snapshots=<string>
658 Specify the snapshots directory which
661 will use to manage this PFS.
662 The snapshots directory does not need to be configured for
663 PFS masters and will default to
664 .Pa <pfs>/snapshots .
666 PFS slaves are mirroring slaves so you cannot configure a snapshots
667 directory on the slave itself to be managed by the slave's machine.
668 In fact, the slave will likely have a
670 sub-directory mirrored
671 from the master, but that directory contains the configuration the master
672 is using for its copy of the file system, not the configuration that we
673 want to use for our slave.
675 It is recommended that
676 .Pa <fs>/var/slaves/<name>
677 be configured for a PFS slave, where
683 is an appropriate label.
684 You can control snapshot retention on your slave independent of the master.
686 Zero out the snapshots directory path for this PFS.
688 .\" ==== pfs-upgrade ====
689 .It Ar pfs-upgrade Ar dirpath
690 Upgrade a PFS from slave to master operation.
691 The PFS will be rolled back to the current end synchronization tid
692 (removing any partial synchronizations), and will then become writable.
696 currently supports only single masters and using
697 this command can easily result in file system corruption
698 if you don't know what you are doing.
700 This directive will refuse to run if any programs have open descriptors
701 in the PFS, including programs chdir'd into the PFS.
702 .\" ==== pfs-downgrade ====
703 .It Ar pfs-downgrade Ar dirpath
704 Downgrade a master PFS from master to slave operation
705 The PFS becomes read-only and access will be locked to its
708 This directive will refuse to run if any programs have open descriptors
709 in the PFS, including programs chdir'd into the PFS.
710 .\" ==== pfs-destroy ====
711 .It Ar pfs-destroy Ar dirpath
712 This permanently destroys a PFS.
714 This directive will refuse to run if any programs have open descriptors
715 in the PFS, including programs chdir'd into the PFS.
716 .\" ==== mirror-read ====
717 .It Ar mirror-read Ar filesystem Op Ar <begin-tid>
718 Generate a mirroring stream to stdout.
719 The stream ends when the transaction id space has been exhausted.
720 .\" ==== mirror-read-stream ====
721 .It Ar mirror-read-stream Ar filesystem Op Ar <begin-tid>
722 Generate a mirroring stream to stdout.
723 Upon completion the stream is paused until new data is synced to the
724 master, then resumed.
725 Operation continues until the pipe is broken.
726 .\" ==== mirror-write ====
727 .It Ar mirror-write Ar filesystem
728 Take a mirroring stream on stdin.
730 This command will fail if the
732 configuration field for the two file systems do not match.
734 If the target PFS does not exist this command will ask you whether
735 you want to create a compatible PFS slave for the target or not.
736 .\" ==== mirror-dump ====
742 to dump an ASCII representation of the mirroring stream.
743 .\" ==== mirror-copy ====
744 .It Ar mirror-copy Ar [[user@]host:]filesystem Ar [[user@]host:]filesystem
745 This is a shortcut which pipes a
750 If a remote host specification is made the program forks a
756 on the appropriate host.
757 The source may be a master or slave PFS, and the target must be a slave PFS.
759 This command also established full duplex communication and turns on
760 the two-way protocol feature which automatically negotiates transaction id
761 ranges without having to use a cyclefile.
762 If the operation completes successfully the target PFS's
765 Note that you must re-chdir into the target PFS to see the updated information.
766 If you do not you will still be in the previous snapshot.
768 If the target PFS does not exist this command will ask you whether
769 you want to create a compatible PFS slave for the target or not.
770 .\" ==== mirror-stream ====
771 .It Ar mirror-stream Ar [[user@]host:]filesystem Ar [[user@]host:]filesystem
772 This command works similarly to
774 but does not exit unless the pipe is broken.
775 This command will resume the mirroring operation whenever the master is synced.
776 The command is commonly used with
780 options to keep the mirroring target in sync with the source on a continuing
782 .\" ==== version ====
783 .It Ar version Ar filesystem
784 This command returns the
786 filesystem version for the specified
787 filesystem as well as the range of versions supported in the kernel.
790 option may be used to remove the summary at the end.
791 .\" ==== version-upgrade ====
792 .It Ar version-upgrade Ar filesystem Ar version Op Ar force
793 This command upgrades the
795 filesystem to the specified version.
796 Once upgraded a filesystem may not be downgraded.
797 If you wish to upgrade a filesystem to a version greater or equal to the
798 work-in-progress version number you must specify the
801 Use of WIP versions should be relegated to testing and may require wiping
802 the filesystem as development progresses, even though the WIP version might
805 NOTE! This command operates on the entire
807 filesystem and is not a per-PFS operation.
808 All PFS's will be affected.
809 .Bl -tag -width indent
812 default version, first
816 Work-in-progress version.
817 This version is developing a new directory hash key.
821 .Sh PSEUDO FILESYSTEM (PFS) NOTES
822 The root of a PFS is not hooked into the primary
824 file system as a directory.
827 creates a special softlink called "@@PFS%05d" (exactly 10 characters long)
832 then modifies the contents of the softlink as read by
834 and thus what you see with an
836 command or if you were to
839 If the PFS is a master the link reflects the current state of the PFS.
840 If the PFS is a slave the link reflects the last completed snapshot, and the
841 contents of the link will change when the next snapshot is completed, and
844 PFS support is currently very new and experimental.
847 utility employs numerous safeties to reduce user foot-shooting.
850 directive requires that the target be configured as a slave and that the
852 field of the mirroring source and target match.
854 .Bl -tag -width ".It Pa <fs>/var/slaves/<name>" -compact
856 default per PFS snapshots directory
857 .It Pa <snapshots>/config
861 .It Pa <fs>/var/slaves/<name>
862 recommended slave PFS snapshots directory
869 .Xr periodic.conf 5 ,
876 utility first appeared in
879 .An Matthew Dillon Aq dillon@backplane.com
projects / dragonfly.git / blob