1 .\" Copyright (c) 2007 The DragonFly Project. All rights reserved.
3 .\" This code is derived from software contributed to The DragonFly Project
4 .\" by Matthew Dillon <dillon@backplane.com>
6 .\" Redistribution and use in source and binary forms, with or without
7 .\" modification, are permitted provided that the following conditions
10 .\" 1. Redistributions of source code must retain the above copyright
11 .\" notice, this list of conditions and the following disclaimer.
12 .\" 2. Redistributions in binary form must reproduce the above copyright
13 .\" notice, this list of conditions and the following disclaimer in
14 .\" the documentation and/or other materials provided with the
16 .\" 3. Neither the name of The DragonFly Project nor the names of its
17 .\" contributors may be used to endorse or promote products derived
18 .\" from this software without specific, prior written permission.
20 .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
21 .\" ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
22 .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
23 .\" FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
24 .\" COPYRIGHT HOLDERS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
25 .\" INCIDENTAL, SPECIAL, EXEMPLARY OR CONSEQUENTIAL DAMAGES (INCLUDING,
26 .\" BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
27 .\" LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED
28 .\" AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
29 .\" OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
30 .\" OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
33 .\" $DragonFly: src/sbin/hammer/hammer.8,v 1.58 2008/11/13 02:04:27 dillon Exp $
40 .Nd HAMMER file system utility
46 .Op Fl f Ar blkdev[:blkdev]*
47 .\" .Op Fl s Ar linkpath
53 This manual page documents the
55 utility which provides miscellaneous functions related to managing a
58 For a general introduction to the
60 file system, its features, and
61 examples on how to set up and maintain one, see
64 The options are as follows:
65 .Bl -tag -width indent
69 Tell the mirror commands to use a 2-way protocol, which allows
70 automatic negotiation of transaction id ranges.
71 This option is automatically enabled by the
75 Specify a bandwidth limit in bytes per second for mirroring streams.
76 This option is typically used to prevent batch mirroring operations from
77 loading down the machine.
78 The bandwidth may be suffixed with
83 to specify values in kilobytes, megabytes, and gigabytes per second.
84 If no suffix is specified, bytes per second is assumed.
86 When pruning and reblocking you can instruction
88 to start at the object id stored in the specified file.
89 If the file does not exist
91 will start at the beginning.
95 specific period of time and is unable to complete the operation it will
96 write out the current object id so the next run can pick up where it left off.
99 runs to completion it will delete
101 .It Fl f Ar blkdev[:blkdev]*
102 Specify the volumes making up a
106 When maintaining a streaming mirroring this option specifies the
107 minimum delay after a batch ends before the next batch is allowed
109 The default is five seconds.
111 Decrease verboseness.
112 May be specified multiple times.
114 Specify recursion for those commands which support it.
116 When pruning and reblocking you can tell the utility to stop after a
117 certain period of time.
118 This option is used along with the
120 option to prune or reblock a portion of the file system incrementally.
122 Increase verboseness.
123 May be specified multiple times.
126 The commands are as follows:
127 .Bl -tag -width indent
128 .\" ==== synctid ====
129 .It Ar synctid Ar filesystem Op quick
130 Generates a guaranteed, formal 64 bit transaction id representing the
131 current state of the specified
134 The file system will be synced to the media.
138 keyword is specified the file system will be soft-synced, meaning that a
139 crash might still undo the state of the file system as of the transaction
140 id returned but any new modifications will occur after the returned
141 transaction id as expected.
143 .It Ar bstats Op interval
146 B-tree statistics until interrupted.
149 seconds between each display.
150 The default interval is one second.
151 .\" ==== iostats ====
152 .It Ar iostats Op interval
155 I/O statistics until interrupted.
158 seconds between each display.
159 The default interval is one second.
160 .\" ==== history ====
161 .It Ar history Ar path ...
162 Show the modification history for
164 file's inode and data.
168 This command needs the
172 .\" Dump the B-tree, record, large-data, and small-data blockmaps, showing
173 .\" physical block assignments and free space percentages.
174 .\" ==== namekey1 ====
175 .It Ar namekey1 Ar filename
178 64 bit directory hash for the specified file name, using
179 the original directory hash algorithm in version 1 of the filesystem.
180 The low 32 bits are used as an iterator for hash collisions and will be
182 .\" ==== namekey2 ====
183 .It Ar namekey2 Ar filename
186 64 bit directory hash for the specified file name, using
187 the new directory hash algorithm in version 2 of the filesystem.
188 The low 32 bits are still used as an iterator but will start out containing
189 part of the hash key.
190 .\" ==== namekey32 ====
191 .It Ar namekey32 Ar filename
192 Generate the top 32 bits of a
194 64 bit directory hash for the specified file name.
195 .\" ==== cleanup ====
196 .It Ar cleanup Op Ar filesystem ...
197 This is a meta-command which executes snapshot, pruning, and reblocking
198 commands on the specified
203 is specified this command will clean-up all
205 file systems in use, including PFS's.
206 To do this it will scan all
210 mounts, extract PFS id's, and clean-up each PFS found.
212 This command will by default access a
218 creating them if necessary.
219 The format of the configuration file is:
220 .Bd -literal -offset indent
221 snapshots <period> <retention-time> [any]
222 prune <period> <max-runtime>
223 rebalance <period> <max-runtime>
224 reblock <period> <1/3 max-runtime>
225 recopy <period> <1/3 max-runtime>
228 snapshots 1d 60d # 0d 60d for PFS /tmp, /var/tmp, /usr/obj
235 Time is given with a suffix of
241 meaning day, hour, minute and second.
243 If the snapshots directive has a period of 0 and a retention time of 0
244 then snapshot generation is disabled, removal of old snapshots are
245 disabled, and prunes will use
246 .Ar prune-everything .
247 If the snapshots directive has a period of 0 but a non-zero retention time
248 then this command will not create any new snapshots but will remove old
249 snapshots it finds based on the retention time.
251 By default only snapshots in the form: snap-yyyymmdd[-hhmm] are processed.
254 directive is specified as a third argument on the snapshots config line
255 then any softlink of the form *[- or .]yyyymmdd[-hhmm] will be processed.
257 A prune max-runtime of 0 means unlimited.
259 If period hasn't passed since the previous
262 For example a day has passed when midnight is passed (localtime).
270 The default configuration file will create a daily snapshot, do a daily
271 pruning and reblocking run and a monthly recopy run.
272 Reblocking is defragmentation with a level of 95%,
273 and recopy is full defragmentation.
275 By default prune and reblock operations are limited to 5 minutes per function,
276 and recopy operations are limited to 10 minutes per function.
277 Reblocking and recopy runs are each broken down into three separate functions
278 (btree, inodes and data)
279 and are thus by default limited to 15 and 30 minutes respectively.
280 Also note that this directive will by default disable snapshots on
287 The defaults may be adjusted by modifying the
290 The pruning and reblocking commands automatically maintain a cyclefile
291 for incremental operation.
292 If you interrupt (^C) the program the cyclefile will be updated, but a sub-command
293 may continue to run in the background for a few seconds until the
295 ioctl detects the interrupt.
298 PFS option can be set to use another location for the snapshots directory.
300 Work on this command is still in progress.
301 Expected additions: An ability to remove snapshots dynamically as the
302 file system becomes full.
303 .\" ==== snapshot ====
304 .It Ar snapshot Oo Ar filesystem Oc Ar snapshot-dir
305 Takes a snapshot of the file system either explicitly given by
307 or implicitly derived from the
309 argument and creates a symlink in the directory provided by
311 pointing to the snapshot.
314 is not a directory, it is assumed to be a format string passed to
316 with the current time as parameter.
319 refers to an existing directory, a default format string of "snap-%Y%d%m-%H%M"
320 is assumed and used as name for the newly created symlink.
322 Snapshot is a per PFS operation, so a
324 file system and each PFS in it have to be snapshot separately.
326 Example, assuming that
332 is a file system on its own, the following invocations:
333 .Bd -literal -offset indent
334 hammer snapshot /mysnapshots
336 hammer snapshot /mysnapshots/%Y-%m-%d
338 hammer snapshot /obj /mysnapshots/obj-%Y-%m-%d
341 would create symlinks similar to:
342 .Bd -literal -offset indent
343 /mysnapshots/snap-20080627-1210 -> /@@0x10d2cd05b7270d16
345 /mysnapshots/2008-06-27 -> /@@0x10d2cd05b7270d16
347 /mysnapshots/obj-2008-06-27 -> /obj@@0x10d2cd05b7270d16
350 .It Ar prune Ar softlink-dir
351 Prune the file system based on previously created snapshot softlinks.
352 Pruning is the act of deleting file system history.
356 will delete file system history such that
357 the file system state is retained for the given snapshots,
358 and all history after the latest snapshot,
359 but all other history is deleted.
361 The target directory is expected to contain softlinks pointing to
362 snapshots of the file systems you wish to retain.
363 The directory is scanned non-recursively and the mount points and
364 transaction ids stored in the softlinks are extracted and sorted.
365 The file system is then explicitly pruned according to what is found.
366 Cleaning out portions of the file system is as simple as removing a softlink
371 As a safety measure pruning only occurs if one or more softlinks are found
372 containing the @@ snapshot id extension.
373 Currently the scanned softlink directory must contain softlinks pointing
377 The softlinks may specify absolute or relative paths.
378 Softlinks must use 20-character (@@0x%016llx) transaction ids,
379 as might be returned from
380 .Dq Nm Ar synctid filesystem .
382 Pruning is a per PFS operation, so a
384 file system and each PFS in it have to be pruned separately.
386 Note that pruning a file system may not immediately free-up space,
387 though typically some space will be freed if a large number of records are
389 The file system must be reblocked to completely recover all available space.
391 Example, lets say your snapshot directory contains the following links:
392 .Bd -literal -offset indent
393 lrwxr-xr-x 1 root wheel 29 May 31 17:57 snap1 ->
394 /usr/obj/@@0x10d2cd05b7270d16
396 lrwxr-xr-x 1 root wheel 29 May 31 17:58 snap2 ->
397 /usr/obj/@@0x10d2cd13f3fde98f
399 lrwxr-xr-x 1 root wheel 29 May 31 17:59 snap3 ->
400 /usr/obj/@@0x10d2cd222adee364
403 If you were to run the
405 command on this directory, then the
408 mount will be pruned to retain the above three snapshots.
409 In addition, history for modifications made to the file system older than
410 the oldest snapshot will be destroyed and history for potentially fine-grained
411 modifications made to the file system more recently than the most recent
412 snapshot will be retained.
414 If you then delete the
416 softlink and rerun the
419 history for modifications pertaining to that snapshot would be destroyed.
420 .\" ==== prune-everything ====
421 .It Ar prune-everything Ar filesystem
422 This command will remove all historical records from the file system.
423 This directive is not normally used on a production system.
424 .\" ==== rebalance ====
425 .It Ar rebalance Ar filesystem Op Ar saturation_level
426 This command will rebalance the B-Tree, nodes with small numbers of
427 elements will be combined and element counts will be smoothed out
430 The saturation level is a percentage between 50 and 100. The default
432 .\" ==== reblock ====
433 .It Ar reblock Ar filesystem Op Ar fill_percentage
434 .It Ar reblock-btree Ar filesystem Op Ar fill_percentage
435 .It Ar reblock-inodes Ar filesystem Op Ar fill_percentage
436 .It Ar reblock-dirs Ar filesystem Op Ar fill_percentage
437 .It Ar reblock-data Ar filesystem Op Ar fill_percentage
438 Attempt to defragment and free space for reuse by reblocking a live
441 Big blocks cannot be reused by
443 until they are completely free.
444 This command also has the effect of reordering all elements, effectively
445 defragmenting the file system.
447 The default fill percentage is 100% and will cause the file system to be
448 completely defragmented.
449 All specified element types will be reallocated and rewritten.
450 If you wish to quickly free up space instead try specifying
451 a smaller fill percentage, such as 90% or 80% (the
453 suffix is not needed).
455 Since this command may rewrite the entire contents of the disk it is
456 best to do it incrementally from a
462 options to limit the run time.
463 The file system would thus be defragmented over long period of time.
465 It is recommended that separate invocations be used for each data type.
466 B-tree nodes, inodes, and directories are typically the most important
467 elements needing defragmentation.
468 Data can be defragmented over a longer period of time.
470 Reblocking is a per PFS operation, so a
472 file system and each PFS in it have to be reblocked separately.
473 .\" ==== pfs-status ====
474 .It Ar pfs-status Ar dirpath ...
475 Retrieve the mirroring configuration parameters for the specified
477 file systems or pseudo-filesystems (PFS's).
478 .\" ==== pfs-master ====
479 .It Ar pfs-master Ar dirpath Op options
480 Create a pseudo-filesystem (PFS) inside a
483 Up to 65535 such file systems can be created.
484 Each PFS uses an independent inode numbering space making it suitable
485 for use as a replication source or target.
489 directive creates a PFS that you can read, write, and use as a mirroring
492 It is recommended to use a
494 mount to access a PFS, for more information see
496 .\" ==== pfs-slave ====
497 .It Ar pfs-slave Ar dirpath Op options
498 Create a pseudo-filesystem (PFS) inside a
501 Up to 65535 such file systems can be created.
502 Each PFS uses an independent inode numbering space making it suitable
503 for use as a replication source or target.
507 directive creates a PFS that you can use as a mirroring target.
508 You will not be able to access a slave PFS until you have completed the
509 first mirroring operation with it as the target (its root directory will
510 not exist until then).
512 Access to the pfs-slave via the special softlink,
518 dynamically modify the snapshot transaction id by returning a dynamic result
523 A PFS can only be truly destroyed with the
526 Removing the softlink will not destroy the underlying PFS.
528 It is recommended to use a
530 mount to access a PFS, for more information see
532 .\" ==== pfs-update ====
533 .It Ar pfs-update Ar dirpath Op options
534 Update the configuration parameters for an existing
537 or pseudo-filesystem.
538 Options that may be specified:
539 .Bl -tag -width indent
540 .It sync-beg-tid=0x16llx
541 This is the automatic snapshot access starting transaction id for
543 This parameter is normally updated automatically by the
547 It is important to note that accessing a mirroring slave
548 with a transaction id greater than the last fully synchronized transaction
549 id can result in an unreliable snapshot since you will be accessing
550 data that is still undergoing synchronization.
552 Manually modifying this field is dangerous and can result in a broken
554 .It sync-end-tid=0x16llx
555 This is the current synchronization point for mirroring slaves.
556 This parameter is normally updated automatically by the
560 Manually modifying this field is dangerous and can result in a broken mirror.
561 .It shared-uuid=<uuid>
562 Set the shared UUID for this file system.
563 All mirrors must have the same shared UUID.
564 For safety purposes the
566 directives will refuse to operate on a target with a different shared UUID.
568 Changing the shared UUID on an existing, non-empty mirroring target,
569 including an empty but not completely pruned target,
570 can lead to corruption of the mirroring target.
571 .It unique-uuid=<uuid>
572 Set the unique UUID for this file system.
573 This UUID should not be used anywhere else,
574 even on exact copies of the file system.
576 Set a descriptive label for this file system.
577 .It snapshots=<string>
578 Specify the snapshots directory which
581 will use to manage this PFS.
582 The snapshots directory does not need to be configured for
583 PFS masters and will default to
584 .Pa <pfs>/snapshots .
586 PFS slaves are mirroring slaves so you cannot configure a snapshots
587 directory on the slave itself to be managed by the slave's machine.
588 In fact, the slave will likely have a
590 sub-directory mirrored
591 from the master, but that directory contains the configuration the master
592 is using for its copy of the file system, not the configuration that we
593 want to use for our slave.
595 It is recommended that
596 .Pa <fs>/var/slaves/<name>
597 be configured for a PFS slave, where
603 is an appropriate label.
604 You can control snapshot retention on your slave independent of the master.
606 Zero out the snapshots directory path for this PFS.
608 .\" ==== pfs-upgrade ====
609 .It Ar pfs-upgrade Ar dirpath
610 Upgrade a PFS from slave to master operation.
611 The PFS will be rolled back to the current end synchronization tid
612 (removing any partial synchronizations), and will then become writable.
616 currently supports only single masters and using
617 this command can easily result in file system corruption
618 if you don't know what you are doing.
620 This directive will refuse to run if any programs have open descriptors
621 in the PFS, including programs chdir'd into the PFS.
622 .\" ==== pfs-downgrade ====
623 .It Ar pfs-downgrade Ar dirpath
624 Downgrade a master PFS from master to slave operation
625 The PFS becomes read-only and access will be locked to its
628 This directive will refuse to run if any programs have open descriptors
629 in the PFS, including programs chdir'd into the PFS.
630 .\" ==== pfs-destroy ====
631 .It Ar pfs-destroy Ar dirpath
632 This permanently destroys a PFS.
634 This directive will refuse to run if any programs have open descriptors
635 in the PFS, including programs chdir'd into the PFS.
636 .\" ==== mirror-read ====
637 .It Ar mirror-read Ar filesystem Op Ar <begin-tid>
638 Generate a mirroring stream to stdout.
639 The stream ends when the transaction id space has been exhausted.
640 .\" ==== mirror-read-stream ====
641 .It Ar mirror-read-stream Ar filesystem Op Ar <begin-tid>
642 Generate a mirroring stream to stdout.
643 Upon completion the stream is paused until new data is synced to the
644 master, then resumed.
645 Operation continues until the pipe is broken.
646 .\" ==== mirror-write ====
647 .It Ar mirror-write Ar filesystem
648 Take a mirroring stream on stdin.
650 This command will fail if the
652 configuration field for the two file systems do not match.
654 If the target PFS does not exist this command will ask you whether
655 you want to create a compatible PFS slave for the target or not.
656 .\" ==== mirror-dump ====
662 to dump an ASCII representation of the mirroring stream.
663 .\" ==== mirror-copy ====
664 .It Ar mirror-copy Ar [[user@]host:]filesystem Ar [[user@]host:]filesystem
665 This is a shortcut which pipes a
670 If a remote host specification is made the program forks a
676 on the appropriate host.
677 The source may be a master or slave PFS, and the target must be a slave PFS.
679 This command also established full duplex communication and turns on
680 the two-way protocol feature which automatically negotiates transaction id
681 ranges without having to use a cyclefile.
682 If the operation completes successfully the target PFS's
685 Note that you must re-chdir into the target PFS to see the updated information.
686 If you do not you will still be in the previous snapshot.
688 If the target PFS does not exist this command will ask you whether
689 you want to create a compatible PFS slave for the target or not.
690 .\" ==== mirror-stream ====
691 .It Ar mirror-stream Ar [[user@]host:]filesystem Ar [[user@]host:]filesystem
692 This command works similarly to
694 but does not exit unless the pipe is broken.
695 This command will resume the mirroring operation whenever the master is synced.
696 The command is commonly used with
700 options to keep the mirroring target in sync with the source on a continuing
702 .\" ==== version ====
703 .It Ar version Ar filesystem
704 This command returns the
706 filesystem version for the specified
707 filesystem as well as the range of versions supported in the kernel.
710 option may be used to remove the summary at the end.
711 .\" ==== version-upgrade ====
712 .It Ar version-upgrade Ar filesystem Ar version Op Ar force
713 This command upgrades the
715 filesystem to the specified version.
716 Once upgraded a filesystem may not be downgraded.
717 If you wish to upgrade a filesystem to a version greater or equal to the
718 work-in-progress version number you must specify the
721 Use of WIP versions should be relegated to testing and may require wiping
722 the filesystem as development progresses, even though the WIP version might
725 NOTE! This command operates on the entire
727 filesystem and is not a per-PFS operation.
728 All PFS's will be affected.
729 .Bl -tag -width indent
732 default version, first
736 Work-in-progress version.
737 This version is developing a new directory hash key.
741 .Sh PSEUDO FILESYSTEM (PFS) NOTES
742 The root of a PFS is not hooked into the primary
744 file system as a directory.
747 creates a special softlink called "@@PFS%05d" (exactly 10 characters long)
752 then modifies the contents of the softlink as read by
754 and thus what you see with an
756 command or if you were to
759 If the PFS is a master the link reflects the current state of the PFS.
760 If the PFS is a slave the link reflects the last completed snapshot, and the
761 contents of the link will change when the next snapshot is completed, and
764 PFS support is currently very new and experimental.
767 utility employs numerous safeties to reduce user foot-shooting.
770 directive requires that the target be configured as a slave and that the
772 field of the mirroring source and target match.
774 .Bl -tag -width ".It Pa <fs>/var/slaves/<name>" -compact
776 default per PFS snapshots directory
777 .It Pa <snapshots>/config
781 .It Pa <fs>/var/slaves/<name>
782 recommended slave PFS snapshots directory
789 .Xr periodic.conf 5 ,
796 utility first appeared in
799 .An Matthew Dillon Aq dillon@backplane.com
projects / dragonfly.git / blob