1 .\" Copyright (c) 2007 The DragonFly Project. All rights reserved.
3 .\" This code is derived from software contributed to The DragonFly Project
4 .\" by Matthew Dillon <dillon@backplane.com>
6 .\" Redistribution and use in source and binary forms, with or without
7 .\" modification, are permitted provided that the following conditions
10 .\" 1. Redistributions of source code must retain the above copyright
11 .\" notice, this list of conditions and the following disclaimer.
12 .\" 2. Redistributions in binary form must reproduce the above copyright
13 .\" notice, this list of conditions and the following disclaimer in
14 .\" the documentation and/or other materials provided with the
16 .\" 3. Neither the name of The DragonFly Project nor the names of its
17 .\" contributors may be used to endorse or promote products derived
18 .\" from this software without specific, prior written permission.
20 .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
21 .\" ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
22 .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
23 .\" FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
24 .\" COPYRIGHT HOLDERS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
25 .\" INCIDENTAL, SPECIAL, EXEMPLARY OR CONSEQUENTIAL DAMAGES (INCLUDING,
26 .\" BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
27 .\" LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED
28 .\" AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
29 .\" OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
30 .\" OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
33 .\" $DragonFly: src/sbin/hammer/hammer.8,v 1.58 2008/11/13 02:04:27 dillon Exp $
40 .Nd HAMMER file system utility
46 .Op Fl f Ar blkdev[:blkdev]*
47 .\" .Op Fl s Ar linkpath
53 This manual page documents the
55 utility which provides miscellaneous functions related to managing a
58 For a general introduction to the
60 file system, its features, and
61 examples on how to set up and maintain one, see
64 The options are as follows:
65 .Bl -tag -width indent
69 Tell the mirror commands to use a 2-way protocol, which allows
70 automatic negotiation of transaction id ranges.
71 This option is automatically enabled by the
75 Specify a bandwidth limit in bytes per second for mirroring streams.
76 This option is typically used to prevent batch mirroring operations from
77 loading down the machine.
78 The bandwidth may be suffixed with
83 to specify values in kilobytes, megabytes, and gigabytes per second.
84 If no suffix is specified, bytes per second is assumed.
86 When pruning and reblocking you can instruction
88 to start at the object id stored in the specified file.
89 If the file does not exist
91 will start at the beginning.
95 specific period of time and is unable to complete the operation it will
96 write out the current object id so the next run can pick up where it left off.
99 runs to completion it will delete
101 .It Fl f Ar blkdev[:blkdev]*
102 Specify the volumes making up a
106 When maintaining a streaming mirroring this option specifies the
107 minimum delay after a batch ends before the next batch is allowed
109 The default is five seconds.
111 Decrease verbosement.
112 May be specified multiple times.
114 Specify recursion for those commands which support it.
116 When pruning and reblocking you can tell the utility to stop after a
117 certain period of time.
118 This option is used along with the
120 option to prune or reblock a portion of the file system incrementally.
122 Increase verboseness.
123 May be specified multiple times.
126 The commands are as follows:
127 .Bl -tag -width indent
128 .\" ==== synctid ====
129 .It Ar synctid Ar filesystem Op quick
130 Generates a guaranteed, formal 64 bit transaction id representing the
131 current state of the specified
134 The file system will be synced to the media.
138 keyword is specified the file system will be soft-synced, meaning that a
139 crash might still undo the state of the file system as of the transaction
140 id returned but any new modifications will occur after the returned
141 transaction id as expected.
143 .It Ar bstats Op interval
146 B-tree statistics until interrupted.
149 seconds between each display.
150 The default interval is one second.
151 .\" ==== iostats ====
152 .It Ar iostats Op interval
155 I/O statistics until interrupted.
158 seconds between each display.
159 The default interval is one second.
160 .\" ==== history ====
161 .It Ar history Ar path ...
162 Show the modification history for
164 file's inode and data.
168 This command needs the
172 .\" Dump the B-tree, record, large-data, and small-data blockmaps, showing
173 .\" physical block assignments and free space percentages.
174 .\" ==== namekey1 ====
175 .It Ar namekey1 Ar filename
178 64 bit directory hash for the specified file name, using
179 the original directory hash algorithm in version 1 of the filesystem.
180 The low 32 bits are used as an iterator for hash collisions and will be
182 .\" ==== namekey2 ====
183 .It Ar namekey2 Ar filename
186 64 bit directory hash for the specified file name, using
187 the new directory hash algorithm in version 2 of the filesystem.
188 The low 32 bits are still used as an iterator but will start out containing
189 part of the hash key.
190 .\" ==== namekey32 ====
191 .It Ar namekey32 Ar filename
192 Generate the top 32 bits of a
194 64 bit directory hash for the specified file name.
195 .\" ==== cleanup ====
196 .It Ar cleanup Op Ar filesystem ...
197 This is a meta-command which executes snapshot, pruning, and reblocking
198 commands on the specified
203 is specified this command will clean-up all
205 file systems in use, including PFS's.
206 To do this it will scan all
210 mounts, extract PFS id's, and clean-up each PFS found.
212 This command will by default access a
218 creating them if necessary.
219 The format of the configuration file is:
220 .Bd -literal -offset indent
221 snapshots <period> <retention-time> [any]
222 prune <period> <max-runtime>
223 reblock <period> <1/3 max-runtime>
224 recopy <period> <1/3 max-runtime>
227 snapshots 1d 60d # 0d 60d for PFS /tmp, /var/tmp, /usr/obj
233 Time is given with a suffix of
239 meaning day, hour, minute and second.
241 If the snapshots directive has a period of 0 and a retention time of 0
242 then snapshot generation is disabled, removal of old snapshots are
243 disabled, and prunes will use
244 .Ar prune-everything .
245 If the snapshots directive has a period of 0 but a non-zero retention time
246 then this command will not create any new snapshots but will remove old
247 snapshots it finds based on the retention time.
249 By default only snapshots in the form: snap-yyyymmdd[-hhmm] are processed.
252 directive is specified as a third argument on the snapshots config line
253 then any softlink of the form *[- or .]yyyymmdd[-hhmm] will be processed.
255 A prune max-runtime of 0 means unlimited.
257 If period hasn't passed since the previous
260 For example a day has passed when midnight is passed (localtime).
268 The default configuration file will create a daily snapshot, do a daily
269 pruning and reblocking run and a monthly recopy run.
270 Reblocking is defragmentation with a level of 95%,
271 and recopy is full defragmentation.
273 By default prune and reblock operations are limited to 5 minutes per function,
274 and recopy operations are limited to 10 minutes per function.
275 Reblocking and recopy runs are each broken down into three separate functions
276 (btree, inodes and data)
277 and are thus by default limited to 15 and 30 minutes respectively.
278 Also note that this directive will by default disable snapshots on
285 The defaults may be adjusted by modifying the
288 The pruning and reblocking commands automatically maintain a cyclefile
289 for incremental operation.
290 If you interrupt (^C) the program the cyclefile will be updated, but a sub-command
291 may continue to run in the background for a few seconds until the
293 ioctl detects the interrupt.
296 PFS option can be set to use another location for the snapshots directory.
298 Work on this command is still in progress.
299 Expected additions: An ability to remove snapshots dynamically as the
300 file system becomes full.
301 .\" ==== snapshot ====
302 .It Ar snapshot Oo Ar filesystem Oc Ar snapshot-dir
303 Takes a snapshot of the file system either explicitly given by
305 or implicitly derived from the
307 argument and creates a symlink in the directory provided by
309 pointing to the snapshot.
312 is not a directory, it is assumed to be a format string passed to
314 with the current time as parameter.
317 refers to an existing directory, a default format string of "snap-%Y%d%m-%H%M"
318 is assumed and used as name for the newly created symlink.
320 Snapshot is a per PFS operation, so a
322 file system and each PFS in it have to be snapshot separately.
324 Example, assuming that
330 is a file system on its own, the following invocations:
331 .Bd -literal -offset indent
332 hammer snapshot /mysnapshots
334 hammer snapshot /mysnapshots/%Y-%m-%d
336 hammer snapshot /obj /mysnapshots/obj-%Y-%m-%d
339 would create symlinks similar to:
340 .Bd -literal -offset indent
341 /mysnapshots/snap-20080627-1210 -> /@@0x10d2cd05b7270d16
343 /mysnapshots/2008-06-27 -> /@@0x10d2cd05b7270d16
345 /mysnapshots/obj-2008-06-27 -> /obj@@0x10d2cd05b7270d16
348 .It Ar prune Ar softlink-dir
349 Prune the file system based on previously created snapshot softlinks.
350 Pruning is the act of deleting file system history.
354 will delete file system history such that
355 the file system state is retained for the given snapshots,
356 and all history after the latest snapshot,
357 but all other history is deleted.
359 The target directory is expected to contain softlinks pointing to
360 snapshots of the file systems you wish to retain.
361 The directory is scanned non-recursively and the mount points and
362 transaction ids stored in the softlinks are extracted and sorted.
363 The file system is then explicitly pruned according to what is found.
364 Cleaning out portions of the file system is as simple as removing a softlink
369 As a safety measure pruning only occurs if one or more softlinks are found
370 containing the @@ snapshot id extension.
371 Currently the scanned softlink directory must contain softlinks pointing
375 The softlinks may specify absolute or relative paths.
376 Softlinks must use 20-character (@@0x%016llx) transaction ids,
377 as might be returned from
378 .Dq Nm Ar synctid filesystem .
380 Pruning is a per PFS operation, so a
382 file system and each PFS in it have to be pruned separately.
384 Note that pruning a file system may not immediately free-up space,
385 though typically some space will be freed if a large number of records are
387 The file system must be reblocked to completely recover all available space.
389 Example, lets say your snapshot directory contains the following links:
390 .Bd -literal -offset indent
391 lrwxr-xr-x 1 root wheel 29 May 31 17:57 snap1 ->
392 /usr/obj/@@0x10d2cd05b7270d16
394 lrwxr-xr-x 1 root wheel 29 May 31 17:58 snap2 ->
395 /usr/obj/@@0x10d2cd13f3fde98f
397 lrwxr-xr-x 1 root wheel 29 May 31 17:59 snap3 ->
398 /usr/obj/@@0x10d2cd222adee364
401 If you were to run the
403 command on this directory, then the
406 mount will be pruned to retain the above three snapshots.
407 In addition, history for modifications made to the file system older than
408 the oldest snapshot will be destroyed and history for potentially fine-grained
409 modifications made to the file system more recently than the most recent
410 snapshot will be retained.
412 If you then delete the
414 softlink and rerun the
417 history for modifications pertaining to that snapshot would be destroyed.
418 .\" ==== prune-everything ====
419 .It Ar prune-everything Ar filesystem
420 This command will remove all historical records from the file system.
421 This directive is not normally used on a production system.
422 .\" ==== reblock ====
423 .It Ar reblock Ar filesystem Op Ar fill_percentage
424 .It Ar reblock-btree Ar filesystem Op Ar fill_percentage
425 .It Ar reblock-inodes Ar filesystem Op Ar fill_percentage
426 .It Ar reblock-dirs Ar filesystem Op Ar fill_percentage
427 .It Ar reblock-data Ar filesystem Op Ar fill_percentage
428 Attempt to defragment and free space for reuse by reblocking a live
431 Big blocks cannot be reused by
433 until they are completely free.
434 This command also has the effect of reordering all elements, effectively
435 defragmenting the file system.
437 The default fill percentage is 100% and will cause the file system to be
438 completely defragmented.
439 All specified element types will be reallocated and rewritten.
440 If you wish to quickly free up space instead try specifying
441 a smaller fill percentage, such as 90% or 80% (the
443 suffix is not needed).
445 Since this command may rewrite the entire contents of the disk it is
446 best to do it incrementally from a
452 options to limit the run time.
453 The file system would thus be defragmented over long period of time.
455 It is recommended that separate invocations be used for each data type.
456 B-tree nodes, inodes, and directories are typically the most important
457 elements needing defragmentation.
458 Data can be defragmented over a longer period of time.
460 Reblocking is a per PFS operation, so a
462 file system and each PFS in it have to be reblocked separately.
463 .\" ==== pfs-status ====
464 .It Ar pfs-status Ar dirpath ...
465 Retrieve the mirroring configuration parameters for the specified
467 file systems or pseudo-filesystems (PFS's).
468 .\" ==== pfs-master ====
469 .It Ar pfs-master Ar dirpath Op options
470 Create a pseudo-filesystem (PFS) inside a
473 Up to 65535 such file systems can be created.
474 Each PFS uses an independent inode numbering space making it suitable
475 for use as a replication source or target.
479 directive creates a PFS that you can read, write, and use as a mirroring
482 It is recommended to use a
484 mount to access a PFS, for more information see
486 .\" ==== pfs-slave ====
487 .It Ar pfs-slave Ar dirpath Op options
488 Create a pseudo-filesystem (PFS) inside a
491 Up to 65535 such file systems can be created.
492 Each PFS uses an independent inode numbering space making it suitable
493 for use as a replication source or target.
497 directive creates a PFS that you can use as a mirroring target.
498 You will not be able to access a slave PFS until you have completed the
499 first mirroring operation with it as the target (its root directory will
500 not exist until then).
502 Access to the pfs-slave via the special softlink,
508 dynamically modify the snapshot transaction id by returning a dynamic result
513 A PFS can only be truly destroyed with the
516 Removing the softlink will not destroy the underlying PFS.
518 It is recommended to use a
520 mount to access a PFS, for more information see
522 .\" ==== pfs-update ====
523 .It Ar pfs-update Ar dirpath Op options
524 Update the configuration parameters for an existing
527 or pseudo-filesystem.
528 Options that may be specified:
529 .Bl -tag -width indent
530 .It sync-beg-tid=0x16llx
531 This is the automatic snapshot access starting transaction id for
533 This parameter is normally updated automatically by the
537 It is important to note that accessing a mirroring slave
538 with a transaction id greater than the last fully synchronized transaction
539 id can result in an unreliable snapshot since you will be accessing
540 data that is still undergoing synchronization.
542 Manually modifying this field is dangerous and can result in a broken
544 .It sync-end-tid=0x16llx
545 This is the current synchronization point for mirroring slaves.
546 This parameter is normally updated automatically by the
550 Manually modifying this field is dangerous and can result in a broken mirror.
551 .It shared-uuid=<uuid>
552 Set the shared UUID for this file system.
553 All mirrors must have the same shared UUID.
554 For safety purposes the
556 directives will refuse to operate on a target with a different shared UUID.
558 Changing the shared UUID on an existing, non-empty mirroring target,
559 including an empty but not completely pruned target,
560 can lead to corruption of the mirroring target.
561 .It unique-uuid=<uuid>
562 Set the unique UUID for this file system.
563 This UUID should not be used anywhere else,
564 even on exact copies of the file system.
566 Set a descriptive label for this file system.
567 .It snapshots=<string>
568 Specify the snapshots directory which
571 will use to manage this PFS.
572 The snapshots directory does not need to be configured for
573 PFS masters and will default to
574 .Pa <pfs>/snapshots .
576 PFS slaves are mirroring slaves so you cannot configure a snapshots
577 directory on the slave itself to be managed by the slave's machine.
578 In fact, the slave will likely have a
580 sub-directory mirrored
581 from the master, but that directory contains the configuration the master
582 is using for its copy of the file system, not the configuration that we
583 want to use for our slave.
585 It is recommended that
586 .Pa <fs>/var/slaves/<name>
587 be configured for a PFS slave, where
593 is an appropriate label.
594 You can control snapshot retention on your slave independent of the master.
596 Zero out the snapshots directory path for this PFS.
598 .\" ==== pfs-upgrade ====
599 .It Ar pfs-upgrade Ar dirpath
600 Upgrade a PFS from slave to master operation.
601 The PFS will be rolled back to the current end synchronization tid
602 (removing any partial synchronizations), and will then become writable.
606 currently supports only single masters and using
607 this command can easily result in file system corruption
608 if you don't know what you are doing.
610 This directive will refuse to run if any programs have open descriptors
611 in the PFS, including programs chdir'd into the PFS.
612 .\" ==== pfs-downgrade ====
613 .It Ar pfs-downgrade Ar dirpath
614 Downgrade a master PFS from master to slave operation
615 The PFS becomes read-only and access will be locked to its
618 This directive will refuse to run if any programs have open descriptors
619 in the PFS, including programs chdir'd into the PFS.
620 .\" ==== pfs-destroy ====
621 .It Ar pfs-destroy Ar dirpath
622 This permanently destroys a PFS.
624 This directive will refuse to run if any programs have open descriptors
625 in the PFS, including programs chdir'd into the PFS.
626 .\" ==== mirror-read ====
627 .It Ar mirror-read Ar filesystem Op Ar <begin-tid>
628 Generate a mirroring stream to stdout.
629 The stream ends when the transaction id space has been exhausted.
630 .\" ==== mirror-read-stream ====
631 .It Ar mirror-read-stream Ar filesystem Op Ar <begin-tid>
632 Generate a mirroring stream to stdout.
633 Upon completion the stream is paused until new data is synced to the
634 master, then resumed.
635 Operation continues until the pipe is broken.
636 .\" ==== mirror-write ====
637 .It Ar mirror-write Ar filesystem
638 Take a mirroring stream on stdin.
640 This command will fail if the
642 configuration field for the two file systems do not match.
644 If the target PFS does not exist this command will ask you whether
645 you want to create a compatible PFS slave for the target or not.
646 .\" ==== mirror-dump ====
652 to dump an ASCII representation of the mirroring stream.
653 .\" ==== mirror-copy ====
654 .It Ar mirror-copy Ar [[user@]host:]filesystem Ar [[user@]host:]filesystem
655 This is a shortcut which pipes a
660 If a remote host specification is made the program forks a
666 on the appropriate host.
667 The source may be a master or slave PFS, and the target must be a slave PFS.
669 This command also established full duplex communication and turns on
670 the two-way protocol feature which automatically negotiates transaction id
671 ranges without having to use a cyclefile.
672 If the operation completes successfully the target PFS's
675 Note that you must re-chdir into the target PFS to see the updated information.
676 If you do not you will still be in the previous snapshot.
678 If the target PFS does not exist this command will ask you whether
679 you want to create a compatible PFS slave for the target or not.
680 .\" ==== mirror-stream ====
681 .It Ar mirror-stream Ar [[user@]host:]filesystem Ar [[user@]host:]filesystem
682 This command works similarly to
684 but does not exit unless the pipe is broken.
685 This command will resume the mirroring operation whenever the master is synced.
686 The command is commonly used with
690 options to keep the mirroring target in sync with the source on a continuing
692 .\" ==== version ====
693 .It Ar version Ar filesystem
694 This command returns the
696 filesystem version for the specified
697 filesystem as well as the range of versions supported in the kernel.
700 option may be used to remove the summary at the end.
701 .\" ==== version-upgrade ====
702 .It Ar version-upgrade Ar filesystem Ar version Op Ar force
703 This command upgrades the
705 filesystem to the specified version.
706 Once upgraded a filesystem may not be downgraded.
707 If you wish to upgrade a filesystem to a version greater or equal to the
708 work-in-progress version number you must specify the
711 Use of WIP versions should be relegated to testing and may require wiping
712 the filesystem as development progresses, even though the WIP version might
715 NOTE! This command operates on the entire
717 filesystem and is not a per-PFS operation.
718 All PFS's will be affected.
719 .Bl -tag -width indent
722 default version, first
726 Work-in-progress version.
727 This version is developing a new directory hash key.
731 .Sh PSEUDO FILESYSTEM (PFS) NOTES
732 The root of a PFS is not hooked into the primary
734 file system as a directory.
737 creates a special softlink called "@@PFS%05d" (exactly 10 characters long)
742 then modifies the contents of the softlink as read by
744 and thus what you see with an
746 command or if you were to
749 If the PFS is a master the link reflects the current state of the PFS.
750 If the PFS is a slave the link reflects the last completed snapshot, and the
751 contents of the link will change when the next snapshot is completed, and
754 PFS support is currently very new and experimental.
757 utility employs numerous safeties to reduce user foot-shooting.
760 directive requires that the target be configured as a slave and that the
762 field of the mirroring source and target match.
764 .Bl -tag -width ".It Pa <fs>/var/slaves/<name>" -compact
766 default per PFS snapshots directory
767 .It Pa <snapshots>/config
771 .It Pa <fs>/var/slaves/<name>
772 recommended slave PFS snapshots directory
779 .Xr periodic.conf 5 ,
786 utility first appeared in
789 .An Matthew Dillon Aq dillon@backplane.com