1 .\" Copyright (c) 2007 The DragonFly Project. All rights reserved.
3 .\" This code is derived from software contributed to The DragonFly Project
4 .\" by Matthew Dillon <dillon@backplane.com>
6 .\" Redistribution and use in source and binary forms, with or without
7 .\" modification, are permitted provided that the following conditions
10 .\" 1. Redistributions of source code must retain the above copyright
11 .\" notice, this list of conditions and the following disclaimer.
12 .\" 2. Redistributions in binary form must reproduce the above copyright
13 .\" notice, this list of conditions and the following disclaimer in
14 .\" the documentation and/or other materials provided with the
16 .\" 3. Neither the name of The DragonFly Project nor the names of its
17 .\" contributors may be used to endorse or promote products derived
18 .\" from this software without specific, prior written permission.
20 .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
21 .\" ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
22 .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
23 .\" FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
24 .\" COPYRIGHT HOLDERS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
25 .\" INCIDENTAL, SPECIAL, EXEMPLARY OR CONSEQUENTIAL DAMAGES (INCLUDING,
26 .\" BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
27 .\" LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED
28 .\" AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
29 .\" OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
30 .\" OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
33 .\" $DragonFly: src/sbin/hammer/hammer.8,v 1.49 2008/09/28 21:27:56 thomas Exp $
34 .Dd September 28, 2008
39 .Nd HAMMER file system utility
45 .Op Fl f Ar blkdev[:blkdev]*
46 .\" .Op Fl s Ar linkpath
52 This manual page documents the
54 utility which provides miscellaneous functions related to managing a
57 For a general introduction to the
59 file system, its features, and
60 examples on how to set up and maintain one, see
63 The options are as follows:
64 .Bl -tag -width indent
68 Tell the mirror commands to use a 2-way protocol, which allows
69 automatic negotiation of transaction id ranges. This option is
70 automatically enabled by the
74 Specify recursion for those commands which support it.
76 Specify a bandwidth limit in bytes per second for mirroring streams.
77 This option is typically used to prevent batch mirroring operations from
78 loading down the machine.
79 The bandwidth may be suffixed with
85 values in kilobytes, megabytes, and gigabytes per second.
87 When pruning and reblocking you can instruction
90 object id stored in the specified file.
91 If the file does not exist
93 will start at the beginning.
97 specific period of time and is unable to complete the operation it will
98 write out the current object id so the next run can pick up where it left
102 runs to completion it will delete the cyclefile.
103 .It Fl f Ar blkdev[:blkdev]*
104 Specify the volumes making up a
108 When maintaining a streaming mirroring this option specifies the
109 minimum delay after a batch ends before the next batch is allowed
111 The default is five seconds.
113 When pruning and reblocking you can tell the utility to stop after a
114 certain period of time. This option is used along with the cycle file
115 option to prune or reblock a portion of the file system incrementally.
117 Increase verboseness. May be specified multiple times.
120 The commands are as follows:
121 .Bl -tag -width indent
122 .\" ==== synctid ====
123 .It Ar synctid Ar filesystem Op quick
124 Generates a guaranteed, formal 64 bit transaction id representing the
125 current state of the specified
127 file system. The file system will
128 be synced to the media.
132 keyword is specified the file system will be soft-synced, meaning that a
133 crash might still undo the state of the file system as of the transaction
134 id returned but any new modifications will occur after the returned
135 transaction id as expected.
137 .It Ar bstats Op interval
140 B-tree statistics until interrupted.
143 seconds between each display.
144 The default interval is one second.
145 .\" ==== iostats ====
146 .It Ar iostats Op interval
149 I/O statistics until interrupted.
152 seconds between each display.
153 The default interval is one second.
154 .\" ==== history ====
155 .It Ar history Ar path ...
156 Show the modification history for
158 file's inode and data.
161 Dump the B-tree. This command needs the
165 .\" Dump the B-tree, record, large-data, and small-data blockmaps, showing
166 .\" physical block assignments and free space percentages.
167 .\" ==== namekey ====
168 .It Ar namekey Ar filename
171 64 bit directory hash for the specified file name.
172 The low 32 bits are used as an iterator for hash collisions and will be
174 .\" ==== namekey32 ====
175 .It Ar namekey32 Ar filename
176 Generate the top 32 bits of a
178 64 bit directory hash for the specified
180 .\" ==== cleanup ====
181 .It Ar cleanup Op Ar filesystem ...
182 This is a meta-command which executes snapshot, pruning, and reblocking
183 commands on the specified
188 is specified this command will clean-up all
190 file systems in use, including PFS's.
191 To do this it will scan all
195 mounts, extract PFS id's, and clean-up each PFS found.
197 This command will by default access a
203 creating them if necessary.
204 The format of the configuration file is:
205 .Bd -literal -offset indent
206 snapshots <period> <retensiontime>
207 prune <period> <max-runtime>
208 reblock <period> <1/3 max-runtime>
209 recopy <period> <1/3 max-runtime>
212 snapshots 1d 60d # 0 60d for PFS /tmp, /var/tmp, /usr/obj
218 Time is in seconds, it can be given with a suffix of
224 meaning day, hour, minute and second.
225 A snapshots period of 0 disables snapshot generation and prunes using
226 .Ar prune-everything ,
227 if no snapshots exists.
228 A prune max-runtime of 0 means unlimited.
230 It is recommended to run
232 once a day, depending on the configured period,
237 .Bd -literal -offset indent
238 15 2 * * * hammer cleanup >>/var/log/hammer.log 2>&1
241 The default configuration file will create a daily snapshot, do a daily
242 pruning and reblocking run and a monthly recopy run.
243 Reblocking is defragmentation with a level of 95%,
244 and recopy is full defragmentation.
246 By default prune and reblock operations are limited to 5 minutes per function,
247 and recopy operations are limited to 10 minutes per function.
248 Reblocking and recopy runs are each broken down into three separate functions
249 (btree, inodes and data)
250 and are thus by default limited to 15 and 30 minutes respectively.
251 Also note that this directive will by default disable snapshots on
258 The defaults may be adjusted by modifying the
261 The pruning and reblocking commands automatically maintain a cyclefile
262 for incremental operation.
263 If you interrupt (^C) the program the cyclefile will be updated, but a sub-command
264 may continue to run in the background for a few seconds until the
266 ioctl detects the interrupt.
269 PFS option can be set to use another location for the snapshots directory.
271 Work on this command is still in progress.
272 Expected additions: An ability to remove snapshots dynamically as the
273 file system becomes full.
275 .It Ar prune Ar softlink-dir
276 Prune the file system based on previously created snapshot softlinks.
277 Pruning is the act of deleting file system history.
281 will delete file system history such that
282 the file system state is retained for the given snapshots,
283 and all history after the latest snapshot,
284 but all other history is deleted.
286 The target directory is expected to contain softlinks pointing to
287 snapshots of the file systems you wish to retain. The directory is scanned
288 non-recursively and the mount points and transaction ids stored in the
289 softlinks are extracted and sorted.
290 The file system is then explicitly pruned according to what is found.
291 Cleaning out portions of the file system is as simple as removing a softlink
296 As a safety measure pruning only occurs if one or more softlinks are found
297 containing the @@ snapshot id extension.
298 Currently the scanned softlink directory must contain softlinks pointing
301 mount. The softlinks may specify absolute or relative
302 paths. Softlinks must use 20-character (@@0x%016llx) transaction ids,
303 as might be returned from
304 .Dq Nm Ar synctid filesystem .
306 Pruning is a per PFS operation, so a
308 file system and each PFS in it have to be pruned separately.
310 Note that pruning a file system may not immediately free-up space,
311 though typically some space will be freed if a large number of records are
312 pruned out. The file system must be reblocked to completely recover all
315 Example, lets say your snapshot directory contains the following links:
316 .Bd -literal -offset indent
317 lrwxr-xr-x 1 root wheel 29 May 31 17:57 snap1 ->
318 /usr/obj/@@0x10d2cd05b7270d16
320 lrwxr-xr-x 1 root wheel 29 May 31 17:58 snap2 ->
321 /usr/obj/@@0x10d2cd13f3fde98f
323 lrwxr-xr-x 1 root wheel 29 May 31 17:59 snap3 ->
324 /usr/obj/@@0x10d2cd222adee364
327 If you were to run the
329 command on this directory, then the
332 mount will be pruned to retain the above three snapshots.
333 In addition, history for modifications made to the file system older than the oldest
334 snapshot will be destroyed and history for potentially fine-grained modifications made
335 to the file system more recently than the most recent snapshot will be
338 If you then delete the snap2 softlink and rerun the
341 history for modifications pertaining to that snapshot would be destroyed.
342 .\" ==== prune-everything ====
343 .It Ar prune-everything Ar filesystem
344 This command will remove all historical records from the file system.
345 This directive is not normally used on a production system.
346 .\" ==== snapshot ====
347 .It Ar snapshot Ar snapshot-dir
348 .It Ar snapshot Ar filesystem snapshot-dir
349 Takes a snapshot of the file system either explicitly given by
351 or implicitly derived from the
353 argument and creates a symlink in the directory provided by
355 pointing to the snapshot.
358 is not a directory, it is assumed to be a format string
361 with the current time as parameter.
364 refers to an existing directory, a default format string of "snap-%Y%d%m-%H%M"
365 is assumed and used as name for the newly created symlink.
367 Snapshot is a per PFS operation, so a
369 file system and each PFS in it have to be snapshot separately.
371 Example, assuming that
377 is a file system on its own, the following invocations:
378 .Bd -literal -offset indent
379 hammer snapshot /mysnapshots
381 hammer snapshot /mysnapshots/%Y-%m-%d
383 hammer snapshot /obj /mysnapshots/obj-%Y-%m-%d
386 would create symlinks similar to:
387 .Bd -literal -offset indent
388 /mysnapshots/snap-20080627-1210 -> /@@0x10d2cd05b7270d16
390 /mysnapshots/2008-06-27 -> /@@0x10d2cd05b7270d16
392 /mysnapshots/obj-2008-06-27 -> /obj@@0x10d2cd05b7270d16
394 .\" ==== reblock ====
395 .It Ar reblock Ar filesystem Op Ar fill_percentage
396 .It Ar reblock-btree Ar filesystem Op Ar fill_percentage
397 .It Ar reblock-inodes Ar filesystem Op Ar fill_percentage
398 .It Ar reblock-dirs Ar filesystem Op Ar fill_percentage
399 .It Ar reblock-data Ar filesystem Op Ar fill_percentage
400 Attempt to defragment and free space for reuse by reblocking a live
403 Big blocks cannot be reused by
405 until they are completely free.
406 This command also has the effect of reordering all elements, effectively
407 defragmenting the file system.
409 The default fill percentage is 100% and will cause the file system to be
410 completely defragmented. All specified element types will be reallocated
411 and rewritten. If you wish to quickly free up space instead try specifying
412 a smaller fill percentage, such as 90% or 80% (the
414 suffix is not needed).
416 Since this command may rewrite the entire contents of the disk it is
417 best to do it incrementally from a
423 options to limit the run time.
424 The file system would thus be defragmented over long period of time.
426 It is recommended that separate invocations be used for each data type.
427 B-tree nodes, inodes, and directories are typically the most important
428 elements needing defragmentation. Data can be defragmented over a longer
431 Reblocking is a per PFS operation, so a
433 file system and each PFS in it have to be reblocked separately.
434 .\" ==== pfs-status ====
435 .It Ar pfs-status Ar dirpath ...
436 Retrieve the mirroring configuration parameters for the specified
438 file systems or pseudo-filesystems.
439 .\" ==== pfs-master ====
440 .It Ar pfs-master Ar dirpath Op options
441 Create a pseudo-filesystem (PFS) inside a
444 Up to 65535 such file systems can be created.
445 Each PFS uses an independent inode numbering space making it suitable
446 for use as a replication source or target.
450 directive creates a PFS that you can read, write, and use as a mirroring
452 .\" ==== pfs-slave ====
453 .It Ar pfs-slave Ar dirpath Op options
454 Create a pseudo-filesystem (PFS) inside a
457 Up to 65535 such file systems can be created.
458 Each PFS uses an independent inode numbering space making it suitable
459 for use as a replication source or target.
463 directive creates a PFS that you can use as a mirroring target.
464 You will not be able to access a slave PFS until you have completed the
465 first mirroring operation with it as the target (its root directory will
466 not exist until then).
468 Access to the pfs-slave via the special softlink,
469 as described in the PFS NOTES below, allows
472 dynamically modify the snapshot transaction id by returning a dynamic result
477 A PFS can only be truly destroyed with the
480 Removing the softlink will not destroy the underlying PFS.
481 .\" ==== pfs-update ====
482 .It Ar pfs-update Ar dirpath Op options
483 Update the configuration parameters for an existing
486 or pseudo-filesystem. Options that may be specified:
487 .Bl -tag -width indent
488 .It sync-beg-tid=0x16llx
489 This is the automatic snapshot access starting transaction id for mirroring slaves.
490 This parameter is normally updated automatically by the
494 It is important to note that accessing a mirroring slave
495 with a transaction id greater than the last fully synchronized transaction
496 id can result in an unreliable snapshot since you will be accessing
497 data that is still undergoing synchronization.
499 Manually modifying this field is dangerous and can result in a broken
501 .It sync-end-tid=0x16llx
502 This is the current synchronization point for mirroring slaves.
503 This parameter is normally updated automatically by the
507 Manually modifying this field is dangerous and can result in a broken
509 .It shared-uuid=<uuid>
510 Set the shared UUID for this file system. All mirrors must have the same
511 shared UUID. For safety purposes the
513 directives will refuse
514 to operate on a target with a different shared UUID.
516 Changing the shared UUID on an existing, non-empty mirroring target,
517 including an empty but not completely pruned target, can lead
518 to corruption of the mirroring target.
519 .It unique-uuid=<uuid>
520 Set the unique UUID for this file system. This UUID should not be used
521 anywhere else, even on exact copies of the file system.
523 Set a descriptive label for this file system.
524 .It snapshots=<string>
525 Specify the snapshots directory which
529 this PFS. The snapshots directory does not need to be configured for
530 PFS masters and will default to
531 .Pa <pfs>/snapshots .
533 PFS slaves are mirroring slaves so you cannot configure a snapshots
534 directory on the slave itself to be managed by the slave's machine.
535 In fact, the slave will likely have a
537 sub-directory mirrored
538 from the master, but that directory contains the configuration the master
539 is using for its copy of the file system, not the configuration that we
540 want to use for our slave.
542 It is recommended that
543 .Pa <fs>/var/slaves/<name>
544 be configured for a PFS slave, where
550 is an appropriate label.
551 You can control snapshot
552 retention on your slave independent of the master.
554 Zero out the snapshots directory path for this PFS.
556 .\" ==== pfs-upgrade ====
557 .It Ar pfs-upgrade Ar dirpath
558 Upgrade a PFS from slave to master operation. The PFS will be rolled back
559 to the current end synchronization tid (removing any partial synchronizations),
560 and will then becomes writable.
564 currently supports only single masters and using
565 this command can easily result in file system corruption if you don't
566 know what you are doing.
568 This directive will refuse to run if any programs have open descriptors
569 in the PFS, including programs chdir'd into the PFS.
570 .\" ==== pfs-downgrade ====
571 .It Ar pfs-downgrade Ar dirpath
572 Downgrade a master PFS from master to slave operation. The PFS becomes
573 read-only and access will be locked to its
576 This directive will refuse to run if any programs have open descriptors
577 in the PFS, including programs chdir'd into the PFS.
578 .\" ==== pfs-destroy ====
579 .It Ar pfs-destroy Ar dirpath
580 This permanently destroys a PFS.
582 This directive will refuse to run if any programs have open descriptors
583 in the PFS, including programs chdir'd into the PFS.
584 .\" ==== mirror-read ====
585 .It Ar mirror-read Ar filesystem Op Ar <begin-tid>
586 Generate a mirroring stream to stdout.
587 The stream ends when the transaction id space has been exhausted.
588 .\" ==== mirror-read-stream ====
589 .It Ar mirror-read-stream Ar filesystem Op Ar <begin-tid>
590 Generate a mirroring stream to stdout.
591 Upon completion the stream is paused until new data is synced to the
592 master, then resumed.
593 Operation continues until the pipe is broken.
594 .\" ==== mirror-write ====
595 .It Ar mirror-write Ar filesystem
596 Take a mirroring stream on stdin.
598 This command will fail if the
600 configuration field for the two file systems do not match.
601 .\" ==== mirror-dump ====
608 representation of the mirroring stream.
609 .\" ==== mirror-copy ====
610 .It Ar mirror-copy Ar [[user@]host:]filesystem Ar [[user@]host:]filesystem
611 This is a shortcut which pipes a
615 command. If a remote host specification is made the program forks a
621 on the appropriate host.
622 The source may be a master or slave PFS, and the target must be a slave PFS.
624 This command also established full duplex communication and turns on
625 the two-way protocol feature which automatically negotiates transaction id ranges
626 without having to use a cycle file.
627 If the operation completes successfully the target PFS's
630 be updated. Note that you must re-chdir into the target PFS to see the
631 updated information. If you do not you will still be in the previous snapshot.
632 .\" ==== mirror-stream ====
633 .It Ar mirror-stream Ar [[user@]host:]filesystem Ar [[user@]host:]filesystem
634 This command works similarly to
636 but does not exit unless the pipe is broken.
637 This command will resume the mirroring operation whenever the master is
638 synced. The command is commonly used with
642 options to keep the mirroring target in sync with the source on a continuing
646 .Sh PSEUDO FILESYSTEM (PFS) NOTES
647 The root of a PFS is not hooked into the primary
653 creates a special softlink called "@@PFS%05d" (exactly 10
654 characters long) in the primary
658 then modifies the contents of the softlink as read by
660 and thus what you see with an
662 command or if you were to
665 If the PFS is a master the link reflects the current state of the PFS.
666 If the PFS is a slave the link reflects the last completed snapshot, and the
667 contents of the link will change when the next snapshot is completed, and
670 PFS support is currently very new and experimental. The
673 employs numerous safeties to reduce user foot-shooting.
676 directive requires that the target be configured as a slave and that the
678 field of the mirroring source and target match.
682 .Bl -tag -width ".It Pa <fs>/var/slaves/<name>" -compact
684 default per PFS snapshots directory
685 .It Pa <snapshots>/config
689 .It Pa <fs>/var/slaves/<name>
690 recommended slave PFS snapshots directory
700 utility first appeared in
703 .An Matthew Dillon Aq dillon@backplane.com