1 .\" Copyright (c) 2007 The DragonFly Project. All rights reserved.
3 .\" This code is derived from software contributed to The DragonFly Project
4 .\" by Matthew Dillon <dillon@backplane.com>
6 .\" Redistribution and use in source and binary forms, with or without
7 .\" modification, are permitted provided that the following conditions
10 .\" 1. Redistributions of source code must retain the above copyright
11 .\" notice, this list of conditions and the following disclaimer.
12 .\" 2. Redistributions in binary form must reproduce the above copyright
13 .\" notice, this list of conditions and the following disclaimer in
14 .\" the documentation and/or other materials provided with the
16 .\" 3. Neither the name of The DragonFly Project nor the names of its
17 .\" contributors may be used to endorse or promote products derived
18 .\" from this software without specific, prior written permission.
20 .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
21 .\" ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
22 .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
23 .\" FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
24 .\" COPYRIGHT HOLDERS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
25 .\" INCIDENTAL, SPECIAL, EXEMPLARY OR CONSEQUENTIAL DAMAGES (INCLUDING,
26 .\" BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
27 .\" LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED
28 .\" AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
29 .\" OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
30 .\" OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
33 .\" $DragonFly: src/sbin/hammer/hammer.8,v 1.54 2008/10/07 22:28:41 thomas Exp $
34 .Dd September 28, 2008
39 .Nd HAMMER file system utility
45 .Op Fl f Ar blkdev[:blkdev]*
46 .\" .Op Fl s Ar linkpath
52 This manual page documents the
54 utility which provides miscellaneous functions related to managing a
57 For a general introduction to the
59 file system, its features, and
60 examples on how to set up and maintain one, see
63 The options are as follows:
64 .Bl -tag -width indent
68 Tell the mirror commands to use a 2-way protocol, which allows
69 automatic negotiation of transaction id ranges.
70 This option is automatically enabled by the
74 Specify a bandwidth limit in bytes per second for mirroring streams.
75 This option is typically used to prevent batch mirroring operations from
76 loading down the machine.
77 The bandwidth may be suffixed with
82 to specify values in kilobytes, megabytes, and gigabytes per second.
84 When pruning and reblocking you can instruction
86 to start at the object id stored in the specified file.
87 If the file does not exist
89 will start at the beginning.
93 specific period of time and is unable to complete the operation it will
94 write out the current object id so the next run can pick up where it left off.
97 runs to completion it will delete
99 .It Fl f Ar blkdev[:blkdev]*
100 Specify the volumes making up a
104 When maintaining a streaming mirroring this option specifies the
105 minimum delay after a batch ends before the next batch is allowed
107 The default is five seconds.
109 Decrease verbosement.
110 May be specified multiple times.
112 Specify recursion for those commands which support it.
114 When pruning and reblocking you can tell the utility to stop after a
115 certain period of time.
116 This option is used along with the
118 option to prune or reblock a portion of the file system incrementally.
120 Increase verboseness.
121 May be specified multiple times.
124 The commands are as follows:
125 .Bl -tag -width indent
126 .\" ==== synctid ====
127 .It Ar synctid Ar filesystem Op quick
128 Generates a guaranteed, formal 64 bit transaction id representing the
129 current state of the specified
132 The file system will be synced to the media.
136 keyword is specified the file system will be soft-synced, meaning that a
137 crash might still undo the state of the file system as of the transaction
138 id returned but any new modifications will occur after the returned
139 transaction id as expected.
141 .It Ar bstats Op interval
144 B-tree statistics until interrupted.
147 seconds between each display.
148 The default interval is one second.
149 .\" ==== iostats ====
150 .It Ar iostats Op interval
153 I/O statistics until interrupted.
156 seconds between each display.
157 The default interval is one second.
158 .\" ==== history ====
159 .It Ar history Ar path ...
160 Show the modification history for
162 file's inode and data.
166 This command needs the
170 .\" Dump the B-tree, record, large-data, and small-data blockmaps, showing
171 .\" physical block assignments and free space percentages.
172 .\" ==== namekey ====
173 .It Ar namekey Ar filename
176 64 bit directory hash for the specified file name.
177 The low 32 bits are used as an iterator for hash collisions and will be
179 .\" ==== namekey32 ====
180 .It Ar namekey32 Ar filename
181 Generate the top 32 bits of a
183 64 bit directory hash for the specified file name.
184 .\" ==== cleanup ====
185 .It Ar cleanup Op Ar filesystem ...
186 This is a meta-command which executes snapshot, pruning, and reblocking
187 commands on the specified
192 is specified this command will clean-up all
194 file systems in use, including PFS's.
195 To do this it will scan all
199 mounts, extract PFS id's, and clean-up each PFS found.
201 This command will by default access a
207 creating them if necessary.
208 The format of the configuration file is:
209 .Bd -literal -offset indent
210 snapshots <period> <retensiontime>
211 prune <period> <max-runtime>
212 reblock <period> <1/3 max-runtime>
213 recopy <period> <1/3 max-runtime>
216 snapshots 1d 60d # 0d 60d for PFS /tmp, /var/tmp, /usr/obj
222 Time is given with a suffix of
228 meaning day, hour, minute and second.
229 A snapshots period of 0 disables snapshot generation and prunes using
230 .Ar prune-everything ,
231 if no snapshots exists.
232 A prune max-runtime of 0 means unlimited.
234 If period hasn't passed since previous
237 For example a day has passed when midnight is passed (localtime).
238 It is recommended to run
240 once a day, depending on the configured period, from a
244 .Bd -literal -offset indent
245 15 2 * * * hammer cleanup >>/var/log/hammer.log 2>&1
248 The default configuration file will create a daily snapshot, do a daily
249 pruning and reblocking run and a monthly recopy run.
250 Reblocking is defragmentation with a level of 95%,
251 and recopy is full defragmentation.
253 By default prune and reblock operations are limited to 5 minutes per function,
254 and recopy operations are limited to 10 minutes per function.
255 Reblocking and recopy runs are each broken down into three separate functions
256 (btree, inodes and data)
257 and are thus by default limited to 15 and 30 minutes respectively.
258 Also note that this directive will by default disable snapshots on
265 The defaults may be adjusted by modifying the
268 The pruning and reblocking commands automatically maintain a cyclefile
269 for incremental operation.
270 If you interrupt (^C) the program the cyclefile will be updated, but a sub-command
271 may continue to run in the background for a few seconds until the
273 ioctl detects the interrupt.
276 PFS option can be set to use another location for the snapshots directory.
278 Work on this command is still in progress.
279 Expected additions: An ability to remove snapshots dynamically as the
280 file system becomes full.
282 .It Ar prune Ar softlink-dir
283 Prune the file system based on previously created snapshot softlinks.
284 Pruning is the act of deleting file system history.
288 will delete file system history such that
289 the file system state is retained for the given snapshots,
290 and all history after the latest snapshot,
291 but all other history is deleted.
293 The target directory is expected to contain softlinks pointing to
294 snapshots of the file systems you wish to retain.
295 The directory is scanned non-recursively and the mount points and
296 transaction ids stored in the softlinks are extracted and sorted.
297 The file system is then explicitly pruned according to what is found.
298 Cleaning out portions of the file system is as simple as removing a softlink
303 As a safety measure pruning only occurs if one or more softlinks are found
304 containing the @@ snapshot id extension.
305 Currently the scanned softlink directory must contain softlinks pointing
309 The softlinks may specify absolute or relative paths.
310 Softlinks must use 20-character (@@0x%016llx) transaction ids,
311 as might be returned from
312 .Dq Nm Ar synctid filesystem .
314 Pruning is a per PFS operation, so a
316 file system and each PFS in it have to be pruned separately.
318 Note that pruning a file system may not immediately free-up space,
319 though typically some space will be freed if a large number of records are
321 The file system must be reblocked to completely recover all available space.
323 Example, lets say your snapshot directory contains the following links:
324 .Bd -literal -offset indent
325 lrwxr-xr-x 1 root wheel 29 May 31 17:57 snap1 ->
326 /usr/obj/@@0x10d2cd05b7270d16
328 lrwxr-xr-x 1 root wheel 29 May 31 17:58 snap2 ->
329 /usr/obj/@@0x10d2cd13f3fde98f
331 lrwxr-xr-x 1 root wheel 29 May 31 17:59 snap3 ->
332 /usr/obj/@@0x10d2cd222adee364
335 If you were to run the
337 command on this directory, then the
340 mount will be pruned to retain the above three snapshots.
341 In addition, history for modifications made to the file system older than
342 the oldest snapshot will be destroyed and history for potentially fine-grained
343 modifications made to the file system more recently than the most recent
344 snapshot will be retained.
346 If you then delete the
348 softlink and rerun the
351 history for modifications pertaining to that snapshot would be destroyed.
352 .\" ==== prune-everything ====
353 .It Ar prune-everything Ar filesystem
354 This command will remove all historical records from the file system.
355 This directive is not normally used on a production system.
356 .\" ==== snapshot ====
357 .It Ar snapshot Ar snapshot-dir
358 .It Ar snapshot Ar filesystem snapshot-dir
359 Takes a snapshot of the file system either explicitly given by
361 or implicitly derived from the
363 argument and creates a symlink in the directory provided by
365 pointing to the snapshot.
368 is not a directory, it is assumed to be a format string passed to
370 with the current time as parameter.
373 refers to an existing directory, a default format string of "snap-%Y%d%m-%H%M"
374 is assumed and used as name for the newly created symlink.
376 Snapshot is a per PFS operation, so a
378 file system and each PFS in it have to be snapshot separately.
380 Example, assuming that
386 is a file system on its own, the following invocations:
387 .Bd -literal -offset indent
388 hammer snapshot /mysnapshots
390 hammer snapshot /mysnapshots/%Y-%m-%d
392 hammer snapshot /obj /mysnapshots/obj-%Y-%m-%d
395 would create symlinks similar to:
396 .Bd -literal -offset indent
397 /mysnapshots/snap-20080627-1210 -> /@@0x10d2cd05b7270d16
399 /mysnapshots/2008-06-27 -> /@@0x10d2cd05b7270d16
401 /mysnapshots/obj-2008-06-27 -> /obj@@0x10d2cd05b7270d16
403 .\" ==== reblock ====
404 .It Ar reblock Ar filesystem Op Ar fill_percentage
405 .It Ar reblock-btree Ar filesystem Op Ar fill_percentage
406 .It Ar reblock-inodes Ar filesystem Op Ar fill_percentage
407 .It Ar reblock-dirs Ar filesystem Op Ar fill_percentage
408 .It Ar reblock-data Ar filesystem Op Ar fill_percentage
409 Attempt to defragment and free space for reuse by reblocking a live
412 Big blocks cannot be reused by
414 until they are completely free.
415 This command also has the effect of reordering all elements, effectively
416 defragmenting the file system.
418 The default fill percentage is 100% and will cause the file system to be
419 completely defragmented.
420 All specified element types will be reallocated and rewritten.
421 If you wish to quickly free up space instead try specifying
422 a smaller fill percentage, such as 90% or 80% (the
424 suffix is not needed).
426 Since this command may rewrite the entire contents of the disk it is
427 best to do it incrementally from a
433 options to limit the run time.
434 The file system would thus be defragmented over long period of time.
436 It is recommended that separate invocations be used for each data type.
437 B-tree nodes, inodes, and directories are typically the most important
438 elements needing defragmentation.
439 Data can be defragmented over a longer period of time.
441 Reblocking is a per PFS operation, so a
443 file system and each PFS in it have to be reblocked separately.
444 .\" ==== pfs-status ====
445 .It Ar pfs-status Ar dirpath ...
446 Retrieve the mirroring configuration parameters for the specified
448 file systems or pseudo-filesystems (PFS's).
449 .\" ==== pfs-master ====
450 .It Ar pfs-master Ar dirpath Op options
451 Create a pseudo-filesystem (PFS) inside a
454 Up to 65535 such file systems can be created.
455 Each PFS uses an independent inode numbering space making it suitable
456 for use as a replication source or target.
460 directive creates a PFS that you can read, write, and use as a mirroring
463 It is recommended to use a
465 mount to access a PFS, for more information see
467 .\" ==== pfs-slave ====
468 .It Ar pfs-slave Ar dirpath Op options
469 Create a pseudo-filesystem (PFS) inside a
472 Up to 65535 such file systems can be created.
473 Each PFS uses an independent inode numbering space making it suitable
474 for use as a replication source or target.
478 directive creates a PFS that you can use as a mirroring target.
479 You will not be able to access a slave PFS until you have completed the
480 first mirroring operation with it as the target (its root directory will
481 not exist until then).
483 Access to the pfs-slave via the special softlink,
489 dynamically modify the snapshot transaction id by returning a dynamic result
494 A PFS can only be truly destroyed with the
497 Removing the softlink will not destroy the underlying PFS.
499 It is recommended to use a
501 mount to access a PFS, for more information see
503 .\" ==== pfs-update ====
504 .It Ar pfs-update Ar dirpath Op options
505 Update the configuration parameters for an existing
508 or pseudo-filesystem.
509 Options that may be specified:
510 .Bl -tag -width indent
511 .It sync-beg-tid=0x16llx
512 This is the automatic snapshot access starting transaction id for
514 This parameter is normally updated automatically by the
518 It is important to note that accessing a mirroring slave
519 with a transaction id greater than the last fully synchronized transaction
520 id can result in an unreliable snapshot since you will be accessing
521 data that is still undergoing synchronization.
523 Manually modifying this field is dangerous and can result in a broken
525 .It sync-end-tid=0x16llx
526 This is the current synchronization point for mirroring slaves.
527 This parameter is normally updated automatically by the
531 Manually modifying this field is dangerous and can result in a broken mirror.
532 .It shared-uuid=<uuid>
533 Set the shared UUID for this file system.
534 All mirrors must have the same shared UUID.
535 For safety purposes the
537 directives will refuse to operate on a target with a different shared UUID.
539 Changing the shared UUID on an existing, non-empty mirroring target,
540 including an empty but not completely pruned target,
541 can lead to corruption of the mirroring target.
542 .It unique-uuid=<uuid>
543 Set the unique UUID for this file system.
544 This UUID should not be used anywhere else,
545 even on exact copies of the file system.
547 Set a descriptive label for this file system.
548 .It snapshots=<string>
549 Specify the snapshots directory which
552 will use to manage this PFS.
553 The snapshots directory does not need to be configured for
554 PFS masters and will default to
555 .Pa <pfs>/snapshots .
557 PFS slaves are mirroring slaves so you cannot configure a snapshots
558 directory on the slave itself to be managed by the slave's machine.
559 In fact, the slave will likely have a
561 sub-directory mirrored
562 from the master, but that directory contains the configuration the master
563 is using for its copy of the file system, not the configuration that we
564 want to use for our slave.
566 It is recommended that
567 .Pa <fs>/var/slaves/<name>
568 be configured for a PFS slave, where
574 is an appropriate label.
575 You can control snapshot retention on your slave independent of the master.
577 Zero out the snapshots directory path for this PFS.
579 .\" ==== pfs-upgrade ====
580 .It Ar pfs-upgrade Ar dirpath
581 Upgrade a PFS from slave to master operation.
582 The PFS will be rolled back to the current end synchronization tid
583 (removing any partial synchronizations), and will then become writable.
587 currently supports only single masters and using
588 this command can easily result in file system corruption
589 if you don't know what you are doing.
591 This directive will refuse to run if any programs have open descriptors
592 in the PFS, including programs chdir'd into the PFS.
593 .\" ==== pfs-downgrade ====
594 .It Ar pfs-downgrade Ar dirpath
595 Downgrade a master PFS from master to slave operation
596 The PFS becomes read-only and access will be locked to its
599 This directive will refuse to run if any programs have open descriptors
600 in the PFS, including programs chdir'd into the PFS.
601 .\" ==== pfs-destroy ====
602 .It Ar pfs-destroy Ar dirpath
603 This permanently destroys a PFS.
605 This directive will refuse to run if any programs have open descriptors
606 in the PFS, including programs chdir'd into the PFS.
607 .\" ==== mirror-read ====
608 .It Ar mirror-read Ar filesystem Op Ar <begin-tid>
609 Generate a mirroring stream to stdout.
610 The stream ends when the transaction id space has been exhausted.
611 .\" ==== mirror-read-stream ====
612 .It Ar mirror-read-stream Ar filesystem Op Ar <begin-tid>
613 Generate a mirroring stream to stdout.
614 Upon completion the stream is paused until new data is synced to the
615 master, then resumed.
616 Operation continues until the pipe is broken.
617 .\" ==== mirror-write ====
618 .It Ar mirror-write Ar filesystem
619 Take a mirroring stream on stdin.
621 This command will fail if the
623 configuration field for the two file systems do not match.
624 .\" ==== mirror-dump ====
630 to dump an ASCII representation of the mirroring stream.
631 .\" ==== mirror-copy ====
632 .It Ar mirror-copy Ar [[user@]host:]filesystem Ar [[user@]host:]filesystem
633 This is a shortcut which pipes a
638 If a remote host specification is made the program forks a
644 on the appropriate host.
645 The source may be a master or slave PFS, and the target must be a slave PFS.
647 This command also established full duplex communication and turns on
648 the two-way protocol feature which automatically negotiates transaction id
649 ranges without having to use a cyclefile.
650 If the operation completes successfully the target PFS's
653 Note that you must re-chdir into the target PFS to see the updated information.
654 If you do not you will still be in the previous snapshot.
655 .\" ==== mirror-stream ====
656 .It Ar mirror-stream Ar [[user@]host:]filesystem Ar [[user@]host:]filesystem
657 This command works similarly to
659 but does not exit unless the pipe is broken.
660 This command will resume the mirroring operation whenever the master is synced.
661 The command is commonly used with
665 options to keep the mirroring target in sync with the source on a continuing
669 .Sh PSEUDO FILESYSTEM (PFS) NOTES
670 The root of a PFS is not hooked into the primary
672 file system as a directory.
675 creates a special softlink called "@@PFS%05d" (exactly 10 characters long)
680 then modifies the contents of the softlink as read by
682 and thus what you see with an
684 command or if you were to
687 If the PFS is a master the link reflects the current state of the PFS.
688 If the PFS is a slave the link reflects the last completed snapshot, and the
689 contents of the link will change when the next snapshot is completed, and
692 PFS support is currently very new and experimental.
695 utility employs numerous safeties to reduce user foot-shooting.
698 directive requires that the target be configured as a slave and that the
700 field of the mirroring source and target match.
704 .Bl -tag -width ".It Pa <fs>/var/slaves/<name>" -compact
706 default per PFS snapshots directory
707 .It Pa <snapshots>/config
711 .It Pa <fs>/var/slaves/<name>
712 recommended slave PFS snapshots directory
723 utility first appeared in
726 .An Matthew Dillon Aq dillon@backplane.com