.\" Copyright (c) 2007 The DragonFly Project. All rights reserved. .\" .\" This code is derived from software contributed to The DragonFly Project .\" by Matthew Dillon .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" .\" 1. Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" 2. Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in .\" the documentation and/or other materials provided with the .\" distribution. .\" 3. Neither the name of The DragonFly Project nor the names of its .\" contributors may be used to endorse or promote products derived .\" from this software without specific, prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS .\" ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS .\" FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE .\" COPYRIGHT HOLDERS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, .\" INCIDENTAL, SPECIAL, EXEMPLARY OR CONSEQUENTIAL DAMAGES (INCLUDING, .\" BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; .\" LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED .\" AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, .\" OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT .\" OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF .\" SUCH DAMAGE. .\" .\" $DragonFly: src/sbin/hammer/hammer.8,v 1.48 2008/09/24 01:42:50 dillon Exp $ .Dd July 27, 2008 .Dt HAMMER 8 .Os .Sh NAME .Nm hammer .Nd HAMMER file system utility .Sh SYNOPSIS .Nm .Op Fl h2rv .Op Fl b Ar bandwidth .Op Fl c Ar cyclefile .Op Fl f Ar blkdev[:blkdev]* .\" .Op Fl s Ar linkpath .Op Fl i Ar delay .Op Fl t Ar seconds .Ar command .Op Ar argument ... .Sh DESCRIPTION This manual page documents the .Nm utility which provides miscellaneous functions related to managing a .Nm HAMMER file system. For a general introduction to the .Nm HAMMER file system, its features, and examples on how to set up and maintain one, see .Xr HAMMER 5 . .Pp The options are as follows: .Bl -tag -width indent .It Fl h Get help. .It Fl 2 Tell the mirror commands to use a 2-way protocol, which allows automatic negotiation of transaction id ranges. This option is automatically enabled by the .Ar mirror-copy command. .It Fl r Specify recursion for those commands which support it. .It Fl b Ar bandwidth Specify a bandwidth limit in bytes per second for mirroring streams. This option is typically used to prevent batch mirroring operations from loading down the machine. The bandwidth may be suffixed with .Sq k , .Sq m , or .Sq g to specify values in kilobytes, megabytes, and gigabytes per second. .It Fl c Ar cyclefile When pruning and reblocking you can instruction .Nm to start at the object id stored in the specified file. If the file does not exist .Nm will start at the beginning. If .Nm is told to run for a specific period of time and is unable to complete the operation it will write out the current object id so the next run can pick up where it left off. If .Nm runs to completion it will delete the cyclefile. .It Fl f Ar blkdev[:blkdev]* Specify the volumes making up a .Nm HAMMER file system. .\" .It Fl s Ar linkpath .\" When pruning a filesystem you can instruct .\" .Nm to create softlinks .\" to available snapshots. .It Fl i Ar delay When maintaining a streaming mirroring this option specifies the minimum delay after a batch ends before the next batch is allowed to start. The default is five seconds. .It Fl t Ar seconds When pruning and reblocking you can tell the utility to stop after a certain period of time. This option is used along with the cycle file option to prune or reblock a portion of the file system incrementally. .It Fl v Increase verboseness. May be specified multiple times. .El .Pp The commands are as follows: .Bl -tag -width indent .\" ==== synctid ==== .It Ar synctid Ar filesystem Op quick Generates a guaranteed, formal 64 bit transaction id representing the current state of the specified .Nm HAMMER file system. The file system will be synced to the media. .Pp If the .Ar quick keyword is specified the file system will be soft-synced, meaning that a crash might still undo the state of the file system as of the transaction id returned but any new modifications will occur after the returned transaction id as expected. .\" ==== bstats ==== .It Ar bstats Op interval Output .Nm HAMMER B-tree statistics until interrupted. Pause .Ar interval seconds between each display. The default interval is one second. .\" ==== iostats ==== .It Ar iostats Op interval Output .Nm HAMMER I/O statistics until interrupted. Pause .Ar interval seconds between each display. The default interval is one second. .\" ==== history ==== .It Ar history Ar path ... Show the modification history for .Nm HAMMER file's inode and data. .\" ==== show ==== .It Ar show Dump the B-tree. This command needs the .Fl f flag. .\" .It Ar blockmap .\" Dump the B-tree, record, large-data, and small-data blockmaps, showing .\" physical block assignments and free space percentages. .\" ==== namekey ==== .It Ar namekey Ar filename Generate a .Nm HAMMER 64 bit directory hash for the specified file name. The low 32 bits are used as an iterator for hash collisions and will be output as 0. .\" ==== namekey32 ==== .It Ar namekey32 Ar filename Generate the top 32 bits of a .Nm HAMMER 64 bit directory hash for the specified file name. .\" ==== cleanup ==== .It Ar cleanup Op Ar filesystem This is a meta-command which executes snapshot, pruning, and reblocking commands on the specified HAMMER filesystem. If no filesystem is specified this command will scan all HAMMER and nullfs mounts, extract PFS id's, and clean-up each PFS found. .Pp This command will access a .Ar snapshots subdirectory and a .Ar snapshots/config file for each filesystem, creating them if necessary. The default configuration file will create a daily snapshot, do a daily pruning and reblocking run with a fragmentation level of 95%, and a monthly unconditional reblocking run. .Pp All operations are limited to 5 minutes per function by default. Reblocking runs are broken down into three separate functions and take 15 minutes. Also note that this directive will disable snapshots on .Pa /tmp , .Pa /var/tmp , and .Pa /usr/obj by default. .Pp The defaults may be adjusted by modifying the config file. The pruning and reblocking commands automatically maintain a cyclefile for incremental operation. If you ^C the program the cyclefile will be updated, but a sub-command may continue to run in the background for a few seconds until the HAMMER ioctl detects the interrupt. .Pp Work on this command is still in progress. Expected additions: An ability to remove snapshots dynamically as the filesystem becomes full. .\" ==== prune ==== .It Ar prune Ar softlink-dir Prune the file system based on previously created snapshot softlinks. Pruning is the act of deleting file system history. The .Ar prune command will delete file system history such that the file system state is retained for the given snapshots, and all history after the latest snapshot, but all other history is deleted. .Pp The target directory is expected to contain softlinks pointing to snapshots of the file systems you wish to retain. The directory is scanned non-recursively and the mount points and transaction ids stored in the softlinks are extracted and sorted. The file system is then explicitly pruned according to what is found. Cleaning out portions of the file system is as simple as removing a softlink and then running the .Ar prune command. .Pp As a safety measure pruning only occurs if one or more softlinks are found containing the @@ snapshot id extension. Currently the scanned softlink directory must contain softlinks pointing to a single .Nm HAMMER mount. The softlinks may specify absolute or relative paths. Softlinks must use 20-character (@@0x%016llx) transaction ids, as might be returned from .Dq Nm Ar synctid filesystem . .Pp Pruning is a per-PFS operation, so a .Nm HAMMER file system and each PFS in it have to be pruned separately. .Pp Note that pruning a file system may not immediately free-up space, though typically some space will be freed if a large number of records are pruned out. The file system must be reblocked to completely recover all available space. .Pp Example, lets say your snapshot directory contains the following links: .Bd -literal lrwxr-xr-x 1 root wheel 29 May 31 17:57 snap1 -> /usr/obj/@@0x10d2cd05b7270d16 lrwxr-xr-x 1 root wheel 29 May 31 17:58 snap2 -> /usr/obj/@@0x10d2cd13f3fde98f lrwxr-xr-x 1 root wheel 29 May 31 17:59 snap3 -> /usr/obj/@@0x10d2cd222adee364 .Ed .Pp If you were to run the .Ar prune command on this directory, then the .Nm HAMMER .Pa /usr/obj mount will be pruned to retain the above three snapshots. In addition, history for modifications made to the file system older than the oldest snapshot will be destroyed and history for potentially fine-grained modifications made to the file system more recently than the most recent snapshot will be retained. .Pp If you then delete the snap2 softlink and rerun the .Ar prune command, history for modifications pertaining to that snapshot would be destroyed. .\" ==== prune-everything ==== .It Ar prune-everything Ar filesystem This command will remove all historical records from the file system. This directive is not normally used on a production system. .\" ==== snapshot ==== .It Ar snapshot Ar snapshot-dir .It Ar snapshot Ar filesystem snapshot-dir Takes a snapshot of the file system either explicitly given by .Ar filesystem or implicitly derived from the .Ar snapshot-dir argument and creates a symlink in the directory provided by .Ar snapshot-dir pointing to the snapshot. If .Ar snapshot-dir is not a directory, it is assumed to be a format string passed to .Xr strftime 3 with the current time as parameter. If .Ar snapshot-dir refers to an existing directory, a default format string of "snap-%Y%d%m-%H%M" is assumed and used as name for the newly created symlink. .Pp Example, assuming that .Pa /mysnapshots is on file system .Pa / and that .Pa /obj is a file system on its own, the following invocations: .Bd -literal hammer snapshot /mysnapshots hammer snapshot /mysnapshots/%Y-%m-%d hammer snapshot /obj /mysnapshots/obj-%Y-%m-%d .Ed .Pp would create symlinks similar to: .Bd -literal /mysnapshots/snap-20080627-1210 -> /@@0x10d2cd05b7270d16 /mysnapshots/2008-06-27 -> /@@0x10d2cd05b7270d16 /mysnapshots/obj-2008-06-27 -> /obj@@0x10d2cd05b7270d16 .Ed .\" ==== reblock ==== .It Ar reblock Ar filesystem Op Ar fill_percentage .It Ar reblock-btree Ar filesystem Op Ar fill_percentage .It Ar reblock-inodes Ar filesystem Op Ar fill_percentage .It Ar reblock-dirs Ar filesystem Op Ar fill_percentage .It Ar reblock-data Ar filesystem Op Ar fill_percentage Attempt to defragment and free space for reuse by reblocking a live .Nm HAMMER file system. Big blocks cannot be reused by .Nm HAMMER until they are completely free. This command also has the effect of reordering all elements, effectively defragmenting the file system. .Pp The default fill percentage is 100% and will cause the file system to be completely defragmented. All specified element types will be reallocated and rewritten. If you wish to quickly free up space instead try specifying a smaller fill percentage, such as 90% or 80% (the .Sq % suffix is not needed). .Pp Since this command may rewrite the entire contents of the disk it is best to do it incrementally from a .Xr cron 8 job along with the .Fl c Ar cyclefile and .Fl t Ar seconds options to limit the run time. The file system would thus be defragmented over long period of time. .Pp It is recommended that separate invocations be used for each data type. B-tree nodes, inodes, and directories are typically the most important elements needing defragmentation. Data can be defragmented over a longer period of time. .Pp Reblocking is a per-PFS operation, so a .Nm HAMMER file system and each PFS in it have to be reblocked separately. .\" ==== pfs-status ==== .It Ar pfs-status Ar dirpath ... Retrieve the mirroring configuration parameters for the specified .Nm HAMMER file systems or pseudo-filesystems. .\" ==== pfs-master ==== .It Ar pfs-master Ar dirpath Op options Create a pseudo-filesystem (PFS) inside a .Nm HAMMER file system. Up to 65535 such file systems can be created. Each PFS uses an independent inode numbering space making it suitable for use as a replication source or target. .Pp The .Ar pfs-master directive creates a PFS that you can read, write, and use as a mirroring source. .\" ==== pfs-slave ==== .It Ar pfs-slave Ar dirpath Op options Create a pseudo-filesystem (PFS) inside a .Nm HAMMER file system. Up to 65535 such file systems can be created. Each PFS uses an independent inode numbering space making it suitable for use as a replication source or target. .Pp The .Ar pfs-slave directive creates a PFS that you can use as a mirroring target. You will not be able to access a slave PFS until you have completed the first mirroring operation with it as the target (its root directory will not exist until then). .Pp Access to the pfs-slave via the special softlink, as described in the PFS NOTES below, allows .Nm HAMMER to dynamically modify the snapshot transaction id by returning a dynamic result from .Xr readlink 2 calls. .Pp A PFS can only be truly destroyed with the .Ar pfs-destroy directive. Removing the softlink will not destroy the underlying PFS. .\" ==== pfs-update ==== .It Ar pfs-update Ar dirpath Op options Update the configuration parameters for an existing .Nm HAMMER file system or pseudo-filesystem. Options that may be specified: .Bl -tag -width indent .It sync-beg-tid=0x16llx This is the automatic snapshot access starting transaction id for mirroring slaves. This parameter is normally updated automatically by the .Ar mirror-write directive. .Pp It is important to note that accessing a mirroring slave with a transaction id greater than the last fully synchronized transaction id can result in an unreliable snapshot since you will be accessing data that is still undergoing synchronization. .Pp Manually modifying this field is dangerous and can result in a broken mirror. .It sync-end-tid=0x16llx This is the current synchronization point for mirroring slaves. This parameter is normally updated automatically by the .Ar mirror-write directive. .Pp Manually modifying this field is dangerous and can result in a broken mirror. .It shared-uuid= Set the shared UUID for this file system. All mirrors must have the same shared UUID. For safety purposes the .Ar mirror-write directives will refuse to operate on a target with a different shared UUID. .Pp Changing the shared UUID on an existing, non-empty mirroring target, including an empty but not completely pruned target, can lead to corruption of the mirroring target. .It unique-uuid= Set the unique UUID for this file system. This UUID should not be used anywhere else, even on exact copies of the file system. .It label= Set a descriptive label for this file system. .It snapshots= Specify the snapshots directory which 'hammer cleanup' will use to manage this PFS. The snapshots directory does not need to be configured for PFS masters and will default to "/snapshots". .Pp PFS slaves are mirroring slaves so you cannot configure a snapshots directory on the slave itself to be managed by the slave's machine. In fact, the slave will likely have a 'snapshots' sub-directory mirrored from the master, but that directory contains the configuration the master is using for its copy of the filesystem, not the configuration that we want to use for our slave. .Pp It is recommended that "/var/slaves/" be configured for a PFS slave, where is an appropriate label. You can control snapshot retention on your slave independant of the master. .It snapshots-clear Zero out the snapshots directory path for this PFS. .El .\" ==== pfs-upgrade ==== .It Ar pfs-upgrade Ar dirpath Upgrade a PFS from slave to master operation. The PFS will be rolled back to the current end synchronization tid (removing any partial synchronizations), and will then becomes writable. .Pp .Em WARNING! .Nm HAMMER currently supports only single masters and using this command can easily result in file system corruption if you don't know what you are doing. .Pp This directive will refuse to run if any programs have open descriptors in the PFS, including programs chdir'd into the PFS. .\" ==== pfs-downgrade ==== .It Ar pfs-downgrade Ar dirpath Downgrade a master PFS from master to slave operation. The PFS becomes read-only and access will be locked to its .Ar sync-end-tid . .Pp This directive will refuse to run if any programs have open descriptors in the PFS, including programs chdir'd into the PFS. .\" ==== pfs-destroy ==== .It Ar pfs-destroy Ar dirpath This permanently destroys a PFS. .Pp This directive will refuse to run if any programs have open descriptors in the PFS, including programs chdir'd into the PFS. .\" ==== mirror-read ==== .It Ar mirror-read Ar filesystem Op Ar Generate a mirroring stream to stdout. The stream ends when the transaction id space has been exhausted. .\" ==== mirror-read-stream ==== .It Ar mirror-read-stream Ar filesystem Op Ar Generate a mirroring stream to stdout. Upon completion the stream is paused until new data is synced to the master, then resumed. Operation continues until the pipe is broken. .\" ==== mirror-write ==== .It Ar mirror-write Ar filesystem Take a mirroring stream on stdin. .Pp This command will fail if the .Ar shared-uuid configuration field for the two file systems do not match. .\" ==== mirror-dump ==== .It Ar mirror-dump A .Ar mirror-read can be piped into a .Ar mirror-dump to dump an ascii representation of the mirroring stream. .\" ==== mirror-copy ==== .It Ar mirror-copy Ar [[user@]host:]filesystem Ar [[user@]host:]filesystem This is a shortcut which pipes a .Ar mirror-read command to a .Ar mirror-write command. If a remote host specification is made the program forks a .Xr ssh 1 and execs the .Ar mirror-read and/or .Ar mirror-write on the appropriate host. The source may be a master or slave PFS, and the target must be a slave PFS. .Pp This command also established full duplex communication and turns on the two-way protocol feature which automatically negotiates transaction id ranges without having to use a cycle file. If the operation completes successfully the target PFS's .Ar sync-end-tid will be updated. Note that you must re-chdir into the target PFS to see the updated information. If you do not you will still be in the previous snapshot. .\" ==== mirror-stream ==== .It Ar mirror-stream Ar [[user@]host:]filesystem Ar [[user@]host:]filesystem This command works similarly to .Ar mirror-copy but does not exit unless the pipe is broken. This command will resume the mirroring operation whenever the master is synced. The command is commonly used with .Fl i Ar delay and .Fl b Ar bandwidth options to keep the mirroring target in sync with the source on a continuing basis. .El .\".Sh EXAMPLES .Sh PSEUDO FILESYSTEM (PFS) NOTES The root of a PFS is not hooked into the primary .Nm HAMMER file system as a directory. Instead, .Nm HAMMER creates a special softlink called "@@PFS%05d" (exactly 10 characters long) in the primary .Nm HAMMER file system. .Nm HAMMER then modifies the contents of the softlink as read by .Xr readlink 2 , and thus what you see with an .Xr ls 1 command or if you were to .Xr cd 1 into the link. If the PFS is a master the link reflects the current state of the PFS. If the PFS is a slave the link reflects the last completed snapshot, and the contents of the link will change when the next snapshot is completed, and so forth. .Pp PFS support is currently very new and experimental. The .Nm utility employs numerous safeties to reduce user foot-shooting. The .Ar mirror-copy directive requires that the target be configured as a slave and that the .Ar shared-uuid field of the mirroring source and target match. .Sh DIAGNOSTICS .Ex -std .Sh SEE ALSO .Xr undo 1 , .Xr HAMMER 5 , .Xr mount_hammer 8 , .Xr newfs_hammer 8 .Sh HISTORY The .Nm utility first appeared in .Dx 1.11 . .Sh AUTHORS .An Matthew Dillon Aq dillon@backplane.com