usr.sbin/makefs: Add HAMMER2 support
authorTomohiro Kusumi <tkusumi@netbsd.org>
Sat, 4 Jun 2022 11:54:35 +0000 (20:54 +0900)
committerTomohiro Kusumi <tkusumi@netbsd.org>
Sat, 4 Jun 2022 13:03:41 +0000 (22:03 +0900)
commit2d60b848f2503f28d840ceae174d07eb149ccce9
tree1f743478a1f3b3169b3fb5409a4c255f531dcc7c
parent4efe7d95b2bef41b8d3a6d1b9a134a88db637ebb
usr.sbin/makefs: Add HAMMER2 support

This commit adds HAMMER2 image creation support for makefs(8).
It runs newfs_hammer2(8) and then sys/vfs/hammer2 logic in userspace
to create HAMMER2 image from a given directory.

This commit splits newfs_hammer2(8) into newfs and mkfs part simlarly
to newfs_msdos(8), so that makefs(8) can use newfs functionality.
The entire sys/vfs/hammer2 (with exception of unneeded
hammer2_{bulkfree,ccms,iocom,ioctl,msgops,synchro}.[hc] and reusable
hammer2_disk.h) is copied to usr.sbin/makefs with below modification.
It intends to have minimum amount of diff against sys/vfs/hammer2.

* Header includes are modified so that it compiles in userspace.
* VFS and other kernel functions are usually implemented as simple
 stub functions in hammer2_compat.h and hammer2_buf.c, but some are
 commented out.
* Kernel functions such as kprintf, kmalloc, kprintf, kstrdup, etc
 are implemented using corresponding libc functions.
* Lock primitives are basically NOP, and they (should) never block
 as makefs(8) is a single thread program.
* struct vnode and struct buf (the ones defined locally in makefs(8),
 not sys/sys/*) have new struct members only used by HAMMER2 to
 emulate VFS behavior required by HAMMER2.
* Since makefs(8) is write-only, VOP_{NRESOLVE,NCREATE,NMKDIR,NLINK,
 NSYMLINK,WRITE,STRATEGY} are implemented, but other VOPs just
 return EOPNOTSUPP.
* VOP_{INACTIVE,RECLAIM} may be implemented and used in future to
 better emulate VFS behavior to address current limitation.
* VOP_WRITE is modified to directly call VOP_STRATEGY function.
* The XOP kernel thread is modified to act as a regular function
 called from VOPs, along with simplified admin code.

It currently has following limitations.

* multi-volumes is unsupported, simply due to makefs(8) only taking 1
 image file path.
* Not necessarily a limitation, but it only supports populating 1 PFS,
 which is "DATA" by default. Other PFSes if any won't have anything
 under the root PFS inode.
* makefs(8) process gets killed by OOM for a directory with *extremely*
 large number of files, depending on available memory. This is due to
 the way it currently tries to flush all chains in a single VFS_SYNC.
 Supporting multiple VFS_SYNC calls by checking available memory along
 the way gives chance to free unused vnodes/inodes and chains. This
 may be implemented in future. This limitation is specific to HAMMER2,
 as all other makefs(8) filesystems are not CoW, meaning they allow
 in-place write based objects creation from a top directory to bottom
 whereas HAMMER2 flushes chains in bottom-up direction.
48 files changed:
sbin/newfs_hammer2/Makefile
sbin/newfs_hammer2/mkfs_hammer2.c [copied from sbin/newfs_hammer2/newfs_hammer2.c with 80% similarity]
sbin/newfs_hammer2/mkfs_hammer2.h [new file with mode: 0644]
sbin/newfs_hammer2/newfs_hammer2.c
usr.sbin/makefs/Makefile
usr.sbin/makefs/ffs/buf.c
usr.sbin/makefs/ffs/buf.h
usr.sbin/makefs/hammer2.c [new file with mode: 0644]
usr.sbin/makefs/hammer2.h [new file with mode: 0644]
usr.sbin/makefs/hammer2/Makefile.inc [new file with mode: 0644]
usr.sbin/makefs/hammer2/hammer2.h [new file with mode: 0644]
usr.sbin/makefs/hammer2/hammer2_admin.c [new file with mode: 0644]
usr.sbin/makefs/hammer2/hammer2_buf.c [new file with mode: 0644]
usr.sbin/makefs/hammer2/hammer2_chain.c [new file with mode: 0644]
usr.sbin/makefs/hammer2/hammer2_cluster.c [new file with mode: 0644]
usr.sbin/makefs/hammer2/hammer2_compat.h [new file with mode: 0644]
usr.sbin/makefs/hammer2/hammer2_flush.c [new file with mode: 0644]
usr.sbin/makefs/hammer2/hammer2_freemap.c [new file with mode: 0644]
usr.sbin/makefs/hammer2/hammer2_inode.c [new file with mode: 0644]
usr.sbin/makefs/hammer2/hammer2_io.c [new file with mode: 0644]
usr.sbin/makefs/hammer2/hammer2_lz4.c [new file with mode: 0644]
usr.sbin/makefs/hammer2/hammer2_lz4.h [new file with mode: 0644]
usr.sbin/makefs/hammer2/hammer2_lz4_encoder.h [new file with mode: 0644]
usr.sbin/makefs/hammer2/hammer2_ondisk.c [new file with mode: 0644]
usr.sbin/makefs/hammer2/hammer2_strategy.c [new file with mode: 0644]
usr.sbin/makefs/hammer2/hammer2_subr.c [new file with mode: 0644]
usr.sbin/makefs/hammer2/hammer2_vfsops.c [new file with mode: 0644]
usr.sbin/makefs/hammer2/hammer2_vnops.c [new file with mode: 0644]
usr.sbin/makefs/hammer2/hammer2_xops.c [new file with mode: 0644]
usr.sbin/makefs/hammer2/zlib/hammer2_zlib.h [new file with mode: 0644]
usr.sbin/makefs/hammer2/zlib/hammer2_zlib_adler32.c [new file with mode: 0644]
usr.sbin/makefs/hammer2/zlib/hammer2_zlib_deflate.c [new file with mode: 0644]
usr.sbin/makefs/hammer2/zlib/hammer2_zlib_deflate.h [new file with mode: 0644]
usr.sbin/makefs/hammer2/zlib/hammer2_zlib_inffast.c [new file with mode: 0644]
usr.sbin/makefs/hammer2/zlib/hammer2_zlib_inffast.h [new file with mode: 0644]
usr.sbin/makefs/hammer2/zlib/hammer2_zlib_inffixed.h [new file with mode: 0644]
usr.sbin/makefs/hammer2/zlib/hammer2_zlib_inflate.c [new file with mode: 0644]
usr.sbin/makefs/hammer2/zlib/hammer2_zlib_inflate.h [new file with mode: 0644]
usr.sbin/makefs/hammer2/zlib/hammer2_zlib_inftrees.c [new file with mode: 0644]
usr.sbin/makefs/hammer2/zlib/hammer2_zlib_inftrees.h [new file with mode: 0644]
usr.sbin/makefs/hammer2/zlib/hammer2_zlib_trees.c [new file with mode: 0644]
usr.sbin/makefs/hammer2/zlib/hammer2_zlib_trees.h [new file with mode: 0644]
usr.sbin/makefs/hammer2/zlib/hammer2_zlib_zconf.h [new file with mode: 0644]
usr.sbin/makefs/hammer2/zlib/hammer2_zlib_zutil.c [new file with mode: 0644]
usr.sbin/makefs/hammer2/zlib/hammer2_zlib_zutil.h [new file with mode: 0644]
usr.sbin/makefs/makefs.8
usr.sbin/makefs/makefs.c
usr.sbin/makefs/makefs.h