Aaron LI [Wed, 14 May 2025 05:08:22 +0000 (13:08 +0800)]
kernel: Clean up DOSPTYP_* defines in subr_diskmbr.c
* Move DOSPTYP_ONTRACK to <sys/diskmbr.h> where other DOSPTYP_* defines
are located.
* Remove DOSPTYP_{EXTENDED,EXTENDEDX} and use DOSPTYP_{EXT,EXTLBA} from
<sys/diskmbr.h> instead.
Aaron LI [Tue, 13 May 2025 05:07:59 +0000 (13:07 +0800)]
kernel: Remove obsolete DSO_COMPATMBR from <sys/disk.h>
The last use of this flag was removed in commit
1c3c151b5fca37695bf972f959efb7dda16cfbe6, so this flag has become
effectively obsolete since then. Now just remove it to reduce any
confusion.
ok by dillon.
Aaron LI [Mon, 12 May 2025 12:40:37 +0000 (20:40 +0800)]
gpt(8): Remove now obsolete "#ifndef _LIBEFIVAR"
Aaron LI [Mon, 12 May 2025 12:35:51 +0000 (20:35 +0800)]
libefivar(3): Fix build by using uuid_dec_le() from libc
It was using le_uuid_dec() from source of gpt(8). As a result, this
library no longer depends on the 'gpt.c' file from gpt(8).
While there, fix a typo in comment (taken from FreeBSD).
Aaron LI [Mon, 12 May 2025 11:56:49 +0000 (19:56 +0800)]
gpt(8): Use uuid_{enc,dec}_le() from libc
See also uuid(3) man page.
Aaron LI [Mon, 12 May 2025 03:23:43 +0000 (11:23 +0800)]
gpt(8): Refactor parse_uuid() and support more GPT type aliases
Refactor parse_uuid() to parse GPT type aliases by using an array.
Meanwhile, extend it to support more GPT type aliases, which are defined
in <sys/gpt.h> as well as "/etc/defaults/uuids".
Aaron LI [Fri, 16 May 2025 03:28:08 +0000 (11:28 +0800)]
gpt(8): Improve "show" command to print humanized size of partitions
Before:
Disk da0: 50.0GB (
104857600 512-byte sectors)
start size index contents
0 1 - PMBR
1 1 - Pri GPT header
2 32 - Pri GPT table
34 2014 - Unused
2048 262144 0 GPT part - EFI System
264192
104591360 1 GPT part - DragonFly Label64
104855552 2015 - Unused
104857567 32 - Sec GPT table
104857599 1 - Sec GPT header
After:
Disk da0: 50.0GB (
104857600 512-byte sectors)
Start Sectors Size Index Contents
0 1 512B - PMBR
1 1 512B - Pri GPT header
2 32 16.0KB - Pri GPT table
34 2014 1007KB - Unused
2048 262144 128MB 0 GPT part - EFI System
264192
104591360 49.9GB 1 GPT part - DragonFly Label64
104855552 2015 1008KB - Unused
104857567 32 16.0KB - Sec GPT table
104857599 1 512B - Sec GPT header
Aaron LI [Sat, 10 May 2025 23:51:06 +0000 (07:51 +0800)]
gpt(8): Improve "show" command to print a disk summary line
Before print the partition entries, print a disk summary line like:
"Disk /dev/da0: 50.0GB (
104857600 512-byte sectors)".
In addition, add a blank line between disks.
Example new output:
```
$ sudo gpt show /dev/da0 ad0
Disk /dev/da0: 50.0GB (
104857600 512-byte sectors)
start size index contents
0 1 - PMBR
1 1 - Pri GPT header
2 32 - Pri GPT table
34 2014 - Unused
2048 262144 0 GPT part - EFI System
264192
104591360 1 GPT part - DragonFly Label64
104855552 2015 - Unused
104857567 32 - Sec GPT table
104857599 1 - Sec GPT header
Disk ad0: 40.0GB (
83886080 512-byte sectors)
start size index contents
0 1 - PMBR
1 1 - Pri GPT header
2 32 - Pri GPT table
34 2014 - Unused
2048
83881984 0 GPT part - FreeBSD UFS/UFS2
83884032 2015 - Unused
83886047 32 - Sec GPT table
83886079 1 - Sec GPT header
```
Aaron LI [Mon, 12 May 2025 11:20:14 +0000 (19:20 +0800)]
uuid(3): Improve the description for uuid_to_string()
Document that uuid_to_string() will allocate a buffer to hold the
converted string and free(3) should be called to release the memory.
Several more minor tweaks.
Derived from FreeBSD.
Aaron LI [Sun, 11 May 2025 10:03:53 +0000 (18:03 +0800)]
etc: Sync entries from <sys/gpt.h>
Some of the GPT type names are derived from FreeBSD's gpart(8) man page.
Aaron LI [Sun, 11 May 2025 09:55:52 +0000 (17:55 +0800)]
<sys/gpt.h>: Small tweaks
* Reorder the DragonFly entries.
* Move the GPT_ENT_TYPE_PREP_BOOT out of the FreeBSD group; it's used by
the boot partition on some PowerPC systems.
* Remove obsolete description on Linux reusing the
GPT_ENT_TYPE_MS_BASIC_DATA entry.
Aaron LI [Sat, 10 May 2025 07:47:53 +0000 (15:47 +0800)]
fdisk(8): Support 4096 sector size and recognize pMBR of 4Kn disks
Tweak the fdisk(8) utility to support 4096-byte sector size, so it now
can read the pMBR on GPT-formatted 4Kn (aka 4K native) disks, e.g.,
# fdisk -s vbd0
/dev/vbd0: 14628 cyl 16 hd 56 sec
Part Start Size Type Flags
1: 1
13107199 0xee 0x00
In addition, tweak read_disk() to report the read error message.
Aaron LI [Sat, 10 May 2025 07:38:50 +0000 (15:38 +0800)]
fdisk(8): Styles and cleanups (no functional change)
Aaron LI [Fri, 9 May 2025 03:13:23 +0000 (11:13 +0800)]
nrelease: Don't disable syslogd in live CD/USB
We're mounting a tmpfs at /var, so it's okay to write under /var/log.
Don't disable syslogd so that it could write the log files that might
be helpful.
Aaron LI [Fri, 9 May 2025 04:26:10 +0000 (12:26 +0800)]
fdisk(8): Clear errno in read_disk() to avoid confusing others
For example, even read_disk() succeeded in detecting the sector size and
reading, the errno would remain EIO, which could confuse the later code
that calls err() or others.
Aaron LI [Thu, 8 May 2025 09:54:18 +0000 (17:54 +0800)]
fdisk(8): Add "vbd0" to search list and order "da0" before "ad0"
The virtio_blk(4) device is now very common, so worth adding it to the
search list.
In addition, order "da0" before "ad0" as SATA/SCSI is much more common
than ATA/IDE.
Aaron LI [Thu, 8 May 2025 09:49:48 +0000 (17:49 +0800)]
fdisk(8): Sync TRIM behavior with gpt(8) and clean up code
The TRIM feature is enabled by default in kernel now, so let fdisk(8)
continue the operation if TRIM if the device doesn't support it, similar
to gpt(8).
Update the man page accordingly.
Aaron LI [Thu, 8 May 2025 09:10:03 +0000 (17:10 +0800)]
fdisk(8): Calculate ncylinders and warn about mismatch
The virtio_blk(4) disk driver was directly reporting the ncylinders
parameter specified by the QEMU emulator, which however is capped at
16383 (the legacy MBR maximum). That confused fdisk(8) to recognize the
disk to have only ~8GB size instead of the full capacity.
The issue virtio_blk(4) has been fixed in a former commit. This commit
improve fdisk(8) by calculating the ncylinders on its own and comparing
it with the one provided by the disk. If there is a mismatch, emit a
warning message to remind a fix to the disk driver.
Derived from the patch submitted by mikdusan in bug #3369.
Bug: https://bugs.dragonflybsd.org/issues/3369
Aaron LI [Thu, 8 May 2025 08:26:24 +0000 (16:26 +0800)]
fdisk(8): Improve summary show of unintialized disk
For an unintialized disk, "fdisk -s" would simply exit an error without
printing any useful information, e.g.,
# fdisk -s vbd0
fdisk: invalid fdisk partition table found
fdisk: read_s0: Undefined error: 0
This change makes "fdisk -s" print the geometry of the disk before
exiting. In addition, check the "errno" value to avoid the confusing
"read_s0: Undefined error: 0" message.
Derived from the patch submitted by mikdusan in bug #3369.
Bug: https://bugs.dragonflybsd.org/issues/3369
Aaron LI [Fri, 9 May 2025 05:05:07 +0000 (13:05 +0800)]
virtio/block: Improve the log to also print sector size
Aaron LI [Fri, 9 May 2025 03:10:18 +0000 (11:10 +0800)]
virtio/block: Fix media_blksize and media_blocks calculation
* Correctly report the media block size instead of hardcoded 512.
* Fix the bug in calculating the media blocks.
Matthew Dillon [Thu, 8 May 2025 05:46:19 +0000 (22:46 -0700)]
virtio - Fix virtio block device cylinder count reporting
* qemu's virtio caps its reported cylinder count to 16383, which
confuses fdisk.
* Check for the legacy cap and recalculate the reported cylinders
based on the other parameters, and report this on the console.
* We decided not to modify fdisk for now. Instead we "fixed" virtio.
Reported-by: mikdusan
DragonFly-PR: 3369
Matthew Dillon [Wed, 7 May 2025 17:13:10 +0000 (10:13 -0700)]
kernel - Fix kernel panic and user process corruption bug
* In certain limited circumstances related to user processes which
fork() (but do not exec()), a test in the vm_map simplification
code can crash the kernel or improperly coalesce two map entries
that should not be, resulting in user process corruption.
* Observed crashes from use of neovim in an xfce terminal.
* Possibly also effected corruption / issues with chromium tab crashes.
Reported-by: marto, arcade
DragonFly-PR: 3381
Aaron LI [Wed, 7 May 2025 02:17:07 +0000 (10:17 +0800)]
diff(1): Update header time format to be compatible with GNU diff
The old code was calling ctime() to format the time, which had a format
like "Tue May 6 19:00:26 2025" that was different from the GNU diff.
Fix the code to format the time in the same format as GNU diff, e.g.,
"2025-05-06 19:00:26.
113084000 +0800".
FreeBSD also made such a change to its diff. However, we don't consider
the '-c' option here, because I checked with GNU diff and it didn't
change the time format with the '-c' option.
Referred-to: FreeBSD
Aaron LI [Wed, 7 May 2025 02:16:19 +0000 (10:16 +0800)]
diff3(1): Add two blank lines for readability
Aaron LI [Wed, 7 May 2025 02:15:09 +0000 (10:15 +0800)]
sdiff(1): Tweak Makefile to use 'MAN' instead of 'MAN1'
For consistency with other makefiles.
Matthew Dillon [Tue, 6 May 2025 22:22:18 +0000 (15:22 -0700)]
kernel - Fix another netinet6 panic, change how ipv6 proxying works (2)
* Also have to set the flag for lo interfaces because host IPs in the
routing table are bound to lo0.
Matthew Dillon [Fri, 2 May 2025 05:20:54 +0000 (22:20 -0700)]
kernel - Fix another netinet6 panic, change how ipv6 proxying works
ipv6 isn't naturally proxying subnets on other interface (not responding
to neighbor solicitations) when forwarding is turned on.
However, there are numerous other issues involved here so for now I
am pushing a hack so at least we are backwards-compatible.
* Add a new interface flag, IFF_ANNOUNCE, named "proxy" in ifconfig.
Add "proxy" and "-proxy" options to ifconfig.
* Most non-localhost interfaces will set this bit by default. The bit
can be turned off in ifconfig using -proxy when configuring the interface.
* Adjust nd6_nbr.c (neighbor solicitation code) to respond to
solicitations if (A) the route has RTF_ANNOUNCE set, or (B)
the interface the route is associated with (note: its parent bridge
if the interface is on a bridge) has the IFF_ANNOUNCE bit set.
This makes ipv6 forwarding work naturally again.
* Also fix a bug where an ipv6 state assertion sometimes triggers.
Do not assert, and instead report the problem. This seems to occur
because the state machine sometimes does not initialize the llinfo
(the MAC) in the cache entry but still sets the state to a value
beyond ND6_LLINFO_INCOMPLETE.
This fix is temporary until we can figure out a better way to deal with
DAD issues.
Roy Marples [Thu, 1 May 2025 22:05:52 +0000 (22:05 +0000)]
rtsock: Apply flags on RTM_CHANGE
Matthew Dillon [Wed, 30 Apr 2025 21:49:59 +0000 (14:49 -0700)]
kernel - Fix ip6_forward_rt global and related panic
* Make ip6_forward_rt per-cpu, which should fix a route caching panic
that can occur sometimes due to code assuming single-threaded
operation.
Aaron LI [Mon, 28 Apr 2025 09:48:14 +0000 (17:48 +0800)]
nrelease: Check GITREV and just fail if empty
The GITREV variable is determined by the tools/gitrev.sh script, which
might fail if the current user has insufficient permission to access the
source directory. That would cause the `uname` to output something
strange that could confuse people. So I think it's better to just fail
the build and require a fix to the build environment.
Reported-by: JustinS
Tomohiro Kusumi [Sun, 27 Apr 2025 06:05:06 +0000 (23:05 -0700)]
usr.sbin/makefs/hammer2: Non functional cleanup
Tomohiro Kusumi [Wed, 23 Apr 2025 07:25:26 +0000 (00:25 -0700)]
usr.sbin/makefs: Sync with sys/vfs/hammer2
Sync with following sys/vfs/hammer2 commits.
$ git log --pretty="%h %s"
b4ddbe789819885eb6f829ae1760b9844c29eb07.. -- sys/vfs/hammer2/
b2cc6686e1 sys/vfs/hammer2: Zero clear z_stream
3bfb85bd96 sys/vfs/hammer2: Mark hammer2_base_find's key_end argument as unused
dff4f0d54e sys/vfs/hammer2: Fix potential NULL dereference
641f0e615d sys/vfs/hammer2: Remove duplicate [vf]chain.bref.mirror_tid initialization
9443de1e69 sys/vfs/hammer2: Remove obsolete comments for unused/reserved ondisk fields
2c8c691b4a sys/vfs/hammer2: Change hammer2_inode_meta::target_type to unused84
Tomohiro Kusumi [Tue, 22 Apr 2025 06:05:39 +0000 (23:05 -0700)]
sys/vfs/hammer2: Zero clear z_stream
Zero clear z_stream local variables as seen in other subsystems
that use zlib. sys/vfs/hammer2/zlib doesn't seem to differ either.
Having non-zero garbage in unused pointer members causes panic in
deflateEnd() / inflateEnd(). These functions actually panic in
FreeBSD when set to zlib or makefs'd with zlib.
Michael Neumann [Sat, 19 Apr 2025 20:04:14 +0000 (22:04 +0200)]
tcplay/libtcplay - Reenable use of cryptoapi
Successfully tested with the following ciphers / chained cipher
combinations (created using Veracrypt on Windows):
* AES-256-XTS
* SERPENT-256-XTS
* TWOFISH-256-XTS
* TWOFISH-256-XTS,AES-256-XTS
* SERPENT-256-XTS,TWOFISH-256-XTS,AES-256-XTS
Also tested with cryptdisks(8), which uses libtcplay.
Michael Neumann [Sat, 19 Apr 2025 09:50:03 +0000 (11:50 +0200)]
tcplay - add --batch-mode or -q option
This allows automated testing to be truely automated.
Use the same option naming as in cryptsetup(8).
Michael Neumann [Sat, 19 Apr 2025 13:23:55 +0000 (15:23 +0200)]
cryptoapi - Add test case
Test that cryptoapi and cryptodev have the same outcome
for encryption and decryption.
Michael Neumann [Sat, 19 Apr 2025 13:01:09 +0000 (15:01 +0200)]
dm_target_crypt_ng - adapt to recent crytpoapi changes
Michael Neumann [Sat, 19 Apr 2025 16:43:45 +0000 (18:43 +0200)]
cryptoapi - constify IV and pass-in IV length as well
* This only affects XTS mode as CBC never mutated the IV.
* Never pass-in data of unknown length.
* In case of XTS, add a tweak to the contex, similar to what
FreeBSD does.
* Get rid of crypto_cipher_iv union type. simplify.
* Also ensure that, in case we pass in a "too short" IV,
we never read beyond the end of the passed-in IV. Zero-pad
it in that case.
Michael Neumann [Sat, 19 Apr 2025 12:18:21 +0000 (14:18 +0200)]
cryptoapi - simplify session handling
* Make cryptoapi_cipher_session an opaque type.
* Rename cryptoapi_cipher_initsession() to
cryptoapi_cipher_newsession() and return a pointer to the allocated
session, or NULL in case kmalloc() fails.
* Allocate enough memory in order to have space for the cipher context
plus padding for proper alignment.
* No longer return an error from cryptoapi_cipher_freesession().
Silently do the job.
Tomohiro Kusumi [Mon, 21 Apr 2025 08:25:09 +0000 (01:25 -0700)]
sbin/hammer2/cmd_volume.c: Print \0 terminated PFS name
Max strlen allowed is HAMMER2_INODE_MAXNAME (not HAMMER2_INODE_MAXNAME-1).
Aaron LI [Fri, 18 Apr 2025 06:16:36 +0000 (14:16 +0800)]
wg: Improve wg_peer_create() to propagate errors
Add the 'int *errp' parameter to wg_peer_create() to propagate the error
from noise_remote_enable() to wg_ioctl_set().
Originally, wg_ioctl_set() returned ENOMEM if wg_peer_create() failed,
but it should actually be ENOSPC because it could only fail because of
noise_remote_enable() that only returns ENOSPC.
See also: https://reviews.freebsd.org/D49796
Aaron LI [Thu, 17 Apr 2025 01:03:25 +0000 (09:03 +0800)]
timeout(1): Sync test cases from FreeBSD (although unused)
Obtained-from: FreeBSD
Aaron LI [Thu, 17 Apr 2025 01:02:20 +0000 (09:02 +0800)]
timeout.1: Fix the SPDX license identifier
There is no FreeBSD version of the 2-Clause BSD License.
Obtained-from: FreeBSD
Aaron LI [Thu, 17 Apr 2025 01:01:45 +0000 (09:01 +0800)]
timeout(1): Add verbose log to kill_self() and tweak another one
Aaron LI [Thu, 17 Apr 2025 00:59:52 +0000 (08:59 +0800)]
timeout(1): Catch SIGVTALRM and SIGPROF
I missed these two signals in commit
13eb7f38ede3f73ecb16d7f0fe4a9d3679e860c5.
Michael Neumann [Tue, 15 Apr 2025 19:35:27 +0000 (21:35 +0200)]
libtcplay - reenable the cryptodev code for now
The new cryptoapi code needs more thorough testing!
I've run into some issues detecting AES-256-XTS tcplay devices. While I
have working code now that detects AES-256-XTS devices, I do not fully
understand the exact reason - compiler optimizations?
Aaron LI [Tue, 15 Apr 2025 06:11:29 +0000 (14:11 +0800)]
libtcplay: Include rijndael-alg-fst.c to fix the static library
The static library (libtcplay.a) is used by sbin/cryptdisks, which is
built as a static binary. The missing source of 'rijndael-alg-fst.c' caused
sbin/cryptdisks to fail the linking with the following error:
/usr/obj/srv/jenkins/workspace/DragonFlyBSD/ctools_x86_64_x86_64/usr/libexec/binutils234/elf/ld.bfd: /usr/obj/srv/jenkins/workspace/DragonFlyBSD/world_x86_64/usr/lib/libtcplay.a(rijndael-api.o): in function `rijndael_set_key':
rijndael-api.c:(.text+0x12): undefined reference to `rijndaelKeySetupEnc'
/usr/obj/srv/jenkins/workspace/DragonFlyBSD/ctools_x86_64_x86_64/usr/libexec/binutils234/elf/ld.bfd: rijndael-api.c:(.text+0x27): undefined reference to `rijndaelKeySetupDec'
/usr/obj/srv/jenkins/workspace/DragonFlyBSD/ctools_x86_64_x86_64/usr/libexec/binutils234/elf/ld.bfd: /usr/obj/srv/jenkins/workspace/DragonFlyBSD/world_x86_64/usr/lib/libtcplay.a(rijndael-api.o): in function `rijndael_decrypt':
rijndael-api.c:(.text+0x48): undefined reference to `rijndaelDecrypt'
/usr/obj/srv/jenkins/workspace/DragonFlyBSD/ctools_x86_64_x86_64/usr/libexec/binutils234/elf/ld.bfd: /usr/obj/srv/jenkins/workspace/DragonFlyBSD/world_x86_64/usr/lib/libtcplay.a(rijndael-api.o): in function `rijndael_encrypt':
rijndael-api.c:(.text+0x66): undefined reference to `rijndaelEncrypt'
collect2: error: ld returned 1 exit status
Add 'rijndael-alg-fst.c' to SRCS to fix it. Actually, this source is
also listed in sys/crypto/Makefile for rijndael.
Aaron LI [Tue, 15 Apr 2025 05:30:38 +0000 (13:30 +0800)]
libtcplay: Fix building within buildworld
Michael Neumann [Mon, 14 Apr 2025 14:39:27 +0000 (16:39 +0200)]
tcplay - Add "return 0" in the successful case
Michael Neumann [Sun, 13 Apr 2025 11:34:28 +0000 (13:34 +0200)]
tcplay - Use cryptoapi instead of /dev/crypto
tcplay(8) is a BSD-licensed Truecrypt/Veracrypt compatible
implementation by Alex Hornung.
libtcplay and tcplay no longer depend on the existence of /dev/crypto.
OpenBSD, for one, removed /dev/crypto. Also, there are two bug
reports #3311 and #3370 related to tcplay and/or cryptodev. I could
not reproduce them anymore before this commit. So they might have
already been fixed by other commits or they might be heisebugs.
Why can't we use the algorithms from LibreSSL?
Well, while we could use AES from LibreSSL, LibreSSL does not support
Twofish and Serpent, two non-standardized algorithms. So we'd either
have to drop Twofish and Serpent, or use cryptoapi for just those.
Note that when using tcplay, all file encryption is performed by
"dm_target_crypt" under the hood.
Tested by creating a Veracrypt-encrypted drive using a chain of AES,
Twofish and Serpent on Windows and then trying to open and use it on
DragonFly. Works flawlessly.
Michael Neumann [Sun, 13 Apr 2025 17:18:50 +0000 (19:18 +0200)]
cryptoapi - Cleanup / style
Suggested-by: Aaron Li
Michael Neumann [Sun, 13 Apr 2025 10:43:23 +0000 (12:43 +0200)]
crypto - Bring in cryptoapi from dm_target_crypt_ng
cryptoapi is a facade for (some) symmetrical block ciphers that
we have in the kernel. In contrast to opencrypto, it provides a
synchronous API and just does one thing.
It can also be used outside the kernel (e.g. for libtcplay).
Also reorder some files in conf/files.
Michael Neumann [Sun, 13 Apr 2025 10:00:57 +0000 (12:00 +0200)]
dm_target_crypt_ng - Make cryptoapi usable outside the kernel
The intention here is to finally use it in libtcplay, mainly
for Serpent and Twofish, as they are not available in libressl.
Michael Neumann [Sun, 13 Apr 2025 09:28:26 +0000 (11:28 +0200)]
crypto - add FALLTHROUGH comment in Twofish
This allows to compile the Twofish algorithm outside the kernel.
Michael Neumann [Sun, 13 Apr 2025 09:19:42 +0000 (11:19 +0200)]
crypto - fix wrong const pointer in aesni_decrypt_cbc
The function decrypts the buffer in-place, so "const" is wrong.
Michael Neumann [Sat, 12 Apr 2025 18:57:01 +0000 (20:57 +0200)]
dm_target_crypt_ng - test compatibility of old and new module
This script tests compatibility between dm_target_crypt and
dm_target_crypt_ng.
It creates an encrypted disk with one implementation, then copies
files to it. Then it mounts it using the other implementation and
verifies that the copied files are still the same (using mtree).
It does this procedure in both directions and various configurations
(e.g. with or w/o AESNI enabled).
Michael Neumann [Sat, 12 Apr 2025 18:44:01 +0000 (20:44 +0200)]
dm_target_crypt_ng - free the crypto session
Michael Neumann [Sat, 12 Apr 2025 16:51:42 +0000 (18:51 +0200)]
dm_target_crypt_ng - further simplify XTS ciphers
Michael Neumann [Sat, 12 Apr 2025 16:38:30 +0000 (18:38 +0200)]
dm_target_crypt_ng - unify encrypt/decrypt functions
Michael Neumann [Sat, 12 Apr 2025 16:23:11 +0000 (18:23 +0200)]
dm_target_crypt_ng - deduplicate XTS cipher implementations
Michael Neumann [Sat, 12 Apr 2025 16:06:19 +0000 (18:06 +0200)]
dm_target_crypt_ng - simplify cipher probing
Michael Neumann [Sat, 12 Apr 2025 13:44:25 +0000 (15:44 +0200)]
dm_target_crypt_ng - Implement Twofish/Serpent CBC/XTS ciphers
This adds quite a lot of duplication concerning the XTS mode which
has to be refactored with a follow up commit.
It also has to be tested properly!
Michael Neumann [Sat, 12 Apr 2025 13:21:14 +0000 (15:21 +0200)]
dm_target_crypt_ng - cryptoapi: rename IV fields
Michael Neumann [Sat, 12 Apr 2025 13:38:57 +0000 (15:38 +0200)]
crypto - use const pointers in Twofish/Serpent
Michael Neumann [Sat, 12 Apr 2025 09:45:41 +0000 (11:45 +0200)]
dm_target_crypt_ng - Rename crypto_cipher to cryptoapi
This should finally end up under sys/crypto/api.
Michael Neumann [Fri, 11 Apr 2025 21:14:47 +0000 (23:14 +0200)]
dm_target_crypt_ng - Improve crypto_cipher API
* Introduce sessions - crypto_cipher_initsession() and
crypto_cipher_freesession().
* This allows us to hide (most) implementation details.
* Expose public functions `crypto_cipher_` instead of calling the
function pointers from crypto_cipher directly. Required to hide
implementation details.
* Move error checking into the API layer. Cipher implementations can
assume that the context is correctly memory aligned and the block
size matches that of the algorithm.
Michael Neumann [Fri, 11 Apr 2025 20:32:45 +0000 (22:32 +0200)]
dm_target_crypt_ng - Use struct initializer in crypto_cipher defs
Michael Neumann [Fri, 11 Apr 2025 19:26:26 +0000 (21:26 +0200)]
dm_target_crypt_ng - Refactor crypto ciphers
* Improve software AES-XTS implementation by passing in a block
function and moving the explicit_bzero to the outer function
* Replace use of macros with inline functions
* Use xor_block and xor_block3 functions
Aaron LI [Sat, 12 Apr 2025 08:59:57 +0000 (16:59 +0800)]
wg: Fix wg_peer_create() to free 'peer->p_remote' on failure
This bug was introduced by me in commit
902964ab24ba9d2c978017d369c0faa8d2fe0f9e.
See also: https://reviews.freebsd.org/D49796
Aaron LI [Fri, 11 Apr 2025 08:40:31 +0000 (16:40 +0800)]
contrib: Remove empty diffutils
contrib/diffutils was replaced with a BSD version in commit
c9733229451fac5faa53b1a016b01866eae75a1c.
That commit removed the contents of these files but missed to remove the
files themselves.
Michael Neumann [Wed, 9 Apr 2025 09:25:42 +0000 (11:25 +0200)]
dm_target_crypt_ng - Remove worker pool
No longer use a worker pool to distribute read/write BIOs to worker
threads. Go back to the callback approach of the original
dm_target_crypt implementation, except that encryption and decryption
are now done synchronously (or in the mpipe callback).
This commit makes the code simpler and also slightly improves
performance over the worker pool implementation. My prior statement
about much better interactivity when running "blogbench" with the new
implementation could not be confirmed - sometimes interactivity was
great, sometimes bad and sometimes the final results of blogbench were
just 50% of those of better days.
In the following, I compared three different implementations:
* old: old dm_target_crypt
* ng_wp: dm_target_crypt_ng with worker pool
* ng_sync: dm_target_crypt_ng w/o worker pool (this commit)
See below for some benchmarks.
---------------------------------------
Benchmark 1 - write/encrypt performance
---------------------------------------
Test write/encrypt performance by copying a ~800 MB file from
unencrypted fs to encrypted fs (aes-cbc):
time cp /usr/obj/huge.mov /crypt
Results:
Implementation Time Throughput
old 54 s 146 MB/s
ng_wp 43 s 184 MB/s (+ 26%)
ng_sync 40 s 198 MB/s (+ 35%)
--------------------------------------
Benchmark 2 - read/decrypt performance
--------------------------------------
Test read/decrypt performance by reading a ~800 MB file on encrypted
fs (aes-cbc):
time cat /crypt/huge2.mov > /dev/null
Results:
Implementation Time Throughput
old 50 s 158 MB/s
ng_wp 27 s 293 MB/s (+ 85%)
ng_sync 24 s 330 MB/s (+ 108%)
---------------------------------------------
Benchmark 3 - raw encrypt performance aes-cbc
---------------------------------------------
Test raw encrypt performance by copying a file to a ~1 GB
tempfs-backed raw dm-crypt disk using aes-cbc (*).
Results:
Implementation Time Throughput
old 3.6 s 274 MB/s
ng_wp 2.4 s 414 MB/s (+ 51%)
ng_sync 1.8 s 537 MB/s (+ 95%)
---------------------------------------------
Benchmark 4 - raw encrypt performance aes-xts
---------------------------------------------
Test raw encrypt performance by copying a file to a ~1 GB
tempfs-backed raw dm-crypt disk using aes-xts (*).
Results:
Implementation Time Throughput
old 6.5 s 153 MB/s
ng_wp 4.6 s 215 MB/s (+ 40%)
ng_sync 4.1 s 240 MB/s (+ 56%)
---------------------------------------
(*) The following script is used:
dd if=/dev/zero of=/tmp/diskimage bs=1m count=1000
vnconfig -c vn0 /tmp/diskimage
echo "password" > /tmp/keyfile
cryptsetup luksFormat --cipher aes-cbc-essiv:sha256 /dev/vn0 /tmp/keyfile
# or
# cryptsetup luksFormat --cipher aes-xts-essiv:sha256 /dev/vn0 /tmp/keyfile
cryptsetup luksOpen --key-file /tmp/keyfile /dev/vn0 testdisk
cat /usr/obj/video1g.mov > /dev/null
time dd if=/usr/obj/video1g.mov of=/dev/mapper/testdisk conv=sync bs=1m
cryptsetup luksClose testdisk
vnconfig -u vn0
Tomohiro Kusumi [Wed, 9 Apr 2025 10:30:56 +0000 (03:30 -0700)]
sbin/hammer2/cmd_recover.c: Fix wrong media size with multi-volumes
lseek(fd, 0L, SEEK_END) only counts 1 volume.
MediaBytes counts all volumes.
Tomohiro Kusumi [Wed, 9 Apr 2025 09:25:08 +0000 (02:25 -0700)]
sbin/hammer2/cmd_recover.c: Fail fast if destdir is not a directory
If <destdir> is not a directory, syscalls in dump_tree() in restoration
pass are not going to succeed. Then just immediately fail before
blockref scan even starts (or mkdir(2) -p the <destdir>).
Tomohiro Kusumi [Wed, 9 Apr 2025 09:18:51 +0000 (02:18 -0700)]
sbin/hammer2/cmd_recover.c: #if 0 unused struct dirent_entry
Not used for anything as enter_dirent() is also #if0'd.
Matthew Dillon [Wed, 9 Apr 2025 06:37:03 +0000 (23:37 -0700)]
mmap - Add MAP_32BIT (2)
* Fix little niggly omissions
Reminded-by: aly
Matthew Dillon [Wed, 9 Apr 2025 00:44:22 +0000 (17:44 -0700)]
mmap - Add MAP_32BIT
* Add MAP_32BIT, allowing userland to mmap() memory in the 32-bit address
space without having to resort to MAP_TRYFIXED iteration. The mapped
memory may overlap or be within the datasize resource (since this resource
defaults to 32GB, the entire 4GB address space would otherwise be skipped
for regular mmap() calls).
Operations will not overlap any area already fully memory-mapped, such
as program static data and bss areas, stack blocks, or guard pages.
Address zero will not be automatically allocated by this function.
* MAP_32BIT is unable to use the vm_map hinting cache shortcut since this
cache is unordered.
* Adjust a bunch of kernel-internal MAP_* flags that are used to control
copy-on-write and other internal pmap work to COWF_* flags.
* Disentangle COWF_PREFAULT_MADVISE and COWF_PREFAULT_RELOCK. The two
had the same flag value for some reason. This might also fix a madvise()
panic in certain circumstances.
* Adjust manual page.
Requested-by: tuxillo
Tomohiro Kusumi [Tue, 8 Apr 2025 07:48:08 +0000 (00:48 -0700)]
sys/vfs/hammer2: Mark hammer2_base_find's key_end argument as unused
Tomohiro Kusumi [Tue, 8 Apr 2025 07:45:01 +0000 (00:45 -0700)]
sys/vfs/hammer2: Fix potential NULL dereference
These two are passing NULL pointer for hammer2_chain_t*.
Note that this won't happen since hammer2_inode_chain_and_parent()
with a similar logic bug will NULL dereference the chain first,
and that won't happen either since clindex is currently always 0.
Tomohiro Kusumi [Mon, 7 Apr 2025 08:39:03 +0000 (01:39 -0700)]
sys/vfs/hammer2: Remove duplicate [vf]chain.bref.mirror_tid initialization
mirror_tid is initialized once in here and again 50 lines after.
Tomohiro Kusumi [Tue, 8 Apr 2025 07:27:33 +0000 (00:27 -0700)]
sbin/hammer2/cmd_recover.c: Remove duplicate empty blockref check
It's been tested right before this one.
Tomohiro Kusumi [Mon, 7 Apr 2025 08:34:34 +0000 (01:34 -0700)]
sbin/hammer2: Add "freemap" to HAMMER2_CHECK_STRINGS
so that hammer2 freemap directive can properly print "freemap" instead of
"unknown(5)" for meth=... in freemap leaf blockrefs.
Michael Neumann [Mon, 7 Apr 2025 07:28:14 +0000 (09:28 +0200)]
dm_target_crypt_ng - incorporate style suggestions
Suggested-by: Aaron Li
Michael Neumann [Sun, 6 Apr 2025 20:04:30 +0000 (22:04 +0200)]
dm_target_crypt_ng - add a flag to mark sleeping workers
Michael Neumann [Sun, 6 Apr 2025 11:56:55 +0000 (13:56 +0200)]
kern - fix typos
Michael Neumann [Sun, 6 Apr 2025 11:50:47 +0000 (13:50 +0200)]
dm_target_crypt - print error if "crypt" target is already defined
Michael Neumann [Sun, 6 Apr 2025 10:07:14 +0000 (12:07 +0200)]
dm_target_crypt_ng - Add next-generation implementation
* No longer use opencrypto or cryptodev. These were asynchronous and
caused a lot (!) of complications to the code.
* Instead, use our own set of crypto ciphers implemented in
crypto_cipher.{c,h}, which is a simplified API to use symmetric block
ciphers. It mostly calls out to the actual crypto algorithm
implementations contained under sys/crypto.
* Interactivity is greatly improved! When running "blogbench" with the
old dm_target_crypt module, "blogbench" basically "freezes" the system
at around 400 number blogs (Nb). The mouse cursor continuously freezes
for about 1 second. While the computer becomes slower when running
dm_target_crypt_ng, no lag of mouse can be observed. The final
"blogbench" performance result stays about the same. For example:
# dm_target_crypt (old)
Final score for writes: 2136
Final score for reads : 200584
# dm_target_crypt_ng (new)
Final score for writes: 2265
Final score for reads : 203668
* BIOs are processed by two worker pools, each having a set of threads
bound to a particular CPU. Read requests need to be decrypted, while
write requests need to be encrypted. Having a separate worker pool for
read and write requests should provide more fairness. Requests are
scheduled to the workers using round-robin, and "struct bio" is used to
chain "requests" to be processed by worker threads. Read workers need no
further memory to be allocated in order to decrypt the blocks. The write
workers use a shared mpipe as before, as we need to copy the write
buffer before encrypting it.
* Currently, dm_target_crypt_ng only supports AES-CBC and AES-XTS.
Twofish and Serpent are not supported. They are said to be more
secure, but also slower and do not have hardware support. It should be
easy to bring them back into crypto_ciphers.{c,h}.
* If you want to try out the new module, add the following line to
/boot/loader.conf:
dm_target_crypt_ng_load=YES
This might not work with "swapon" as it tries to manually load
"dm_target_crypt".
* My plan is to make "dm_target_crypt_ng" the default and remove the
old "dm_target_crypt" after more intensive testing and feedback.
Michael Neumann [Sun, 6 Apr 2025 10:01:54 +0000 (12:01 +0200)]
dm_target_crypt - Start next-generation "ng" version
This starts out as a copy of dm/crypt in order to more easliy
spot the changes I made.
This kmod is mutual exclusive at runtime with dm/crypt
(dm_target_crypt), as both define the "crypt" target.
Ultimately, this replaces the "legacy" dm_target_crypt.
Michael Neumann [Sun, 6 Apr 2025 09:00:55 +0000 (11:00 +0200)]
crypto - Move core AESNI implementation to sys/crypto/aesni
AESNI is a set of CPU instructions that help to accelerate the AES
cipher. It is available on almost any x86 CPU since about 2010.
Moving this makes the code (and as such the AESNI instruction) usable
from outside the opencrypto framework while keeping aesni(4) from
dev/crypto/aesni still operational.
Michael Neumann [Fri, 4 Apr 2025 12:41:35 +0000 (14:41 +0200)]
crypto - move rmd160.{c,h} from opencrypto/ to crypto/rmd160/
This follows the same approach as other ciphers or hash functions.
Michael Neumann [Fri, 4 Apr 2025 09:15:09 +0000 (11:15 +0200)]
Remove hifn(4), safe(4), ubsec(4) and padlock(4) drivers
These were removed in OpenBSD 7.1 and it's unlikely that
they are still in use by someone today.
Padlock can be used (on VIA CPUs) without having a kernel
driver for it as it is a CPU instruction. OpenSSL, for
instance, ships with a padlock engine and corresponding
assembly code.
Aaron LI [Thu, 3 Apr 2025 12:44:03 +0000 (20:44 +0800)]
<sys/_clock_id.h>: Rename the include guard for consistency
Reported-by: swildner
Aaron LI [Thu, 3 Apr 2025 12:42:59 +0000 (20:42 +0800)]
include: Remove timers.h from Makefile
Fix building.
Reported-by: swildner
Aaron LI [Mon, 31 Mar 2025 05:31:07 +0000 (13:31 +0800)]
Remove <timers.h>
This header just includes <sys/timers.h> and seems to be never used.
Also, this header doesn't exist on NetBSD, OpenBSD, and Linux.
Discussed-with: swildner
Aaron LI [Thu, 3 Apr 2025 11:17:43 +0000 (19:17 +0800)]
rc.d/root: Add 'nojail' keyword to fix jail boot
A jail cannot remount the root filesystem so it was failing to boot
because of error:
Mounting root filesystem rw failed; startup aborted.
Add the 'nojail' keyword to exclude this rc script in jail boot.
FreeBSD also has this keyword for this script for 20+ years.
With this fix, a jail boots OK but there are still some errors during
the boot, so there are more rc scripts that need the 'nojail' keyword
or need tweaks for jail. Will look into this later.
Reported-by: fgudin (Francis GUDIN) on IRC
Aaron LI [Mon, 31 Mar 2025 15:13:31 +0000 (23:13 +0800)]
tcplay(8): Add newline to two fprintf() messages
Aaron LI [Mon, 31 Mar 2025 15:16:47 +0000 (23:16 +0800)]
sys: Use NULL instead of 0 for issignal()
Aaron LI [Mon, 31 Mar 2025 15:12:36 +0000 (23:12 +0800)]
sys: Fix typo (arithmatic -> arithmetic) in comments
Aaron LI [Thu, 27 Mar 2025 06:28:10 +0000 (14:28 +0800)]
sys: Add <sys/_clock_id.h> to deduplicate <sys/time.h> and <time.h>
See-also: https://reviews.freebsd.org/D31056
Discussed-with: swildner
Aaron LI [Fri, 14 Mar 2025 02:41:30 +0000 (10:41 +0800)]
dsynth(1): Use PROC_REAP_KILL to reap all