From: Aaron LI Date: Sat, 24 Nov 2018 10:52:03 +0000 (+0800) Subject: initrd: Rework build and install stages X-Git-Tag: v5.7.0~743 X-Git-Url: https://gitweb.dragonflybsd.org/dragonfly.git/commitdiff_plain/bc897c50d49f0df1bf5eea38a4a7d904c2e92795 initrd: Rework build and install stages * Rewrite the Makefile to provide only the 'all' and 'install' targets. Do not separate the handling of the rescue tools and initrd image. Thus greatly simplify the whole logic. The 'all' target will build the rescue tools and initrd contents, and does not require root priviledge, while the 'install' target will install the rescue tools to the system, create the initrd image and install it. * Update the top-level makefiles to call the new 'all' and 'install'. The 'buildworld' target now doesn't require root priviledge, same as before the refactoring in 9b724c0dcb2b9548a82d28d97e34375f64668669. * Only provide the 'initrd' top-level target to keep it simple. This target will install the rescue tools and initrd image built by the 'buildworld', overwriting the existing ones. * Clean up the nrelease/Makefile a bit, since 'installworld' will also install the rescue tools and initrd image in such a case. * Update the mkinitrd.sh script a bit. * Update the build(7) man page and clean up a bit. Reviewed-by: swildner --- diff --git a/Makefile b/Makefile index 7ec8ebc22f..837188e2f3 100644 --- a/Makefile +++ b/Makefile @@ -8,12 +8,10 @@ # quickworld - Skip bootstrap, build and cross-build tool steps. # realquickworld - Skip above steps, plus depend. # crossworld - Just do the bootstrap, build, and cross-build steps. -# installworld - Install everything built by "buildworld". -# (includes rescue and initrd image) +# installworld - Install everything built by "buildworld", and the +# rescue tools and initrd image if they do not exist. # installworld-force - Install everything built by "buildworld"; # special case for old systems. -# installrescue - Install /rescue built by buildworld -# installinitrd - Install initrd image built by buildworld # world - buildworld + installworld. # buildkernel - Rebuild the kernel and the kernel-modules from scratch # using build/bootstrap/cross tools from the last @@ -31,8 +29,8 @@ # the installworld. # upgrade - Upgrade the files in /etc and also setup the rest # of the system for DragonFly. ex. two compilers. -# rescue - Build and install the statically linked rescue tools. -# initrd - Build the rescue tools and create the initrd image. +# initrd - Install the statically linked rescue tools and the +# initrd image built by "buildworld". # backupworld - Copy /bin /sbin /usr/bin /usr/sbin /usr/lib # /usr/libexec to manual backup dir. # restoreworld - Install binaries from manual backup dir to world. @@ -67,8 +65,7 @@ # 5. `make installworld' # 6. `make upgrade' # 7. `reboot' -# 8. `make installrescue' (after making sure that the new world works well). -# 9. `make installinitrd' (after making sure that the new world works well). +# 8. `make initrd' (after making sure that the new world works well). # # If TARGET_ARCH=arch (e.g. x86_64) is specified you can # cross build world for other architectures using the buildworld target, @@ -82,9 +79,7 @@ TGTS= all all-man buildkernel quickkernel realquickkernel nativekernel \ buildworld crossworld quickworld realquickworld checkdpadd clean \ cleandepend cleandir depend everything \ hierarchy install installcheck installkernel \ - reinstallkernel installworld installworld-force \ - rescue initrd \ - quickrescue quickinitrd installrescue installinitrd \ + reinstallkernel installworld installworld-force initrd \ libraries lint maninstall \ manlint mk obj objlink regress rerelease tags \ backupworld restoreworld restoreworld-auto \ @@ -119,7 +114,7 @@ MAKE= PATH=${PATH} make -m ${.CURDIR}/share/mk -f Makefile.inc1 # # Handle the user-driven targets, using the source relative mk files. # -${TGTS} ${BITGTS}: +${TGTS} ${BITGTS}: .PHONY @cd ${.CURDIR}; \ ${MAKE} ${.TARGET} diff --git a/Makefile.inc1 b/Makefile.inc1 index 3bf48f2d2d..f53282022c 100644 --- a/Makefile.inc1 +++ b/Makefile.inc1 @@ -295,10 +295,10 @@ CROSSENV= MAKEOBJDIRPREFIX=${WORLDDEST} \ HOST_CCVER=${HOST_CCVER} \ CCVER=${WORLD_CCVER} \ LDVER=${WORLD_LDVER} \ - WORLDBUILD=1 \ BINUTILSVER=${WORLD_BINUTILSVER} WMAKEENV= ${CROSSENV} \ + WORLDBUILD=1 \ DESTDIR=${WORLDDEST} \ _SHLIBDIRPREFIX=${WORLDDEST} \ INSTALL="sh ${.CURDIR}/tools/install.sh" \ @@ -408,18 +408,12 @@ everything: @echo ">>> stage 4d: building everything.." @echo "--------------------------------------------------------------" cd ${.CURDIR}; ${WMAKE} all -_rescue: - @echo - @echo "--------------------------------------------------------------" - @echo ">>> stage 5a: building rescue topology" - @echo "--------------------------------------------------------------" - cd ${.CURDIR}; ${WMAKE} quickrescue _initrd: @echo @echo "--------------------------------------------------------------" - @echo ">>> stage 5b: building initrd topology" + @echo ">>> stage 5: building rescue and initrd" @echo "--------------------------------------------------------------" - cd ${.CURDIR}; ${WMAKE} quickinitrd + cd ${.CURDIR}/initrd; ${WMAKEENV} make all _bwdone: @echo "--------------------------------------------------------------" @echo ">>> buildworld target complete" @@ -455,16 +449,14 @@ WMAKE_TGTS+= _obj _build-tools .if !defined(SUBDIR_OVERRIDE) WMAKE_TGTS+= _cross-tools .endif -WMAKE_TGTS+= _includes _libraries _depend everything -WMAKE_TGTS+= _rescue _initrd +WMAKE_TGTS+= _includes _libraries _depend everything _initrd .if defined(WMAKE_TGTS_OVERRIDE) SUBDIR= WMAKE_TGTS= ${WMAKE_TGTS_OVERRIDE} .endif -QMAKE_TGTS= _mtreetmp _obj _includes _libraries _depend everything -QMAKE_TGTS+= _rescue _initrd +QMAKE_TGTS= _mtreetmp _obj _includes _libraries _depend everything _initrd buildworld: _bwinit ${WMAKE_TGTS} _bwdone @@ -501,12 +493,6 @@ installcheck: _iwinit # Backs up the current world if ${AUTO_BACKUP} is writable. # Installs everything compiled by a 'buildworld'. # -.if (!defined(DESTDIR) || ${DESTDIR} == "" || ${DESTDIR} == "/") -DESTDIR_RESCUE=/ -.else -DESTDIR_RESCUE=${DESTDIR} -.endif - installworld: installcheck -@mkdir -p ${AUTO_BACKUP} > /dev/null 2>&1 @cd ${.CURDIR}; \ @@ -517,32 +503,13 @@ installworld: installcheck cd ${.CURDIR}; ${IMAKE} re${.TARGET:S/world$//} ${INSTALL} -o root -g wheel -m 644 ${.CURDIR}/Makefile_upgrade.inc \ ${DESTDIR}/etc/upgrade/ - -.if !exists(${DESTDIR_RESCUE}/rescue/rescue) - @echo "--------------------------------------------------------------" - @echo ">>> /rescue is empty, setting up" - @echo "--------------------------------------------------------------" - (cd ${.CURDIR} && make installrescue) -.endif -.if !exists(${DESTDIR_RESCUE}/boot/kernel/initrd.img.gz) - @echo "--------------------------------------------------------------" - @echo ">>> /boot/kernel has no initrd.img.gz, setting up" - @echo "--------------------------------------------------------------" - (cd ${.CURDIR} && make installinitrd) -.endif + cd ${.CURDIR}; ${IMAKE} -DWORLDINSTALL initrd sync @echo "--------------------------------------------------------------" @echo ">>> installworld target complete" @echo "--------------------------------------------------------------" -installrescue: - cpdup -i0 ${WORLDDEST}/rescue ${DESTDIR}/rescue - -installinitrd: - ${INSTALL} -o root -g wheel -m 644 \ - ${WORLDDEST}/boot/kernel/initrd.img.gz ${DESTDIR}/boot/kernel/ - installworld-force: @echo "Doing a forced installworld. This will install to a temporary directory," @echo "then copy the main binaries and libraries with a static cpdup to ${DESTDIR}/" @@ -570,6 +537,7 @@ installworld-force: sleep 5 (cd ${.CURDIR}; ${MAKE} installworld OVERRIDE_CHECKS=TRUE) (cd ${.CURDIR}; ${MAKE} upgrade OVERRIDE_CHECKS=TRUE) + (cd ${.CURDIR}; ${MAKE} initrd) # # reinstall @@ -589,6 +557,13 @@ reinstall: @echo "--------------------------------------------------------------" cd ${.CURDIR}; make -f Makefile.inc1 install +# initrd +# +# Install the rescue tools and the initrd image built by 'buildworld'. +# +initrd: .PHONY + (cd ${.CURDIR}/initrd; ${IMAKEENV} make install) + # # buildkernel, nativekernel, quickkernel, realquickkernel, and installkernel # @@ -1234,12 +1209,6 @@ backup-auto-clean: backup-clean: rm -f ${WORLD_BACKUP}/binaries.tar.gz -# Build and install the statically linked rescue tools, and create the -# initrd image. -# -rescue initrd quickrescue quickinitrd: .PHONY - (cd ${.CURDIR}/initrd; make ${.TARGET}) - restoreworld: .if !exists(${WORLD_BACKUP}/binaries.tar.gz) @echo "There does not seem to be a valid archive present." diff --git a/initrd/Makefile b/initrd/Makefile index e220a3958b..730303c308 100644 --- a/initrd/Makefile +++ b/initrd/Makefile @@ -1,68 +1,64 @@ INITRD_MAKE= LC_ALL=C \ make -m ${.CURDIR}/../share/mk +INITRD_SUBDIRS= etc +RESCUE_SUBDIRS= oinit rescue rescue.libcrypto -# Target directory to install the rescue tools -RESCUE_DIR?= /rescue +# Temporary directory to stage the files for creating the initrd image. +# +# NOTE: Do not place this directory in the object tree, so that the entire +# object tree can be exported read-only for upgrading other machines. +# +TMPDIR?= /tmp +STAGEDIR:= ${TMPDIR}/initrd_staged.${.MAKE.PID} -# Temporary directory to stage the initrd files (e.g., /etc) -# NOTE: Ignore ${DESTDIR} when staging files into this directory -INITRD_STAGEDIR!= mktemp -d -t initrd-stage -.END: - @rm -rf ${INITRD_STAGEDIR} - -_build_rescue: - @echo ">>> Building rescue tools ..." -.for _dir in oinit rescue rescue.libcrypto - (cd ${.CURDIR}/${_dir} && \ - ${INITRD_MAKE} obj && \ +all: + @echo "--------------------------------------------------------------" + @echo ">>> building rescue tools and initrd contents ..." + @echo "--------------------------------------------------------------" +.for _dir in ${RESCUE_SUBDIRS} ${INITRD_SUBDIRS} + (cd ${.CURDIR}/${_dir}; \ + ${INITRD_MAKE} obj; \ ${INITRD_MAKE} depend all) .endfor -_install_rescue: - @echo ">>> Installing rescue tools ..." -.for _dir in oinit rescue rescue.libcrypto - (cd ${.CURDIR}/${_dir} && \ - ${INITRD_MAKE} BINDIR=${RESCUE_DIR} install) -.endfor -_stage_initrd: - @echo ">>> Staging initrd contents ..." - (cd ${.CURDIR}/etc && \ - mkdir ${INITRD_STAGEDIR}/etc && \ - ${INITRD_MAKE} obj && \ - ${INITRD_MAKE} depend all && \ - ${INITRD_MAKE} BINDIR=${INITRD_STAGEDIR}/etc DESTDIR="" install) +install: + # If called by 'installworld' (i.e., 'WORLDINSTALL' defined), + # do not overwrite the existing ones. +.if !(defined(WORLDINSTALL) && exists(${DESTDIR}/rescue/rescue)) + @echo "--------------------------------------------------------------" + @echo ">>> installing rescue tools to ${DESTDIR}/rescue ..." + @echo "--------------------------------------------------------------" +.for _dir in ${RESCUE_SUBDIRS} + (cd ${.CURDIR}/${_dir}; \ + ${INITRD_MAKE} BINDIR=/rescue install) +.endfor +.endif -_make_initrd: - @echo ">>> Creating initrd.img.gz ..." +.if !(defined(WORLDINSTALL) && exists(${DESTDIR}/boot/kernel/initrd.img.gz)) + @echo "--------------------------------------------------------------" + @echo ">>> setting up ${DESTDIR}/boot/kernel/initrd.img.gz ..." + @echo "--------------------------------------------------------------" + rm -rf ${STAGEDIR} +.for _dir in ${INITRD_SUBDIRS} + (cd ${.CURDIR}/${_dir}; \ + mkdir -p ${STAGEDIR}/${_dir}; \ + ${INITRD_MAKE} BINDIR=${STAGEDIR}/${_dir} DESTDIR="" install) +.endfor sh ${.CURDIR}/mkinitrd.sh \ -b ${DESTDIR}/boot \ - -r ${DESTDIR}${RESCUE_DIR} \ - -c ${INITRD_STAGEDIR} + -r ${DESTDIR}/rescue \ + -c ${STAGEDIR} + rm -rf ${STAGEDIR} +.endif -_clean_rescue: - @echo ">>> Cleaning rescue ..." -.for _dir in oinit rescue rescue.libcrypto + +clean: +.for _dir in ${RESCUE_SUBDIRS} ${INITRD_SUBDIRS} (cd ${.CURDIR}/${_dir} && ${INITRD_MAKE} clean cleandepend) .endfor -_clean_initrd: - @echo ">>> Cleaning initrd ..." - (cd ${.CURDIR}/etc && ${INITRD_MAKE} clean cleandepend) - -clean: _clean_rescue _clean_initrd - -_quickrescue: _build_rescue _install_rescue -rescue: _clean_rescue _quickrescue .PHONY -.ORDER: _clean_rescue _build_rescue _install_rescue - -_quickinitrd: _stage_initrd _make_initrd -_initrd: _clean_initrd _quickinitrd -.ORDER: _clean_initrd _stage_initrd _make_initrd -quickrescue: _quickrescue -quickinitrd: _quickrescue _quickinitrd -initrd: rescue _initrd -.ORDER: _quickrescue _quickinitrd -.ORDER: rescue _initrd +.ORDER: all install +.MAIN: all diff --git a/initrd/mkinitrd.sh b/initrd/mkinitrd.sh index c6b4305179..7c54767816 100755 --- a/initrd/mkinitrd.sh +++ b/initrd/mkinitrd.sh @@ -65,7 +65,7 @@ BOOT_DIR="/boot" # Maximum size (number of MB) allowed for the initrd image INITRD_SIZE_MAX="15" # MB -# When run from the buildworld environment do not require that +# When run from the buildworld/installworld environment do not require that # things like uniq, kldload, mount, newfs, etc be in the cross-tools. # These must run natively for the current system version. # @@ -131,12 +131,15 @@ calc_initrd_size() { mb = $1/1024/1024; print ceil(mb); }') - # Add additional 1 MB + # Reserve another 1 MB for advanced user to add custom files to the + # initrd without creating it from scratch. echo $((${isize_mb} + 1)) } create_vn() { - kldload -n vn + kldstat -qm vn || kldload -n vn || + error 1 "Failed to load vn kernel module" + VN_DEV=$(vnconfig -c -S ${INITRD_SIZE}m -Z -T vn ${INITRD_FILE}) && echo "Configured ${VN_DEV}" || error 1 "Failed to configure VN device" @@ -298,7 +301,7 @@ if [ -f "${INITRD_DEST}" ]; then echo " OK (${INITRD_DEST}.old)" fi -echo -n "Copying ${INITRD_FILE}.gz to ${INITRD_DEST} ..." +echo -n "Installing ${INITRD_FILE}.gz to ${INITRD_DEST} ..." install -o root -g wheel -m 444 ${INITRD_FILE}.gz ${INITRD_DEST} echo " OK" rm -f ${INITRD_FILE}.gz diff --git a/nrelease/Makefile b/nrelease/Makefile index a116495be1..f429a3081d 100644 --- a/nrelease/Makefile +++ b/nrelease/Makefile @@ -191,13 +191,12 @@ buildiso: ( cd ${.CURDIR}/../etc; MAKEOBJDIRPREFIX=${NRLOBJDIR}/nrelease \ make -m ${.CURDIR}/../share/mk DESTDIR=${ISOROOT} distribution ) cpdup ${ISOROOT}/etc ${ISOROOT}/etc.hdd - cd ${.CURDIR}/..; \ - for kernconf in ${KERNCONF}; do \ - make DESTDIR=${ISOROOT} reinstallkernel KERNCONF=$${kernconf}; \ - done + ( cd ${.CURDIR}/..; \ + for kernconf in ${KERNCONF}; do \ + make DESTDIR=${ISOROOT} reinstallkernel KERNCONF=$${kernconf}; \ + done ) rm -rf ${ISOROOT}/boot/kernel.old ln -sf kernel ${ISOROOT}/boot/kernel/kernel.BOOTP - ( cd ${.CURDIR}/..; make DESTDIR=${ISOROOT} initrd ) mtree -deU -f ${.CURDIR}/../etc/mtree/BSD.var.dist -p ${ISOROOT}/var dev_mkdb -f ${ISOROOT}/var/run/dev.db ${ISOROOT}/dev diff --git a/share/man/man7/build.7 b/share/man/man7/build.7 index 41f976e979..d623d4bbb9 100644 --- a/share/man/man7/build.7 +++ b/share/man/man7/build.7 @@ -24,7 +24,7 @@ .\" .\" $FreeBSD: src/share/man/man7/build.7,v 1.19.2.1 2002/03/18 08:33:02 murray Exp $ .\" -.Dd June 27, 2018 +.Dd November 23, 2018 .Dt BUILD 7 .Os .Sh NAME @@ -99,7 +99,9 @@ but also skip the depend step. Just do the bootstrap, build and cross-build steps. .It Cm installworld Install everything built by -.Cm buildworld . +.Cm buildworld , +as well as the rescue tools and the initial ramdisk if they do not exist +yet. .It Cm installworld-force Force an .Cm installworld . @@ -152,10 +154,9 @@ Upgrade the files in /etc and also setup the rest of the system for the version of .Dx just installed. -.It Cm rescue -Build and install the statically linked rescue tools. .It Cm initrd -Create the initial ramdisk based on the above rescue tools and install. +Install the statically linked rescue tools and the initial ramdisk built by +.Cm buildworld . .It Cm backupworld Manually archive binaries from installed world to location specified by .Ev WORLD_BACKUP . @@ -303,7 +304,9 @@ make initrd It is usually a good idea to run this command after rebooting into the new world that you installed (so you know the world you installed is good). This command will update the initrd image in -.Pa /boot/kernel . +.Pa /boot/kernel +and the rescue tools in +.Pa /rescue . .Sh FILES .Bl -tag -width ".Pa /usr/src/Makefile_upgrade.inc" -compact .It Pa /etc/make.conf @@ -323,14 +326,10 @@ make buildkernel KERNCONF=FOO make installkernel KERNCONF=FOO make installworld make upgrade +reboot +make initrd .Ed .Pp -After running these commands a system reboot is required, -otherwise many programs which have been rebuilt (such as -.Xr ps 1 , -.Xr top 1 , -etc.) may not work with the old kernel which is still running. -.Pp The above mentioned build and install order enforces that the new kernel is installed before the new world. Sometimes it might be necessary to reboot the system between those two