Add integrated dports build feature to build-all and install-all targets When WITH_DPORTS=YES is used build-all and install-all, these targets will also build the base set of ports and all of their dependencies using configuration data and the "dragonfly/base" dport found in "/usr/src/dports.base". This feature is intended to be used by the nrelease build in an upcoming commit, and may also be used manually. The /usr/dports tree must exist and contains the dports repo to use as the basis of the dports build. It will be null-mounted in the chroot and the "/usr/src/dports.base/dragonfly" directory will be null-mounted under it to provide the "dragonfly/base" meta-port. The DPORTSBASE make variable may be used to override the location of the main dports repo. The build-all and install-all targets use a sophisticated chrooted environment to do a completely clean dports build with dsynth. It It will actually "make installworld" plus a clean "make distribution" inside the chroot environment to create the environment, and the host system's /etc/resolv.conf will be copied in. build-all - Augmented when used WITH_DPORTS=YES install-all - Augmented when used WITH_DPORTS=YES buildportschroot - Build a fresh chroot environment for dsynth mountports - Do all null mounts and cp's needed for the env buildports - Run dsynth build dragonfly/base in the env installports - Install the built ports on the host system (see note) umountports - Remove the null mounts WITH_DPORTS=YES - executes buildportschroot, mountports, buildports, and umountports when run from build-all. executes mountports, installports, umountports when run from install-all. DPORTSBASE=path - Defaults to /usr/dports, may be used to override the location where the main dports tree can be found. NOTE: Installports may require interactivity as the base system might already have installed ports. Generally speaking it is best to not revert any ports you might already have installed, so feel free to tell it not to install anything. * Added various make targets and augmented build-all and install-all * /usr/local/sbin added and the stale /usr/pkg/bin removed from _HOSTPATH. * Add /usr/src/dports.base infrastructure. It contains the dsynth.ini and pkg.conf files that will be used for the local dsynth operations. Some paths are replaced with SED when copied to the chroot. The dsynth.ini is relative to the chroot. It also contains the dragonfly/base dport which is the meta-port that dsynth will be told to build. All other dports related files and directories are null-mounted from /usr/dports. * Add /usr/distfiles.base to mtree/BSD.usr.dist. This directory is also created by the build if it does not exist since the host system might not have run a new installworld recently. This directory will be used during build-all (buildports) to store the distfiles required to build dragonfly/base, so they only need to be fetched once. * Add the "-C ConfigBase" option to dsynth to make running it from the chrooted environment easier. This overrides the /etc/dsynth directory and removes /etc/dsynth as a fallback. * dsynth conditionalizes calling syscap_set() for backwards compatibility with older host systems that might not have been upgraded to include the feature yet.
Makefile: Make 'installworld' depend on 'preupgrade' The 'preupgrade' target will creates new users/groups, which may be required by the 'installworld' target, so make 'installworld' depend on 'preupgrade'. This will fix the installation of nvmmctl(8) that requires the 'nvmm' group. As another result, no longer need to check for users/groups in the 'installcheck' target.
build - Add build-all and install-all targets * Add a 'smart' build-all and install-all target that spews to a log file instead of to the screen and. Will automatically run the build stage -j N, and both targets can be specified on one command line. make build-all install-all build-all: runs buildworld and buildkernel -j hw.ncpu install-all: runs installkernel and installworld -j 1 * The usual warnings and cautions still apply when installing a new world and kernel.
Remove groff from base. There is a dports package available for people who need it. Also, for some manual pages in dports, man(1) will report that mandoc(1) cannot render them, and advise to install groff too, which man(1) will then use. Also remove vgrind(1) and the old base me(7) macros. Disable the groff part of manlint for now, also mandiff. I will decide later if it should be made to work with dports groff. There are a number of related utilities, like colcrt, checknr etc. that I have not touched yet. Their fate will be decided later. There are also some references to these tools left in existing manual pages.
initrd: Rework build and install stages * Rewrite the Makefile to provide only the 'all' and 'install' targets. Do not separate the handling of the rescue tools and initrd image. Thus greatly simplify the whole logic. The 'all' target will build the rescue tools and initrd contents, and does not require root priviledge, while the 'install' target will install the rescue tools to the system, create the initrd image and install it. * Update the top-level makefiles to call the new 'all' and 'install'. The 'buildworld' target now doesn't require root priviledge, same as before the refactoring in 9b724c0dcb2b9548a82d28d97e34375f64668669. * Only provide the 'initrd' top-level target to keep it simple. This target will install the rescue tools and initrd image built by the 'buildworld', overwriting the existing ones. * Clean up the nrelease/Makefile a bit, since 'installworld' will also install the rescue tools and initrd image in such a case. * Update the mkinitrd.sh script a bit. * Update the build(7) man page and clean up a bit. Reviewed-by: swildner
build - refactor rescue and initrd handling * buildworld now builds the rescue directory tree and initrd image in /usr/obj. * installworld will install the rescue and initrd stuff if it does not exist on the destination. If rescue or initrd stuff is already installed, it is not overwritten. * Add 'installrescue' and 'installinitrd' targets. These will copy the rescue and initrd image built by buildworld. * The normal 'rescue' and 'initrd' targets are unchanged and should build (from scratch) and install the rescue and initrd stuff. * NOTE: crunchgen is currently hacked to add system paths when WORLDBUILD is defined, in order to access native binaries for 'uniq' and a few others, instead of building them in cross-tools. In particular, we have to use the native 'vnconfig' binary as building one in crosstools might not be compatible with the currently running system. buildworld now uses 'vn' to create the initrd image. Possibly needs some work.
initrd: Refactor Makefile and introduce quick{rescue,initrd} Refactor the Makefile by splitting the original targets into smaller ones. Introduce targets quick{rescue,initrd} which skip the clean step. Such similar behavior was achieved by passing '-DNO_CLEAN' to make. (suggested-by: swildner)
initrd: Detach from the world and build on the fly Why --- Currently, the files for initrd image creation are built and installed at /usr/share/initrd by "buildworld" and "installworld", respectively, and then mkinitrd(8) simply packs them to create the initrd.img.gz. For normal users, the shipped /usr/share/initrd and mkinitrd(8) don't make much sense, since they don't need to modify the initrd contents and create a modified initrd image. From a developer's perspective, the required steps (buildworld, installworld and mkinitrd) to create and test a new initrd image can be annoying. How --- Detach the build and installation of the initrd contents from the world. Build the (rescue/initrd) tools on-the-fly when creating the initrd image. As per ftigeot's suggestion, these statically linked tools can be useful when dealing with a broken system, so install them under /rescue to be more intuitive and easier to use (similar to FreeBSD and maybe other BSDs). What ---- * Move "share/initrd" to be top-level, and detach it from world. + Update the patch in bsd.crunchgen.mk accordingly. + Add Makefile.inc to simplify subdir's makefiles. + Rewrite the Makefile with targets: - rescue: build and install the rescue tools into /rescue. - initrd: further packs the etc and rescue staffs to create the initrd image. * Move "sbin/mkinitrd/mkinitrd.sh" under the new "initrd" directory and adapt it to work with the "initrd" make target. Remove the other parts of mkinitrd(8). * Export the above "rescue" and "initrd" targets to be top-level. * Update nrelease framework to use the new "make initrd". * Update build.7 man page accordingly. * Print the message about updating the initrd image and 3rd-party packages after "make upgrade" instead of "make installworld". * Document this change in UPDATING. * Other small updates and style cleanups. TODO ---- * Add initrd.7 man page * Crunch more useful tools * Crunch bin/sbin/etc. into a bundle to save space (??) Reviewed-by: swildner Thanks-to: dillon, swildner, ftigeot
Fix lack of buildworld and quickworld messages The "world" target has a nice start/stop message. However, if somebody chooses to use "make buildworld" followed by "make installworld" rather than "make world", they get no messages at all. I think this has been an ongoing oversight. This commit improves the "world" target messages (e.g. all targets are ELF so don't need to specify that, and pre/post-world targets are shown.) More importantly it adds start/complete messages to buildworld, and start message to installworld. The installworld already had a complete message, but this was redone to match the format of the other ones. Similar messages were added to quickworld and realquickworld targets too.
build - add installworld-force target and better checks * Add better checks to installworld. Print an error and exit if the base system is too old, directing users to run installworld-force. * Add installworld-force. This automates installing the world on a base system which is too old and thus has chicken-and-egg problems with binaries and libraries. This install target will install to a temporary directory, compile up a static cpdup, and then use it to just blast copy /bin, /sbin, /lib, /libexec, /usr/bin, /usr/sbin, /usr/lib, and /usr/libexec. It will then execute a normal make installworld and make upgrade.