Update some copyright notices to become more legal compliant.
[dragonfly.git] / share / man / man7 / development.7
CommitLineData
033a4603 1.\" Copyright (c) 1998 Matthew Dillon. Terms and conditions are those of
984263bc
MD
2.\" the BSD Copyright as specified in the file "/usr/src/COPYRIGHT" in
3.\" the FreeBSD source tree.
4.\"
5.\" $FreeBSD: src/share/man/man7/development.7,v 1.4.2.2 2003/05/23 07:48:35 brueffer Exp $
033a4603 6.\" $DragonFly: src/share/man/man7/development.7,v 1.3 2004/06/28 02:57:10 drhodus Exp $
984263bc
MD
7.\"
8.Dd December 21, 2002
9.Dt DEVELOPMENT 7
10.Os
11.Sh NAME
12.Nm development
13.Nd introduction to development with the FreeBSD codebase
14.Sh DESCRIPTION
15This manual page describes how an ordinary sysop,
16.Ux admin, or developer
17can, without any special permission, obtain, maintain, and modify the
18.Fx
19codebase as well as how to maintaining a master build which can
20then be exported to other machines in your network.
21This manual page
22is targeted to system operators, programmers, and developers.
23.Pp
24Please note that what is being described here is based on a complete
25FreeBSD environment, not just the FreeBSD kernel.
26The methods described
27here are as applicable to production installations as it is to development
28environments.
29You need a good 12-17GB of disk space on one machine to make this work
30conveniently.
31.Sh SETTING UP THE ENVIRONMENT ON THE MASTER SERVER
32Your master server should always run a stable, production version of the
33.Fx
34operating system. This does not prevent you from doing -CURRENT
35builds or development. The last thing you want to do is to run an
36unstable environment on your master server which could lead to a situation
37where you lose the environment and/or cannot recover from a mistake.
38.Pp
39Create a huge partition called /FreeBSD.
408-12GB is recommended.
41This partition will contain nearly all the development environment,
42including the CVS tree, broken-out source, and possibly even object files.
43You are going to export this partition to your other machines via a
44READ-ONLY NFS export so do not mix it with other more security-sensitive
45partitions.
46.Pp
47You have to make a choice in regards to
48.Pa /usr/obj .
49You can put
50.Pa /usr/obj
51in
52.Pa /FreeBSD
53or you can make
54.Pa /usr/obj
55its own partition.
56I recommend making it a separate partition for several reasons. First,
57as a safety measure since this partition is written to a great deal.
58Second, because you typically do not have to back it up.
59Third, because it makes it far easier to mix and match the development
60environments which are described later in this document.
61I recommend a
62.Pa /usr/obj
63partition of at least 5GB.
64.Pp
65On the master server, use cvsup to automatically pull down and maintain
66the
67.Fx
68CVS archive once a day. The first pull will take a long time,
69it is several gigabytes, but once you have it the daily syncs will be quite
70small.
71.Bd -literal -offset 4n
72mkdir /FreeBSD/FreeBSD-CVS
73rm -rf /home/ncvs
74ln -s /FreeBSD/FreeBSD-CVS /home/ncvs
75.Ed
76.Pp
77The cron job should look something like this (please randomize the time of
78day!).
79Note that you can use the cvsup file example directly from
80/usr/share/examples without modification by supplying appropriate arguments
81to cvsup.
82.Bd -literal -offset 4n
8333 6 * * * /usr/local/bin/cvsup -g -r 20 -L 2 -h cvsup.freebsd.org /usr/share/examples/cvsup/cvs-supfile
84.Ed
85.Pp
86Run the cvsup manually the first time to pull down the archive. It could take
87all day depending on how fast your connection is!
88You will run all cvsup and cvs operations as root and you need to set
89up a ~/.cvsrc (/root/.cvsrc) file, as shown below, for proper cvs operation.
90Using ~/.cvsrc to specify cvs defaults is an excellent way
91to "file and forget", but you should never forget that you put them in there.
92.Bd -literal -offset 4n
93# cvs -q
94diff -u
95update -Pd
96checkout -P
97.Ed
98.Pp
99Now use cvs to checkout a -STABLE source tree and a -CURRENT source tree,
100as well as ports and docs, to create your initial source environment.
101Keeping the broken-out source and ports in /FreeBSD allows you to export
102it to other machines via read-only NFS.
103This also means you only need to edit/maintain files in one place and all
104your clients automatically pick up the changes.
105.Bd -literal -offset 4n
106mkdir /FreeBSD/FreeBSD-4.x
107mkdir /FreeBSD/FreeBSD-current
108
109cd /FreeBSD/FreeBSD-4.x
110cvs -d /home/ncvs checkout -rRELENG_4 src
111
112cd /FreeBSD/FreeBSD-current
113cvs -d /home/ncvs checkout src
114cvs -d /home/ncvs checkout ports
115cvs -d /home/ncvs checkout doc
116.Ed
117.Pp
118Now create a softlink for /usr/src and /usr/src2.
119On the main server I always point /usr/src at -STABLE and /usr/src2 at
120-CURRENT. On client machines I usually do not have a /usr/src2 and I make
121/usr/src point at whatever version of FreeBSD the client box is intended to
122run.
123.Bd -literal -offset 4n
124cd /usr
125rm -rf src src2
126ln -s /FreeBSD/FreeBSD-4.x/src src (could be -CURRENT on a client)
127ln -s /FreeBSD/FreeBSD-current/src src2 (MASTER SERVER ONLY)
128.Ed
129.Pp
130Now you have to make a choice for /usr/obj.
131Well, hopefully you made it already and chose the partition method. If you
132chose poorly you probably intend to put it in /FreeBSD and, if so, this is
133what you want to do:
134.Bd -literal -offset 4n
135(ONLY IF YOU MADE A POOR CHOICE AND PUT /usr/obj in /FreeBSD!)
136mkdir /FreeBSD/obj
137cd /usr
138rm -rf obj
139ln -s /FreeBSD/obj obj
140.Ed
141.Pp
142Alternatively you may chose simply to leave /usr/obj in /usr. If your
143/usr is large enough this will work, but I do not recommend it for
144safety reasons (/usr/obj is constantly being modified, /usr is not).
145.Pp
146Note that exporting /usr/obj via read-only NFS to your other boxes will
147allow you to build on your main server and install from your other boxes.
148If you also want to do builds on some or all of the clients you can simply
149have /usr/obj be a local directory on those clients.
150You should never export /usr/obj read-write, it will lead to all sorts of
151problems and issues down the line and presents a security problem as well.
152It is far easier to do builds on the master server and then only do installs
153on the clients.
154.Pp
155I usually maintain my ports tree via CVS.
156It is sitting right there in the master CVS archive and I've even told you
157to check it out (see above).
158With some fancy softlinks you can make the ports tree available both on your
159master server and on all of your other machines.
160Note that the ports tree exists only on the HEAD cvs branch, so its always
161-CURRENT even on a -STABLE box. This is what you do:
162.Bd -literal -offset 4n
163(THESE COMMANDS ON THE MASTER SERVER AND ON ALL CLIENTS)
164cd /usr
165rm -rf ports
166ln -s /FreeBSD/FreeBSD-current/ports ports
167
168cd /usr/ports (this pushes into the softlink)
169rm -rf distfiles (ON MASTER SERVER ONLY)
170ln -s /usr/ports.distfiles distfiles (ON MASTER SERVER ONLY)
171
172mkdir /usr/ports.distfiles
173mkdir /usr/ports.workdir
174.Ed
175.Pp
176Since /usr/ports is softlinked into what will be read-only on all of your
177clients, you have to tell the ports system to use a different working
178directory to hold ports builds.
179You want to add a line to your /etc/make.conf file on the master server
180and on all your clients:
181.Bd -literal -offset 4n
182WRKDIRPREFIX=/usr/ports.workdir
183.Ed
184.Pp
185You should try to make the directory you use for the ports working directory
186as well as the directory used to hold distfiles consistent across all of your
187machines.
188If there isn't enough room in /usr/ports.distfiles and /usr/ports.workdir I
189usually make those softlinks (since this is on /usr these are per-machine) to
190where the distfiles and working space really are.
191.Sh EXPORTING VIA NFS FROM THE MASTER SERVER
192The master server needs to export /FreeBSD and /usr/obj via NFS so all the
193rest of your machines can get at them.
194I strongly recommend using a read-only export for both security and safety.
195The environment I am describing in this manual page is designed primarily
196around read-only NFS exports.
197Your exports file on the master server should contain the following lines:
198.Bd -literal -offset 4n
199/FreeBSD -ro -alldirs -maproot=root: -network YOURLAN -mask YOURLANMASK
200/usr/obj -ro -alldirs -maproot=root: -network YOURLAN -mask YOURLANMASK
201.Ed
202.Pp
203Of course, NFS server operations must also be configured on that machine.
204This is typically done via your /etc/rc.conf:
205.Bd -literal -offset 4n
206nfs_server_enable="YES"
207nfs_server_flags="-u -t -n 4"
208.Ed
209.Sh THE CLIENT ENVIRONMENT
210All of your client machines can import the development/build environment
211directory simply by NFS mounting /FreeBSD and /usr/obj from the master
212server.
213A typical /etc/fstab entry on your client machines will be something like this:
214.Bd -literal -offset 4n
215masterserver:/FreeBSD /FreeBSD nfs ro,bg 0 0
216masterserver:/usr/obj /usr/obj nfs ro,bg 0 0
217.Ed
218.Pp
219And, of course, you should configure the client for NFS client operations
220via /etc/rc.conf.
221In particular, this will turn on nfsiod which will improve client-side NFS
222performance:
223.Bd -literal -offset 4n
224nfs_client_enable="YES"
225.Ed
226.Pp
227Each client should create softlinks for /usr/ports and /usr/src that point
228into the NFS-mounted environment.
229If a particular client is running -CURRENT, /usr/src
230should be a softlink to /FreeBSD/FreeBSD-current/src.
231If it is running -STABLE, /usr/src should be a softlink to
232/FreeBSD/FreeBSD-4.x/src. I do not usually create a /usr/src2 softlink on
233clients, that is used as a convenient shortcut when working on the source
234code on the master server only and could create massive confusion (of the
235human variety) on a client.
236.Bd -literal -offset 4n
237(ON EACH CLIENT)
238cd /usr
239rm -rf ports src
240ln -s /FreeBSD/FreeBSD-current/ports ports
241ln -s /FreeBSD/FreeBSD-XXX/src src
242.Ed
243.Pp
244Don't forget to create the working directories so you can build ports, as
245previously described.
246If these are not good locations, make them softlinks to the correct location.
247Remember that /usr/ports/distfiles is exported by
248the master server and is therefore going to point to the same place
249(typically /usr/ports.distfiles) on every machine.
250.Bd -literal -offset 4n
251mkdir /usr/ports.distfiles
252mkdir /usr/ports.workdir
253.Ed
254.Sh BUILDING KERNELS
255Here is how you build a -STABLE kernel (on your main development box).
256If you want to create a custom kernel, cp GENERIC to YOURKERNEL and then
257edit it before configuring and building.
258The kernel configuration file lives in /usr/src/sys/i386/conf/KERNELNAME.
259.Bd -literal -offset 4n
260cd /usr/src
261make buildkernel KERNCONF=KERNELNAME
262.Ed
263.Pp
264.Sy WARNING!
265If you are familiar with the old config/cd/make method of building
266a -STABLE kernel, note that the config method will put the build
267environment in /usr/src/sys/compile/KERNELNAME instead of in /usr/obj.
268.Pp
269Building a -CURRENT kernel
270.Bd -literal -offset 4n
271cd /usr/src2 (on the master server)
272make buildkernel KERNCONF=KERNELNAME
273.Ed
274.Sh INSTALLING KERNELS
275Installing a -STABLE kernel (typically done on a client.
276Only do this on your main development server if you want to install a new
277kernel for your main development server):
278.Bd -literal -offset 4n
279cd /usr/src
280make installkernel KERNCONF=KERNELNAME
281.Ed
282.Pp
283If you are using the older config/cd/make build mechanism for stable, you
284would install using:
285.Bd -literal -offset 4n
286cd /usr/src/sys/compile/KERNELNAME
287make install
288.Ed
289.Pp
290Installing a -CURRENT kernel (typically done only on a client)
291.Bd -literal -offset 4n
292(remember /usr/src is pointing to the client's specific environment)
293cd /usr/src
294make installkernel KERNCONF=KERNELNAME
295.Ed
296.Pp
297.Sh BUILDING THE WORLD
298This environment is designed such that you do all builds on the master server,
299and then install from each client.
300You can do builds on a client only if /usr/obj is local to that client.
301Building the world is easy:
302.Bd -literal -offset 4n
303cd /usr/src
304make buildworld
305.Ed
306.Pp
307If you are on the master server you are running in a -STABLE environment, but
308that does not prevent you from building the -CURRENT world.
309Just cd into the appropriate source directory and you are set. Do not
310accidentally install it on your master server though!
311.Bd -literal -offset 4n
312cd /usr/src2
313make buildworld
314.Ed
315.Sh INSTALLING THE WORLD
316You can build on your main development server and install on clients.
317The main development server must export /FreeBSD and /usr/obj via
318read-only NFS to the clients.
319.Pp
320.Em NOTE!!!
321If /usr/obj is a softlink on the master server, it must also be the EXACT
322SAME softlink on each client.
323If /usr/obj is a directory in /usr or a mount point on the master server,
324then it must be (interchangeably) a directory in /usr or a mount point on
325each client.
326This is because the
327absolute paths are expected to be the same when building the world as when
328installing it, and you generally build it on your main development box
329and install it from a client.
330If you do not setup /usr/obj properly you will not be able to build on
331machine and install on another.
332.Bd -literal -offset 4n
333(ON THE CLIENT)
334(remember /usr/src is pointing to the client's specific environment)
335cd /usr/src
336make installworld
337.Ed
338.Pp
339.Sy WARNING!
340If builds work on the master server but installs do not work from the
341clients, for example you try to install and the client complains that
342the install tried to write into the read-only /usr/obj, then it is likely
343that the /etc/make.conf file on the client does not match the one on the
344master server closely enough and the install is trying to install something
345that was not built.
346.Sh DOING DEVELOPMENT ON A CLIENT (NOT JUST INSTALLING)
347Developers often want to run buildkernel's or buildworld's on client
348boxes simply to life-test the box.
349You do this in the same manner that you buildkernel and buildworld on your
350master server.
351All you have to do is make sure that /usr/obj is pointing to local storage.
352If you followed my advise and made /usr/obj its own partition on the master
353server,
354then it is typically going to be an NFS mount on the client.
355Simply unmounting /usr/obj will leave you with a /usr/obj that is a
356subdirectory in /usr which is typically local to the client.
357You can then do builds to your heart's content!
358.Sh MULTIPLE VERSIONS OF THE SOURCE TREE
359I have described how to maintain two versions of the source tree, a stable
360version in /FreeBSD/FreeBSD-4.x and a current version
361in /FreeBSD/FreeBSD-current.
362There is absolutely nothing preventing you
363from breaking out other versions of the source tree
364into /FreeBSD/XXX.
365In fact, my /FreeBSD partition also contains
366.Ox ,
367.Nx ,
368and various flavors of Linux.
369You may not necessarily be able to build non-FreeBSD operating systems on
370your master server, but being able
371to collect and manage source distributions from a central server is a very
372useful thing to be able to do and you can certainly export to machines
373which can build those other operating systems.
374.Sh UPDATING VIA CVS
375The advantage of using cvsup to maintain an updated copy of the CVS
376repository instead of using it to maintain source trees directly is that you
377can then pick and choose when you bring your source tree (or pieces of your
378source tree) up to date.
379By using a cron job to maintain an updated CVS repository, you can update
380your source tree at any time without any network cost as follows:
381.Bd -literal -offset 4n
382(on the main development server)
383cd /usr/src
384cvs -d /home/ncvs update
385cd /usr/src2
386cvs -d /home/ncvs update
387cd /usr/ports
388cvs -d /home/ncvs update
389.Ed
390.Pp
391It is that simple, and since you are exporting the whole lot to your
392clients, your clients have immediately visibility into the updated
393source.
394This is a good time to also remind you that most of the cvs operations
395you do will be done as root, and that certain options are
396required for CVS to operate properly on the
397.Fx
398repository. For example,
399.Fl Pd
400is necessary when running "cvs update".
401These options are typically placed in your ~/.cvsrc (as already described)
402so you do not have to respecify them every time you run a CVS command.
403Maintaining the CVS repository also gives you far more flexibility
404in regards to breaking out multiple versions of the source tree.
405It is a good idea to give your /FreeBSD partition a lot of space (I recommend
4068-12GB) precisely for that reason.
407If you can make it 15GB I would do it.
408.Pp
409I generally do not cvs update via a cron job.
410This is because I generally want the source to not change out from under me
411when I am developing code.
412Instead I manually update the source every so often... when I feel it is
413a good time.
414My recommendation is to only keep the cvs repository synchronized via cron.
415.Sh SEE ALSO
416.Xr crontab 1 ,
417.Xr crontab 5 ,
418.Xr build 7 ,
419.Xr firewall 7 ,
420.Xr release 7 ,
421.Xr tuning 7 ,
422.Xr diskless 8
423.Sh HISTORY
424The
425.Nm
426manual page was originally written by
427.An Matthew Dillon Aq dillon@FreeBSD.org
428and first appeared
429in
430.Fx 5.0 ,
431December 2002.