2 .\" $FreeBSD: src/share/doc/papers/jail/paper.ms,v 1.4.2.3 2000/12/07 10:00:04 ru Exp $
3 .\" $DragonFly: src/share/doc/papers/jail/paper.ms,v 1.2 2003/06/17 04:36:56 dillon Exp $
15 Jails: Confining the omnipotent root.
17 This paper was presented at the 2nd International System Administration and Networking Conference "SANE 2000" May 22-25, 2000 in Maastricht, The Netherlands and is published in the proceedings.
20 Poul-Henning Kamp <phk@FreeBSD.org>
22 Robert N. M. Watson <rwatson@FreeBSD.org>
26 This work was sponsored by \fChttp://www.servetheweb.com/\fP and
27 donated to the FreeBSD Project for inclusion in the FreeBSD
28 OS. FreeBSD 4.0-RELEASE was the first release including this
30 Follow-on work was sponsored by Safeport Network Services,
31 \fChttp://www.safeport.com/\fP
34 The traditional UNIX security model is simple but inexpressive.
35 Adding fine-grained access control improves the expressiveness,
36 but often dramatically increases both the cost of system management
37 and implementation complexity.
38 In environments with a more complex management model, with delegation
39 of some management functions to parties under varying degrees of trust,
40 the base UNIX model and most natural
41 extensions are inappropriate at best.
42 Where multiple mutually un-trusting parties are introduced,
43 ``inappropriate'' rapidly transitions to ``nightmarish'', especially
44 with regards to data integrity and privacy protection.
46 The FreeBSD ``Jail'' facility provides the ability to partition
47 the operating system environment, while maintaining the simplicity
48 of the UNIX ``root'' model.
49 In Jail, users with privilege find that the scope of their requests
50 is limited to the jail, allowing system administrators to delegate
51 management capabilities for each virtual machine
53 Creating virtual machines in this manner has many potential uses; the
54 most popular thus far has been for providing virtual machine services
55 in Internet Service Provider environments.
60 The UNIX access control mechanism is designed for an environment with two
61 types of users: those with, and without administrative privilege.
62 Within this framework, every attempt is made to provide an open
63 system, allowing easy sharing of files and inter-process communication.
64 As a member of the UNIX family, FreeBSD inherits these
66 Users of FreeBSD in non-traditional UNIX environments must balance
67 their need for strong application support, high network performance
68 and functionality, and low total cost of ownership with the need
69 for alternative security models that are difficult or impossible to
70 implement with the UNIX security mechanisms.
72 One such consideration is the desire to delegate some (but not all)
73 administrative functions to untrusted or less trusted parties, and
74 simultaneously impose system-wide mandatory policies on process
75 interaction and sharing.
76 Attempting to create such an environment in the current-day FreeBSD
77 security environment is both difficult and costly: in many cases,
78 the burden of implementing these policies falls on user
79 applications, which means an increase in the size and complexity
80 of the code base, in turn translating to higher development
81 and maintenance cost, as well as less overall flexibility.
83 This abstract risk becomes more clear when applied to a practical,
85 many web service providers turn to the FreeBSD
86 operating system to host customer web sites, as it provides a
87 high-performance, network-centric server environment.
88 However, these providers have a number of concerns on their plate, both in
89 terms of protecting the integrity and confidentiality of their own
90 files and services from their customers, as well as protecting the files
91 and services of one customer from (accidental or
92 intentional) access by any other customer.
93 At the same time, a provider would like to provide
94 substantial autonomy to customers, allowing them to install and
95 maintain their own software, and to manage their own services,
96 such as web servers and other content-related daemon programs.
98 This problem space points strongly in the direction of a partitioning
99 solution, in which customer processes and storage are isolated from those of
100 other customers, both in terms of accidental disclosure of data or process
101 information, but also in terms of the ability to modify files or processes
102 outside of a compartment.
103 Delegation of management functions within the system must
104 be possible, but not at the cost of system-wide requirements, including
105 integrity and privacy protection between partitions.
107 However, UNIX-style access control makes it notoriously difficult to
108 compartmentalise functionality.
109 While mechanisms such as chroot(2) provide a modest
110 level compartmentalisation, it is well known
111 that these mechanisms have serious shortcomings, both in terms of the
112 scope of their functionality, and effectiveness at what they provide \s-2[CHROOT]\s+2.
114 In the case of the chroot(2) call, a process's visibility of
115 the file system name-space is limited to a single subtree.
116 However, the compartmentalisation does not extend to the process
117 or networking spaces and therefore both observation of and interference
118 with processes outside their compartment is possible.
120 To this end, we describe the new FreeBSD ``Jail'' facility, which
121 provides a strong partitioning solution, leveraging existing
122 mechanisms, such as chroot(2), to what effectively amounts to a
123 virtual machine environment. Processes in a jail are provided
124 full access to the files that they may manipulate, processes they
125 may influence, and network services they can make use of, and neither
126 access nor visibility of files, processes or network services outside
129 Unlike other fine-grained security solutions, Jail does not
130 substantially increase the policy management requirements for the
131 system administrator, as each Jail is a virtual FreeBSD environment
132 permitting local policy to be independently managed, with much the
133 same properties as the main system itself, making Jail easy to use
134 for the administrator, and far more compatible with applications.
136 Traditional UNIX Security, or, ``God, root, what difference?" \s-2[UF]\s+2.
138 The traditional UNIX access model assigns numeric uids to each user of the
139 system. In turn, each process ``owned'' by a user will be tagged with that
140 user's uid in an unforgeable manner. The uids serve two purposes: first,
141 they determine how discretionary access control mechanisms will be applied, and
142 second, they are used to determine whether special privileges are accorded.
144 In the case of discretionary access controls, the primary object protected is
145 a file. The uid (and related gids indicating group membership) are mapped to
146 a set of rights for each object, courtesy the UNIX file mode, in effect acting
147 as a limited form of access control list. Jail is, in general, not concerned
148 with modifying the semantics of discretionary access control mechanisms,
149 although there are important implications from a management perspective.
151 For the purposes of determining whether special privileges are accorded to a
152 process, the check is simple: ``is the numeric uid equal to 0 ?''.
154 process is acting with ``super-user privileges'', and all access checks are
155 granted, in effect allowing the process the ability to do whatever it wants
158 \&... no matter how patently stupid it may be.
161 For the purposes of human convenience, uid 0 is canonically allocated
162 to the ``root'' user \s-2[ROOT]\s+2.
163 For the purposes of jail, this behaviour is extremely relevant: many of
164 these privileged operations can be used to manage system hardware and
165 configuration, file system name-space, and special network operations.
167 Many limitations to this model are immediately clear: the root user is a
168 single, concentrated source of privilege that is exposed to many pieces of
169 software, and as such an immediate target for attacks. In the event of a
170 compromise of the root capability set, the attacker has complete control over
171 the system. Even without an attacker, the risks of a single administrative
172 account are serious: delegating a narrow scope of capability to an
173 inexperienced administrator is difficult, as the granularity of delegation is
174 that of all system management abilities. These features make the omnipotent
175 root account a sharp, efficient and extremely dangerous tool.
177 The BSD family of operating systems have implemented the ``securelevel''
178 mechanism which allows the administrator to block certain configuration
179 and management functions from being performed by root,
180 until the system is restarted and brought up into single-user mode.
181 While this does provide some amount of protection in the case of a root
182 compromise of the machine, it does nothing to address the need for
183 delegation of certain root abilities.
185 Other Solutions to the Root Problem
187 Many operating systems attempt to address these limitations by providing
188 fine-grained access controls for system resources \s-2[BIBA]\s+2.
189 These efforts vary in
190 degrees of success, but almost all suffer from at least three serious
193 First, increasing the granularity of security controls increases the
194 complexity of the administration process, in turn increasing both the
195 opportunity for incorrect configuration, as well as the demand on
196 administrator time and resources. In many cases, the increased complexity
197 results in significant frustration for the administrator, which may result
199 disastrous types of policy: ``all doors open as it's too much trouble'', and
200 ``trust that the system is secure, when in fact it isn't''.
202 The extent of the trouble is best illustrated by the fact that an entire
203 niche industry has emerged providing tools to manage fine grained security
204 controls \s-2[UAS]\s+2.
206 Second, usefully segregating capabilities and assigning them to running code
207 and users is very difficult. Many privileged operations in UNIX seem
208 independent, but are in fact closely related, and the handing out of one
209 privilege may, in effect, be transitive to the many others. For example, in
210 some trusted operating systems, a system capability may be assigned to a
211 running process to allow it to read any file, for the purposes of backup.
212 However, this capability is, in effect, equivalent to the ability to switch to
213 any other account, as the ability to access any file provides access to system
214 keying material, which in turn provides the ability to authenticate as any
215 user. Similarly, many operating systems attempt to segregate management
216 capabilities from auditing capabilities. In a number of these operating
217 systems, however, ``management capabilities'' permit the administrator to
218 assign ``auditing capabilities'' to itself, or another account, circumventing
219 the segregation of capability.
221 Finally, introducing new security features often involves introducing new
222 security management APIs. When fine-grained capabilities are introduced to
223 replace the setuid mechanism in UNIX-like operating systems, applications that
224 previously did an ``appropriateness check'' to see if they were running as
225 root before executing must now be changed to know that they need not run as
226 root. In the case of applications running with privilege and executing other
227 programs, there is now a new set of privileges that must be voluntarily given
228 up before executing another program. These change can introduce significant
229 incompatibility for existing applications, and make life more difficult for
230 application developers who may not be aware of differing security semantics on
231 different systems \s-2[POSIX1e]\s+2.
233 The Jail Partitioning Solution
235 Jail neatly side-steps the majority of these problems through partitioning.
237 than introduce additional fine-grained access control mechanism, we partition
238 a FreeBSD environment (processes, file system, network resources) into a
239 management environment, and optionally subset Jail environments. In doing so,
240 we simultaneously maintain the existing UNIX security model, allowing
241 multiple users and a privileged root user in each jail, while
242 limiting the scope of root's activities to his jail.
243 Consequently the administrator of a
244 FreeBSD machine can partition the machine into separate jails, and provide
245 access to the super-user account in each of these without losing control of
246 the over-all environment.
248 A process in a partition is referred to as ``in jail''. When a FreeBSD
249 system is booted up after a fresh install, no processes will be in jail.
251 a process is placed in a jail, it, and any descendents of the process created
252 after the jail creation, will be in that jail. A process may be in only one
253 jail, and after creation, it can not leave the jail.
254 Jails are created when a
255 privileged process calls the jail(2) syscall, with a description of the jail as an
256 argument to the call. Each call to jail(2) creates a new jail; the only way
257 for a new process to enter the jail is by inheriting access to the jail from
258 another process already in that jail.
260 leave the jail they created, or were created in.
262 .if t .PSPIC jail01.eps 4i
264 Fig. 1 \(em Schematic diagram of machine with two configured jails
268 Membership in a jail involves a number of restrictions: access to the file
269 name-space is restricted in the style of chroot(2), the ability to bind network
270 resources is limited to a specific IP address, the ability to manipulate
271 system resources and perform privileged operations is sharply curtailed, and
272 the ability to interact with other processes is limited to only processes
273 inside the same jail.
275 Jail takes advantage of the existing chroot(2) behaviour to limit access to the
276 file system name-space for jailed processes. When a jail is created, it is
277 bound to a particular file system root.
278 Processes are unable to manipulate files that they cannot address,
279 and as such the integrity and confidentiality of files outside of the jail
280 file system root are protected. Traditional mechanisms for breaking out of
281 chroot(2) have been blocked.
282 In the expected and documented configuration, each jail is provided
283 with its exclusive file system root, and standard FreeBSD directory layout,
284 but this is not mandated by the implementation.
286 Each jail is bound to a single IP address: processes within the jail may not
287 make use of any other IP address for outgoing or incoming connections; this
288 includes the ability to restrict what network services a particular jail may
289 offer. As FreeBSD distinguishes attempts to bind all IP addresses from
290 attempts to bind a particular address, bind requests for all IP addresses are
291 redirected to the individual Jail address. Some network functionality
292 associated with privileged calls are wholesale disabled due to the nature of the
293 functionality offered, in particular facilities which would allow ``spoofing''
294 of IP numbers or disruptive traffic to be generated have been disabled.
296 Processes running without root privileges will notice few, if any differences
297 between a jailed environment or un-jailed environment. Processes running with
298 root privileges will find that many restrictions apply to the privileged calls
299 they may make. Some calls will now return an access error \(em for example, an
300 attempt to create a device node will now fail. Others will have a more
301 limited scope than normal \(em attempts to bind a reserved port number on all
302 available addresses will result in binding only the address associated with
303 the jail. Other calls will succeed as normal: root may read a file owned by
304 any uid, as long as it is accessible through the jail file system name-space.
306 Processes within the jail will find that they are unable to interact or
307 even verify the existence of
308 processes outside the jail \(em processes within the jail are
309 prevented from delivering signals to processes outside the jail, as well as
310 connecting to those processes with debuggers, or even see them in the
311 sysctl or process file system monitoring mechanisms. Jail does not prevent,
312 nor is it intended to prevent, the use of covert channels or communications
313 mechanisms via accepted interfaces \(em for example, two processes may communicate
314 via sockets over the IP network interface. Nor does it attempt to provide
315 scheduling services based on the partition; however, it does prevent calls
316 that interfere with normal process operation.
318 As a result of these attempts to retain the standard FreeBSD API and
319 framework, almost all applications will run unaffected. Standard system
320 services such as Telnet, FTP, and SSH all behave normally, as do most third
321 party applications, including the popular Apache web server.
325 Processes running with root privileges in the jail find that there are serious
326 restrictions on what it is capable of doing \(em in particular, activities that
327 would extend outside of the jail:
329 \(bu Modifying the running kernel by direct access and loading kernel
330 modules is prohibited.
332 \(bu Modifying any of the network configuration, interfaces, addresses, and
333 routing table is prohibited.
335 \(bu Mounting and unmounting file systems is prohibited.
337 \(bu Creating device nodes is prohibited.
339 \(bu Accessing raw, divert, or routing sockets is prohibited.
341 \(bu Modifying kernel runtime parameters, such as most sysctl settings, is
344 \(bu Changing securelevel-related file flags is prohibited.
346 \(bu Accessing network resources not associated with the jail is prohibited.
348 Other privileged activities are permitted as long as they are limited to the
351 \(bu Signalling any process within the jail is permitted.
353 \(bu Changing the ownership and mode of any file within the jail is permitted, as
354 long as the file flags permit this.
356 \(bu Deleting any file within the jail is permitted, as long as the file flags
359 \(bu Binding reserved TCP and UDP port numbers on the jails IP address is
360 permitted. (Attempts to bind TCP and UDP ports using INADDR_ANY will be
361 redirected to the jails IP address.)
363 \(bu Functions which operate on the uid/gid space are all permitted since they
364 act as labels for filesystem objects of proceses
365 which are partitioned off by other mechanisms.
367 These restrictions on root access limit the scope of root processes, enabling
368 most applications to run un-hindered, but preventing calls that might allow an
369 application to reach beyond the jail and influence other processes or
370 system-wide configuration.
372 .so implementation.ms
378 The jail facility provides FreeBSD with a conceptually simple security
379 partitioning mechanism, allowing the delegation of administrative rights
380 within virtual machine partitions.
382 The implementation relies on
383 restricting access within the jail environment to a well-defined subset
384 of the overall host environment. This includes limiting interaction
385 between processes, and to files, network resources, and privileged
386 operations. Administrative overhead is reduced through avoiding
387 fine-grained access control mechanisms, and maintaining a consistent
388 administrative interface across partitions and the host environment.
390 The jail facility has already seen widespread deployment in particular as
391 a vehicle for delivering "virtual private server" services.
393 The jail code is included in the base system as part of FreeBSD 4.0-RELEASE,
394 and fully documented in the jail(2) and jail(8) man-pages.
398 .IP \s-2[BIBA]\s+2 .5i
399 K. J. Biba, Integrity Considerations for Secure
400 Computer Systems, USAF Electronic Systems Division, 1977
401 .IP \s-2[CHROOT]\s+2 .5i
402 Dr. Marshall Kirk Mckusick, private communication:
403 ``According to the SCCS logs, the chroot call was added by Bill Joy
404 on March 18, 1982 approximately 1.5 years before 4.2BSD was released.
405 That was well before we had ftp servers of any sort (ftp did not
406 show up in the source tree until January 1983). My best guess as
407 to its purpose was to allow Bill to chroot into the /4.2BSD build
408 directory and build a system using only the files, include files,
409 etc contained in that tree. That was the only use of chroot that
410 I remember from the early days.''
411 .IP \s-2[LOTTERY1]\s+2 .5i
412 David Petrou and John Milford. Proportional-Share Scheduling:
413 Implementation and Evaluation in a Widely-Deployed Operating System,
416 \s-2\fChttp://www.cs.cmu.edu/~dpetrou/papers/freebsd_lottery_writeup98.ps\fP\s+2
417 \s-2\fChttp://www.cs.cmu.edu/~dpetrou/code/freebsd_lottery_code.tar.gz\fP\s+2
418 .IP \s-2[LOTTERY2]\s+2 .5i
419 Carl A. Waldspurger and William E. Weihl. Lottery Scheduling: Flexible Proportional-Share Resource Management, Proceedings of the First Symposium on Operating Systems Design and Implementation (OSDI '94), pages 1-11, Monterey, California, November 1994.
421 \s-2\fChttp://www.research.digital.com/SRC/personal/caw/papers.html\fP\s+2
422 .IP \s-2[POSIX1e]\s+2 .5i
423 Draft Standard for Information Technology \(em
424 Portable Operating System Interface (POSIX) \(em
425 Part 1: System Application Program Interface (API) \(em Amendment:
426 Protection, Audit and Control Interfaces [C Language]
427 IEEE Std 1003.1e Draft 17 Editor Casey Schaufler
428 .IP \s-2[ROOT]\s+2 .5i
429 Historically other names have been used at times, Zilog for instance
430 called the super-user account ``zeus''.
431 .IP \s-2[UAS]\s+2 .5i
432 One such niche product is the ``UAS'' system to maintain and audit
433 RACF configurations on MVS systems.
435 \s-2\fChttp://www.entactinfo.com/products/uas/\fP\s+2
437 Quote from the User-Friendly cartoon by Illiad.
439 \s-2\fChttp://www.userfriendly.org/cartoons/archives/98nov/19981111.html\fP\s+2
projects / dragonfly.git / blob