1 .\" Copyright (c) 2007 Tim Kientzle
2 .\" All rights reserved.
4 .\" Redistribution and use in source and binary forms, with or without
5 .\" modification, are permitted provided that the following conditions
7 .\" 1. Redistributions of source code must retain the above copyright
8 .\" notice, this list of conditions and the following disclaimer.
9 .\" 2. Redistributions in binary form must reproduce the above copyright
10 .\" notice, this list of conditions and the following disclaimer in the
11 .\" documentation and/or other materials provided with the distribution.
13 .\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
14 .\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
15 .\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
16 .\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
17 .\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
18 .\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
19 .\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
20 .\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
21 .\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
22 .\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
32 .Nd format of cpio archive files
36 archive format collects any number of files, directories, and other
37 file system objects (symbolic links, device nodes, etc.) into a single
40 Each file system object in a
42 archive comprises a header record with basic numeric metadata
43 followed by the full pathname of the entry and the file data.
44 The header record stores a series of integer values that generally
50 The variants differ primarily in how they store those integers
51 (binary, octal, or hexadecimal).
52 The header is followed by the pathname of the
53 entry (the length of the pathname is stored in the header)
55 The end of the archive is indicated by a special record with
61 format is the original format, when cpio was introduced as part of the
62 Programmer's Work Bench system, a variant of 6th Edition UNIX. It
63 stores numbers as 2-byte and 4-byte binary values.
64 Each entry begins with a header in the following format:
66 .Bd -literal -offset indent
67 struct header_pwb_cpio {
84 fields here are 16-bit integer values, while the
86 fields are 32 bit integers. Since PWB UNIX, like the 6th Edition UNIX
87 it was based on, only ran on PDP-11 computers, they
88 are in PDP-endian format, which has little-endian shorts, and
89 big-endian longs. That is, the long integer whose hexadecimal
90 representation is 0x12345678 would be stored in four successive bytes
91 as 0x34, 0x12, 0x78, 0x56.
92 The fields are as follows:
93 .Bl -tag -width indent
95 The integer value octal 070707.
96 .It Va h_dev , Va h_ino
97 The device and inode numbers from the disk.
98 These are used by programs that read
100 archives to determine when two entries refer to the same file.
101 Programs that synthesize
103 archives should be careful to set these to distinct values for each entry.
105 The mode specifies both the regular permissions and the file type, and
106 it also holds a couple of bits that are irrelevant to the cpio format,
107 because the field is actually a raw copy of the mode field in the inode
108 representing the file. These are the IALLOC flag, which shows that
109 the inode entry is in use, and the ILARG flag, which shows that the
110 file it represents is large enough to have indirect blocks pointers in
112 The mode is decoded as follows:
114 .Bl -tag -width "MMMMMMM" -compact
116 IALLOC flag - irrelevant to cpio.
118 This masks the file type bits.
120 File type value for directories.
122 File type value for character special devices.
124 File type value for block special devices.
126 ILARG flag - irrelevant to cpio.
134 The lower 9 bits specify read/write/execute permissions
135 for world, group, and user following standard POSIX conventions.
137 .It Va h_uid , Va h_gid
138 The numeric user id and group id of the owner.
140 The number of links to this file.
141 Directories always have a value of at least two here.
142 Note that hardlinked files include file data with every copy in the archive.
144 For block special and character special entries,
145 this field contains the associated device number, with the major
146 number in the high byte, and the minor number in the low byte.
147 For all other entry types, it should be set to zero by writers
148 and ignored by readers.
150 Modification time of the file, indicated as the number
151 of seconds since the start of the epoch,
152 00:00:00 UTC January 1, 1970.
154 The number of bytes in the pathname that follows the header.
155 This count includes the trailing NUL byte.
157 The size of the file. Note that this archive format is limited to 16
158 megabyte file sizes, because PWB UNIX, like 6th Edition, only used
159 an unsigned 24 bit integer for the file size internally.
162 The pathname immediately follows the fixed header.
165 is odd, an additional NUL byte is added after the pathname.
166 The file data is then appended, again with an additional NUL
167 appended if needed to get the next header at an even offset.
169 Hardlinked files are not given special treatment;
170 the full file contents are included with each copy of the
172 .Ss New Binary Format
175 format showed up when cpio was adopted into late 7th Edition UNIX.
176 It is exactly like the PWB binary format, described above, except for
179 First, UNIX now ran on more than one hardware type, so the endianness
180 of 16 bit integers must be determined by observing the magic number at
181 the start of the header. The 32 bit integers are still always stored
182 with the most significant word first, though, so each of those two, in
183 the struct shown above, was stored as an array of two 16 bit integers,
184 in the traditional order. Those 16 bit integers, like all the others
185 in the struct, were accessed using a macro that byte swapped them if
188 Next, 7th Edition had more file types to store, and the IALLOC and ILARG
189 flag bits were re-purposed to accommodate these. The revised use of the
190 various bits is as follows:
192 .Bl -tag -width "MMMMMMM" -compact
194 This masks the file type bits.
196 File type value for sockets.
198 File type value for symbolic links.
199 For symbolic links, the link body is stored as file data.
201 File type value for regular files.
203 File type value for block special devices.
205 File type value for directories.
207 File type value for character special devices.
209 File type value for named pipes or FIFOs.
217 The lower 9 bits specify read/write/execute permissions
218 for world, group, and user following standard POSIX conventions.
221 Finally, the file size field now represents a signed 32 bit integer in
222 the underlying file system, so the maximum file size has increased to
225 Note that there is no obvious way to tell which of the two binary
226 formats an archive uses, other than to see which one makes more
227 sense. The typical error scenario is that a PWB format archive
228 unpacked as if it were in the new format will create named sockets
229 instead of directories, and then fail to unpack files that should
230 go in those directories. Running
232 on an unknown archive will make it obvious which it is: if it's
233 PWB format, directories will be listed with an 's' instead of
234 a 'd' as the first character of the mode string, and the larger
235 files will have a '?' in that position.
236 .Ss Portable ASCII Format
238 standardized an ASCII variant that is portable across all
240 It is commonly known as the
245 It stores the same numeric fields as the old binary format, but
246 represents them as 6-character or 11-character octal values.
248 .Bd -literal -offset indent
249 struct cpio_odc_header {
264 The fields are identical to those in the new binary format.
265 The name and file body follow the fixed header.
266 Unlike the binary formats, there is no additional padding
267 after the pathname or file contents.
268 If the files being archived are themselves entirely ASCII, then
269 the resulting archive will be entirely ASCII, except for the
270 NUL byte that terminates the name field.
272 The "new" ASCII format uses 8-byte hexadecimal fields for
273 all numbers and separates device numbers into separate fields
274 for major and minor numbers.
276 .Bd -literal -offset indent
277 struct cpio_newc_header {
295 Except as specified below, the fields here match those specified
296 for the new binary format above.
297 .Bl -tag -width indent
302 This field is always set to zero by writers and ignored by readers.
303 See the next section for more details.
306 The pathname is followed by NUL bytes so that the total size
307 of the fixed header plus pathname is a multiple of four.
308 Likewise, the file data is padded to a multiple of four bytes.
309 Note that this format supports only 4 gigabyte files (unlike the
310 older ASCII format, which supports 8 gigabyte files).
312 In this format, hardlinked files are handled by setting the
313 filesize to zero for each entry except the first one that
314 appears in the archive.
316 The CRC format is identical to the new ASCII format described
317 in the previous section except that the magic field is set
322 field is set to the sum of all bytes in the file data.
323 This sum is computed treating all bytes as unsigned values
324 and using unsigned arithmetic.
325 Only the least-significant 32 bits of the sum are stored.
329 implementation distributed with HPUX used XXXX but stored
330 device numbers differently XXX.
331 .Ss Other Extensions and Variants
332 Sun Solaris uses additional file types to store extended file
333 data, including ACLs and extended attributes, as special
334 entries in cpio archives.
343 utility is no longer a part of POSIX or the Single Unix Standard.
346 It has been supplanted in subsequent standards by
348 The portable ASCII format is currently part of the specification for the
352 The original cpio utility was written by Dick Haight
353 while working in AT&T's Unix Support Group.
354 It appeared in 1977 as part of PWB/UNIX 1.0, the
355 .Dq Programmer's Work Bench
358 that was used internally at AT&T.
359 Both the new binary and old character formats were in use
360 by 1980, according to the System III source released
364 The character format was adopted as part of
366 XXX when did "newc" appear? Who invented it? When did HP come out with their variant? When did Sun introduce ACLs and extended attributes? XXX
370 format is mis-named, as it uses a simple checksum and
371 not a cyclic redundancy check.
373 The binary formats are limited to 16 bits for user id, group id,
374 device, and inode numbers. They are limited to 16 megabyte and 2
375 gigabyte file sizes for the older and newer variants, respectively.
377 The old ASCII format is limited to 18 bits for
378 the user id, group id, device, and inode numbers.
379 It is limited to 8 gigabyte file sizes.
381 The new ASCII format is limited to 4 gigabyte file sizes.
383 None of the cpio formats store user or group names,
384 which are essential when moving files between systems with
385 dissimilar user or group numbering.
387 Especially when writing older cpio variants, it may be necessary
388 to map actual device/inode values to synthesized values that
389 fit the available fields.
390 With very large filesystems, this may be necessary even for