1 .\" Copyright (c) 2003-2007 Tim Kientzle
2 .\" All rights reserved.
4 .\" Redistribution and use in source and binary forms, with or without
5 .\" modification, are permitted provided that the following conditions
7 .\" 1. Redistributions of source code must retain the above copyright
8 .\" notice, this list of conditions and the following disclaimer.
9 .\" 2. Redistributions in binary form must reproduce the above copyright
10 .\" notice, this list of conditions and the following disclaimer in the
11 .\" documentation and/or other materials provided with the distribution.
13 .\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
14 .\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
15 .\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
16 .\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
17 .\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
18 .\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
19 .\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
20 .\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
21 .\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
22 .\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
25 .\" $FreeBSD: src/lib/libarchive/libarchive-formats.5,v 1.16 2008/05/26 17:00:23 kientzle Exp $
28 .Dt libarchive-formats 3
31 .Nm libarchive-formats
32 .Nd archive formats supported by the libarchive library
36 library reads and writes a variety of streaming archive formats.
37 Generally speaking, all of these archive formats consist of a series of
39 Each entry stores a single file system object, such as a file, directory,
42 The following provides a brief description of each format supported
43 by libarchive, with some information about recognized extensions or
44 limitations of the current library support.
45 Note that just because a format is supported by libarchive does not
46 imply that a program that uses libarchive will support that format.
47 Applications that use libarchive specify which formats they wish
52 library can read most tar archives.
53 However, it only writes POSIX-standard
59 All tar formats store each entry in one or more 512-byte records.
60 The first record is used for file metadata, including filename,
61 timestamp, and mode information, and the file data is stored in
63 Later variants have extended this by either appropriating undefined
64 areas of the header record, extending the header to multiple records,
65 or by storing special entries that modify the interpretation of
68 .Bl -tag -width indent
72 library can read GNU-format tar archives.
73 It currently supports the most popular GNU extensions, including
74 modern long filename and linkname support, as well as atime and ctime data.
75 The libarchive library does not support multi-volume
76 archives, nor the old GNU long filename format.
77 It can read GNU sparse file entries, including the new POSIX-based
78 formats, but cannot write GNU sparse file entries.
82 library can read and write POSIX-compliant pax interchange format
84 Pax interchange format archives are an extension of the older ustar
85 format that adds a separate entry with additional attributes stored
87 The presence of this additional entry is the only difference between
88 pax interchange format and the older ustar format.
89 The extended attributes are of unlimited length and are stored
90 as UTF-8 Unicode strings.
91 Keywords defined in the standard are in all lowercase; vendors are allowed
92 to define custom keys by preceding them with the vendor name in all uppercase.
93 When writing pax archives, libarchive uses many of the SCHILY keys
94 defined by Joerg Schilling's
97 The libarchive library can read most of the SCHILY keys.
98 It silently ignores any keywords that it does not understand.
100 The libarchive library can also write pax archives in which it
101 attempts to suppress the extended attributes entry whenever
103 The result will be identical to a ustar archive unless the
104 extended attributes entry is required to store a long file
105 name, long linkname, extended ACL, file flags, or if any of the standard
106 ustar data (user name, group name, UID, GID, etc) cannot be fully
107 represented in the ustar header.
108 In all cases, the result can be dearchived by any program that
109 can read POSIX-compliant pax interchange format archives.
110 Programs that correctly read ustar format (see below) will also be
111 able to read this format; any extended attributes will be extracted as
112 separate files stored in
116 The libarchive library can both read and write this format.
117 This format has the following limitations:
120 Device major and minor numbers are limited to 21 bits.
121 Nodes with larger numbers will not be added to the archive.
123 Path names in the archive are limited to 255 bytes.
124 (Shorter if there is no / character in exactly the right place.)
126 Symbolic links and hard links are stored in the archive with
127 the name of the referenced file.
128 This name is limited to 100 bytes.
130 Extended attributes, file flags, and other extended
131 security information cannot be stored.
133 Archive entries are limited to 2 gigabytes in size.
135 Note that the pax interchange format has none of these restrictions.
138 The libarchive library can also read a variety of commonly-used extensions to
139 the basic tar format.
140 In particular, it supports base-256 values in certain numeric fields.
141 This essentially removes the limitations on file size, modification time,
144 The first tar program appeared in Seventh Edition Unix in 1979.
145 The first official standard for the tar file format was the
147 (Unix Standard Tar) format defined by POSIX in 1988.
148 POSIX.1-2001 extended the ustar format to create the
152 The libarchive library can read a number of common cpio variants and can write
157 A cpio archive stores each entry as a fixed-size header followed
158 by a variable-length filename and variable-length data.
159 Unlike tar, cpio does only minimal padding of the header or file data.
160 There are a variety of cpio formats, which differ primarily in
161 how they store the initial header: some store the values as
162 octal or hexadecimal numbers in ASCII, others as binary values of
163 varying byte order and length.
164 .Bl -tag -width indent
166 The libarchive library can read both big-endian and little-endian
167 variants of the original binary cpio format.
168 This format used 32-bit binary values for file size and mtime,
169 and 16-bit binary values for the other fields.
171 The libarchive library can both read and write this
172 POSIX-standard format.
173 This format stores the header contents as octal values in ASCII.
174 It is standard, portable, and immune from byte-order confusion.
175 File sizes and mtime are limited to 33 bits (8GB file size),
176 other fields are limited to 18 bits.
178 The libarchive library can read both CRC and non-CRC variants of
180 The SVR4 format uses eight-digit hexadecimal values for
182 This limits file size to 4GB, and also limits the mtime and
183 other fields to 32 bits.
184 The SVR4 format can optionally include a CRC of the file
185 contents, although libarchive does not currently verify this CRC.
188 Cpio first appeared in PWB/UNIX 1.0, which was released within
190 PWB/UNIX 1.0 formed the basis of System III Unix, released outside
192 This makes cpio older than tar, although cpio was not included
193 in Version 7 AT&T Unix.
194 As a result, the tar command became much better known in universities
195 and research groups that used Version 7.
196 The combination of the
200 utilities provided very precise control over file selection.
201 Unfortunately, the format has many limitations that make it unsuitable
203 Only the POSIX format permits files over 4GB, and its 18-bit
204 limit for most other fields makes it unsuitable for modern systems.
205 In addition, cpio formats only store numeric UID/GID values (not
206 usernames and group names), which can make it very difficult to correctly
207 transfer archives across systems with dissimilar user numbering.
211 is a shell script that, when executed on a POSIX-compliant
212 system, will recreate a collection of file system objects.
213 The libarchive library can write two different kinds of shar archives:
214 .Bl -tag -width indent
216 The traditional shar format uses a limited set of POSIX
222 It is suitable for portably archiving small collections of plain text files.
223 However, it is not generally well-suited for large archives
224 (many implementations of
226 have limits on the size of a script) nor should it be used with non-text files.
228 This format is similar to shar but encodes files using
230 so that the result will be a plain text file regardless of the file contents.
231 It also includes additional shell commands that attempt to reproduce as
232 many file attributes as possible, including owner, mode, and flags.
233 The additional commands used to restore file attributes make
234 shardump archives less portable than plain shar archives.
237 Libarchive can read and extract from files containing ISO9660-compliant
239 It also has partial support for Rockridge extensions.
240 In many cases, this can remove the need to burn a physical CDROM.
241 It also avoids security and complexity issues that come with
242 virtual mounts and loopback devices.
244 Libarchive can extract from most zip format archives.
245 It currently only supports uncompressed entries and entries
249 Older zip compression algorithms are not supported.
250 .Ss Archive (library) file format
251 The Unix archive format (commonly created by the
253 archiver) is a general-purpose format which is
254 used almost exclusively for object files to be
255 read by the link editor
257 The ar format has never been standardised.
258 There are two common variants:
259 the GNU format derived from SVR4,
260 and the BSD format, which first appeared in 4.4BSD.
261 Libarchive provides read and write support for both variants.
263 Libarchive can read files in
265 format. This format is not a true archive format, but rather a description
266 of a file hierarchy. When requested, libarchive obtains the contents of
267 the files described by the
269 format from files on disk instead.
projects / dragonfly.git / blob