1 .\" Copyright (c) 2003-2009 Tim Kientzle
2 .\" All rights reserved.
4 .\" Redistribution and use in source and binary forms, with or without
5 .\" modification, are permitted provided that the following conditions
7 .\" 1. Redistributions of source code must retain the above copyright
8 .\" notice, this list of conditions and the following disclaimer.
9 .\" 2. Redistributions in binary form must reproduce the above copyright
10 .\" notice, this list of conditions and the following disclaimer in the
11 .\" documentation and/or other materials provided with the distribution.
13 .\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
14 .\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
15 .\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
16 .\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
17 .\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
18 .\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
19 .\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
20 .\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
21 .\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
22 .\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
25 .\" $FreeBSD: head/lib/libarchive/libarchive-formats.5 201077 2009-12-28 01:50:23Z kientzle $
28 .Dt LIBARCHIVE-FORMATS 5
31 .Nm libarchive-formats
32 .Nd archive formats supported by the libarchive library
36 library reads and writes a variety of streaming archive formats.
37 Generally speaking, all of these archive formats consist of a series of
39 Each entry stores a single file system object, such as a file, directory,
42 The following provides a brief description of each format supported
43 by libarchive, with some information about recognized extensions or
44 limitations of the current library support.
45 Note that just because a format is supported by libarchive does not
46 imply that a program that uses libarchive will support that format.
47 Applications that use libarchive specify which formats they wish
48 to support, though many programs do use libarchive convenience
49 functions to enable all supported formats.
53 library can read most tar archives.
54 However, it only writes POSIX-standard
60 All tar formats store each entry in one or more 512-byte records.
61 The first record is used for file metadata, including filename,
62 timestamp, and mode information, and the file data is stored in
64 Later variants have extended this by either appropriating undefined
65 areas of the header record, extending the header to multiple records,
66 or by storing special entries that modify the interpretation of
69 .Bl -tag -width indent
73 library can read GNU-format tar archives.
74 It currently supports the most popular GNU extensions, including
75 modern long filename and linkname support, as well as atime and ctime data.
76 The libarchive library does not support multi-volume
77 archives, nor the old GNU long filename format.
78 It can read GNU sparse file entries, including the new POSIX-based
79 formats, but cannot write GNU sparse file entries.
83 library can read and write POSIX-compliant pax interchange format
85 Pax interchange format archives are an extension of the older ustar
86 format that adds a separate entry with additional attributes stored
87 as key/value pairs immediately before each regular entry.
88 The presence of these additional entries is the only difference between
89 pax interchange format and the older ustar format.
90 The extended attributes are of unlimited length and are stored
91 as UTF-8 Unicode strings.
92 Keywords defined in the standard are in all lowercase; vendors are allowed
93 to define custom keys by preceding them with the vendor name in all uppercase.
94 When writing pax archives, libarchive uses many of the SCHILY keys
95 defined by Joerg Schilling's
97 archiver and a few LIBARCHIVE keys.
98 The libarchive library can read most of the SCHILY keys
99 and most of the GNU keys introduced by GNU tar.
100 It silently ignores any keywords that it does not understand.
101 .It Cm restricted pax
102 The libarchive library can also write pax archives in which it
103 attempts to suppress the extended attributes entry whenever
105 The result will be identical to a ustar archive unless the
106 extended attributes entry is required to store a long file
107 name, long linkname, extended ACL, file flags, or if any of the standard
108 ustar data (user name, group name, UID, GID, etc) cannot be fully
109 represented in the ustar header.
110 In all cases, the result can be dearchived by any program that
111 can read POSIX-compliant pax interchange format archives.
112 Programs that correctly read ustar format (see below) will also be
113 able to read this format; any extended attributes will be extracted as
114 separate files stored in
118 The libarchive library can both read and write this format.
119 This format has the following limitations:
122 Device major and minor numbers are limited to 21 bits.
123 Nodes with larger numbers will not be added to the archive.
125 Path names in the archive are limited to 255 bytes.
126 (Shorter if there is no / character in exactly the right place.)
128 Symbolic links and hard links are stored in the archive with
129 the name of the referenced file.
130 This name is limited to 100 bytes.
132 Extended attributes, file flags, and other extended
133 security information cannot be stored.
135 Archive entries are limited to 8 gigabytes in size.
137 Note that the pax interchange format has none of these restrictions.
140 The libarchive library also reads a variety of commonly-used extensions to
141 the basic tar format.
142 These extensions are recognized automatically whenever they appear.
143 .Bl -tag -width indent
144 .It Numeric extensions.
145 The POSIX standards require fixed-length numeric fields to be written with
146 some character position reserved for terminators.
147 Libarchive allows these fields to be written without terminator characters.
148 This extends the allowable range; in particular, ustar archives with this
149 extension can support entries up to 64 gigabytes in size.
150 Libarchive also recognizes base-256 values in most numeric fields.
151 This essentially removes all limitations on file size, modification time,
153 .It Solaris extensions
154 Libarchive recognizes ACL and extended attribute records written
156 Currently, libarchive only has support for old-style ACLs; the
157 newer NFSv4 ACLs are recognized but discarded.
160 The first tar program appeared in Seventh Edition Unix in 1979.
161 The first official standard for the tar file format was the
163 (Unix Standard Tar) format defined by POSIX in 1988.
164 POSIX.1-2001 extended the ustar format to create the
168 The libarchive library can read a number of common cpio variants and can write
173 A cpio archive stores each entry as a fixed-size header followed
174 by a variable-length filename and variable-length data.
175 Unlike the tar format, the cpio format does only minimal padding
176 of the header or file data.
177 There are several cpio variants, which differ primarily in
178 how they store the initial header: some store the values as
179 octal or hexadecimal numbers in ASCII, others as binary values of
180 varying byte order and length.
181 .Bl -tag -width indent
183 The libarchive library transparently reads both big-endian and little-endian
184 variants of the original binary cpio format.
185 This format used 32-bit binary values for file size and mtime,
186 and 16-bit binary values for the other fields.
188 The libarchive library can both read and write this
189 POSIX-standard format, which is officially known as the
190 .Dq cpio interchange format
192 .Dq octet-oriented cpio archive format
193 and sometimes unofficially referred to as the
194 .Dq old character format .
195 This format stores the header contents as octal values in ASCII.
196 It is standard, portable, and immune from byte-order confusion.
197 File sizes and mtime are limited to 33 bits (8GB file size),
198 other fields are limited to 18 bits.
200 The libarchive library can read both CRC and non-CRC variants of
202 The SVR4 format uses eight-digit hexadecimal values for
204 This limits file size to 4GB, and also limits the mtime and
205 other fields to 32 bits.
206 The SVR4 format can optionally include a CRC of the file
207 contents, although libarchive does not currently verify this CRC.
210 Cpio first appeared in PWB/UNIX 1.0, which was released within
212 PWB/UNIX 1.0 formed the basis of System III Unix, released outside
214 This makes cpio older than tar, although cpio was not included
215 in Version 7 AT&T Unix.
216 As a result, the tar command became much better known in universities
217 and research groups that used Version 7.
218 The combination of the
222 utilities provided very precise control over file selection.
223 Unfortunately, the format has many limitations that make it unsuitable
225 Only the POSIX format permits files over 4GB, and its 18-bit
226 limit for most other fields makes it unsuitable for modern systems.
227 In addition, cpio formats only store numeric UID/GID values (not
228 usernames and group names), which can make it very difficult to correctly
229 transfer archives across systems with dissimilar user numbering.
233 is a shell script that, when executed on a POSIX-compliant
234 system, will recreate a collection of file system objects.
235 The libarchive library can write two different kinds of shar archives:
236 .Bl -tag -width indent
238 The traditional shar format uses a limited set of POSIX
244 It is suitable for portably archiving small collections of plain text files.
245 However, it is not generally well-suited for large archives
246 (many implementations of
248 have limits on the size of a script) nor should it be used with non-text files.
250 This format is similar to shar but encodes files using
252 so that the result will be a plain text file regardless of the file contents.
253 It also includes additional shell commands that attempt to reproduce as
254 many file attributes as possible, including owner, mode, and flags.
255 The additional commands used to restore file attributes make
256 shardump archives less portable than plain shar archives.
259 Libarchive can read and extract from files containing ISO9660-compliant
261 In many cases, this can remove the need to burn a physical CDROM
262 just in order to read the files contained in an ISO9660 image.
263 It also avoids security and complexity issues that come with
264 virtual mounts and loopback devices.
265 Libarchive supports the most common Rockridge extensions and has partial
266 support for Joliet extensions.
267 If both extensions are present, the Joliet extensions will be
268 used and the Rockridge extensions will be ignored.
269 In particular, this can create problems with hardlinks and symlinks,
270 which are supported by Rockridge but not by Joliet.
272 Libarchive can read and write zip format archives that have
273 uncompressed entries and entries compressed with the
276 Older zip compression algorithms are not supported.
277 It can extract jar archives, archives that use Zip64 extensions and many
278 self-extracting zip archives.
279 Libarchive reads Zip archives as they are being streamed,
280 which allows it to read archives of arbitrary size.
281 It currently does not use the central directory; this
282 limits libarchive's ability to support some self-extracting
283 archives and ones that have been modified in certain ways.
284 .Ss Archive (library) file format
285 The Unix archive format (commonly created by the
287 archiver) is a general-purpose format which is
288 used almost exclusively for object files to be
289 read by the link editor
291 The ar format has never been standardised.
292 There are two common variants:
293 the GNU format derived from SVR4,
294 and the BSD format, which first appeared in 4.4BSD.
295 The two differ primarily in their handling of filenames
296 longer than 15 characters:
297 the GNU/SVR4 variant writes a filename table at the beginning of the archive;
298 the BSD format stores each long filename in an extension
299 area adjacent to the entry.
300 Libarchive can read both extensions,
301 including archives that may include both types of long filenames.
302 Programs using libarchive can write GNU/SVR4 format
303 if they provide a filename table to be written into
304 the archive before any of the entries.
305 Any entries whose names are not in the filename table
306 will be written using BSD-style long filenames.
307 This can cause problems for programs such as
308 GNU ld that do not support the BSD-style long filenames.
310 Libarchive can read and write files in
313 This format is not a true archive format, but rather a textual description
314 of a file hierarchy in which each line specifies the name of a file and
315 provides specific metadata about that file.
316 Libarchive can read all of the keywords supported by both
317 the NetBSD and FreeBSD versions of
319 although many of the keywords cannot currently be stored in an
322 When writing, libarchive supports use of the
323 .Xr archive_write_set_options 3
324 interface to specify which keywords should be included in the
326 If libarchive was compiled with access to suitable
327 cryptographic libraries (such as the OpenSSL libraries),
328 it can compute hash entries such as
332 from file data being written to the mtree writer.
334 When reading an mtree file, libarchive will locate the corresponding
335 files on disk using the
337 keyword if present or the regular filename.
338 If it can locate and open the file on disk, it will use that
339 to fill in any metadata that is missing from the mtree file
340 and will read the file contents and return those to the program
342 If it cannot locate and open the file on disk, libarchive
343 will return an error for any attempt to read the entry
346 libarchive has limited support to read files in RAR format. Currently,
347 libarchive can read single RAR files in RARv3 format which have been either
348 created uncompressed, or compressed using any of the compression methods
349 supported by the RARv3 format. libarchive can also extract RAR files which have
350 been created as self-extracting RAR files.
nant's projects / dragonfly.git / blob