src-test2/lib/libarchive, branch release/8.4.0

MFC r228797:

2012-01-05T12:57:14Z

Use contrib sources for building libarchive, tar and cpio. Make "make test" fully operational.

MFC r228744, r228745, r228748:

2012-01-05T11:45:22Z

MFC r228744 [1]: Merge vendor revision 3723: Fixes extraction of Zip entries that use length-at-end without specifying either the compressed or uncompressed length. In particular, fixes bsdtar extraction of such files. Reported by: Patrick Lamaiziere (freebsd-stable@) MFC r228745: Update libarchive, tar and cpio to version 2.8.5 The following additional vendor revisions are applied: Revision 3740: Use archive_clear_error() to clear the error markers. Obtained from: http://code.google.com/p/libarchive MFC r228748: Sync libarchive with vendor branch release/2.8: 3730: Fix issue 174 (Windows path names, not relevant for FreeBSD) 3734: Merge r1989: archive_clear_error should set errno to 0. 3735: Merge r3247 from trunk: Clear errors before returning from archive_read_support_format_all() 3799: Check the position before dereferencing the pointer. This avoids dereferencing one byte past the end of a string 3824: Merge r3823 from trunk for issue 199 (hang in iso9660 reading) Obtained from: http://code.google.com/p/libarchive

MFC r225167, r225329:

2011-09-05T09:49:28Z

MFC r225167: Import bugfix for reading and extracting of FreeBSD ISO images with tar. Vendor revision 3645 (merge of 3642): Change the mechanism handling a rr_moved directory, which is Rockridge extension that can exceed the limitation of a maximum directory depth of ISO 9660. - Stop reading all entries at a time. - Connect "CL" entry to "RE" entry dynamically, which "CL" and "RE" have information to rebuild a full directory tree. - Tweak some related tests since we use Headsort for re-ordering entries and it cannot make a steady order when the keies of the entries are the same. http://code.google.com/p/libarchive/issues/detail?id=168 MFC r225329: Import additional bugfix for reading and extracting makefs-created ISO images with tar. Vendor revision 3648 (merge of 3647): Additional fix to issue 168 because the change of r3642 was not sufficient. - Make sure "CL" entry appear after its "RE" entry which the "CL" entry should be connected with. - Give consideration to the case that the top level "RE" entry has already been exposed outside before its tree. Obtained from: libarchive (release/2.8, svn rev 3645, 3648)

MFC r224691, r224700 [1]:

2011-08-10T08:40:59Z

MFC r224961: Add compatibility for ISO images created with unfixed makefs that violated ECMA-119 (ISO9660): allow reserved4 to be 0x20 in PVD. This allows tar to read FreeBSD distribution ISO images created with makefs prior to NetBSD bin/45217 bugfix (up to 9.0-BETA1). In addition, merge following important bugfixes from libarchive's release/2.8 branch: Revision 2812: Merge 2811 from trunk: Don't try to verify that compression-level=0 produces larger results than the default compression, since this isn't true for all versions of liblzma. Revision 2817: Merge 2814 from trunk: Fix Issue 121 (mtree parser error) http://code.google.com/p/libarchive/issues/detail?id=121 Revision 2820: Fix issue 119. Change the file location check that a file location does not exceed volume block. New one is that a file content does not exceed volume block(end of an ISO image). It is better than previous check even if the issue did not happen. While reading an ISO image generated by an older version of mkisofs utility, a file location indicates the end the ISO image if its file size is zero and it is the last file of all files of the ISO image, so it is possible that the location value is the same as the number of the total block of the ISO image. http://code.google.com/p/libarchive/issues/detail?id=119 Revision 2955: Issue 134: Fix libarchive 2.8 crashing in archive_write_finish() when the open has failed and we're trying to write Zip format. http://code.google.com/p/libarchive/issues/detail?id=134 Revision 2958: Followup on Issue 134: 1) Port test_open_failure to libarchive 2.8 branch to test the problem reported in Issue 134. This test also shows that archive_read_open() sometimes fails to report open errors correctly. 2) Fix the bug in archive_read.c 3) Comment out the tests that close functions are invoked promptly when open fails; that's fully fixed in libarchive 3.0, but I don't think it's worth fixing here. Revision 3484: Use uintmax_t with %ju Revision 3487: Fix issue 163. Correctly allocate enough memory for a input buffer saved. http://code.google.com/p/libarchive/issues/detail?id=163 Revision 3542: Merge 2516, 2536 from trunk: Allow path table offset values of 0 and 18, which are used by some ISO writers. MFC r224700 [1]: Merge revision 3554 from libarchive's release/2.8 branch: Partial merge of 2431 from trunk: Retry writes on EINTR. This should fix the SIGINT handler in bsdtar. Note: The rest of r2431 can't be merged, since it interacts with a big write-side rearchitecture. PR: bin/149409 [1] Reviewed by: kientzle

Sync libarchive with 9-CURRENT:

2011-08-01T07:59:02Z

MFC on lib/libarchive: 196961,200351,201077,201079-201097,201099-201112,201155-201157, 201159-201163,201165,201167-201171,201246-201250,201253,202868-202869, 202871-202873,203952,208027,208169,208184,208263,208291,210825,213573, 214822,214867,214905,216258,221472,224152 MFC 196961 (kientzle): Update tests to match r195873, which corrected how hardlinked files on iso9660 images were returned. While I'm poking around, update some comments around this area to try to clarify what's going on and what still remains to be improved. MFC 200351 (kientzle): Merge two cpio fixes from libarchive.googlecode.com: 1) Avoid an infinite loop in the header resync for certain malformed archives. 2) Don't try to match hardlinks if the nlinks count is < 2. This reduces the likelihood of a false hardlink match due to ino truncation. MFC 201077 (kientzle): Update format manpages for libarchive. MFC 201079 (kientzle): Update the hard-coded configuration for libarchive. MFC 201080 (kientzle): Remove some unused variables and dead assignments. MFC 201081 (kientzle): Remove a dead assignment. MFC 201082 (kientzle): Catch certain gzip failures at close time. MFC 201083 (kientzle): Compatibility fix for some older systems with non-POSIX getgrnam_r/getpwnam_r and a minor style fix for the hash function. MFC 201084 (kientzle): Sync with googlecode some changes that have no impact on FreeBSD. MFC 201085 (kientzle): Compatibility: I found some more compilers that don't like 'inline' keyword. Protection: Discourage people from using this header outside libarchive. MFC 201086 (kientzle): Discourage people from using this header outside of libarchive. MFC 201087 (kientzle): Enforce this as an internal-only header. MFC 201088 (kientzle): Allow more formats to be active at a time; discourage outside use. MFC 201089 (kientzle): Portability: terminate abnormally via abort() instead of segfault, watch the return value from write(), and avoid signed arithmetic on unsigned values. MFC 201091 (kientzle): Remove dead assignment. MFC 201092 (kientzle): Discourage outside use of this internal header. MFC 201093 (kientzle): Portability: Set the file descriptor to binary mode on Win32 MFC 201094 (kientzle): Eliminate an unused assignment. MFC 201095 (kientzle): Remove an unused variable and an unnecessary increment. MFC 201096 (kientzle): Various portability fixes, plus: * New "ino64" field. * New UTF8 interfaces for hardlink/symlink updates MFC 201097 (kientzle): Portability fixes. MFC 201099 (kientzle): Use new ino64 interface. MFC 201110 (kientzle): Clarify an example so Linux folks won't keep getting confused by this point. MFC 201111 (kientzle): Remove some dead assignments, fix some declarations. MFC 201112 (kientzle): Portability. MFC 201155 (kientzle): 64-bit ino support and discourage bad use of this header. MFC 201156 (kientzle): Various style and portability fixes, including: * Enforce option interface can only be used before the archive is opened * Correctly handle large skips on platforms with 32-bit off_t * Use int64_t instead of off_t MFC 201157 (kientzle): Catch decompression failures earlier. In particular, this gives immediate feedback if we fail to fork an external decompression program. MFC 201159 (kientzle): Various fixes when creating objects on disk: * Write xattrs last instead of first (required on platforms that use system xattrs for security attributes) * Better handling of chdir() failures * Don't bother trying to shorten files via seek()/write() * Fix build on systems that lack link()/symlink()/mknod() * Prefer futimens()/utimensat() when they're present MFC 201160 (kientzle): Various style fixes to the 'newc' cpio writer: * Warn about truncation of ino * Use a macro to hide the somewhat obscure pad-to-multiple-of-4 calculation * Eliminate some unused variables MFC 201161 (kientzle): Style fixes to tar reader: For portability, prefer int64_t to off_t. Improve numeric overflow handling when parsing. Fix some variable types. Eliminate some unused results. MFC 201162 (kientzle): Style and portability fixes for pax writer. Mostly very routine, except for: * Use ino64 where appropriate * Don't pass atime or nsec to ustar formatter MFC 201163 (kientzle): Be a little more skeptical of dev/ino matches when reading cpio files. This eliminates some false-positives in the hardlink detection logic. MFC 201165 (kientzle): Minor style fixes plus a fix for an obscure crash on certain malformed mtree files. MFC 201167 (kientzle): If we're linking against liblzma, allow the decompressor to use up to 1<<30 bytes of memory, which is required for some streams. Also, try to make the taster more discriminating about raw lzma streams. The detection here is still really weak, though; please use xz instead of lzma. MFC 201168 (kientzle): Zip write support for libarchive. The initial implementation was developed by Anselm Strauss as part of Google Summer of Code 2008, then completed by Joerg Sonnenberger. MFC 201169 (kientzle): Clear extraneous error messages. MFC 201170 (kientzle): Essentially eliminate problems with false hardlinks in POSIX cpio archives by generating synthetic ino values and mapping values from disk into the new values. MFC 201171 (kientzle): Rationalize the support for cryptographic hashes. MFC 201246 (kientzle): Merge Michihiro NAKAJIMA's significant work on the ISO9660 reader from googlecode: * Support for zisofs compressed entries * Support for relocated deep directories * Direct calculation of link counts for accurate nlink values even on images that lack Rockridge extensions * Faster handling of the internal file lists. * Better detection of ISO variants MFC 201247 (kientzle): A raft of test changes and improvements from the Googlecode repository. In particular, this includes tests for the new features I've merged over the last few days. MFC 201248 (kientzle): UU decoder. Now that libarchive can recursively taste input streams, you can do things like this: tar xvf archive.tar.gz.uu MFC 201249 (kientzle): Libarchive in FreeBSD is now synced up with 2.7.901a from libarchive.googlecode.com MFC 201250 (kientzle): Reserve constants for RPM unwrapper and XAR reader. MFC 201253 (kientzle): Remove some test files that are no longer used. MFC 202868 (kientzle): Fix a memory leak when a filter fails to initialize. MFC 202869 (kientzle): Initialize the allocated 'struct zip'. MFC 202871 (kientzle): If we can't stat a file, return the correct ARCHIVE_FAILED (this entry can't be processed any further) and a suitable error string. In particular, this improves the error-reporting when cpio -o is given a nonexistent filename. MFC 202872 (kientzle): Use a simpler memory-management strategy for the file objects. Instead of trying to reference-count them and free them as soon as they are no longer needed, we now just keep them around and free them all when we release the archive object. This fixes a number of minor memory leaks, especially when reading damaged archives. MFC 202873 (kientzle): Don't try to hardlink block or character device nodes that come out of cpio archives. MFC 203952 (uqs): Add missing newline in last line of file. Uncovered via: fromcvs vs. svn MFC 208027 (uqs): mdoc: move CAVEATS, BUGS and SECURITY CONSIDERATIONS sections to the bottom of the manpages and order them consistently. GNU groff doesn't care about the ordering, and doesn't even mention CAVEATS and SECURITY CONSIDERATIONS as common sections and where to put them. Found by: mdocml lint run MFC 208169 (kientzle): Reorganize slightly in preparation for making lzma and bz2 support conditional. MFC 208184 (delphij): Fix build. MFC 208263 (kientzle): Retry reads that fail with EINTR. This fixes a problem with bsdtar failing on SIGINT. MFC 208291 (uqs): mdoc: consistently spell our email addresses MFC 210825 (joel): Fix typo. MFC 213573 (uqs): mdoc: drop redundant .Pp and .LP calls They have no effect when coming in pairs, or before .Bl/.Bd MFC 214822 (kientzle): Clarify the naming: Methods that free an object should be called "free". Retain the old "finish" names to preserve source compatibility for now. MFC 214867 (uqs): Fix manpage markup. MFC 214905 (kientzle): If the Zip reader doesn't see a PK signature block because there's inter-entry garbage, just scan forward to find the next one. This allows us to handle a lot of Zip archives that have been modified in-place. Thanks to: Gleb Kurtsou for sending me a sample archive MFC 216258 (kientzle): Don't write data into an empty "file." In particular, this check avoids a warning when extracting directory entries from certain GNU tar archives that store directory contents. MFC 221472 (obrien): libarchive is mixing libmd and libcrypto -- correct to use one or the other. [mixing the two can be quite bad -- they define the same context structures, but with differing structure members (and sizes)] MFC 224152 (mm): - Update libarchive to 2.8.4 - Add support for extracting xar and rpm archives - Add libarchive_fe subdir (common code for tar and cpio)

MFC archive_file_count() utility function.

2010-11-13T06:07:39Z

- Add liblzma support to libarchive and usr.bin/tar

2010-10-10T08:41:08Z

- Do not link usr.bin/ar to liblzma if building bootstrap-tools and the local system doesn't include liblzma (fixes world build on FreeBSD 7.x) This is a direct commit. Approved by: delphij (mentor)

MFC r206622,207939,208593,208732,209031: mdoc cleanup

2010-08-27T10:47:17Z

- order prologue by Dd/Dt/Os - fix prologue - remove empty lines and other garbage - uppercase document title

MFC r204111: Fix common misspelling of hierarchy

2010-08-26T20:44:12Z

MFC r210191:

2010-07-24T11:15:40Z

s/an struct/a struct PR: docs/148660 Submitted by: Warren Block (wblock at wonkity dot com) Reviewed by: Tim Kientzle Mentored by: jkois