Upgrade to Bzip2 version 1.0.8.

MFC after:	3 days

18 files changed, 76 insertions, 53 deletions

diff --git a/contrib/bzip2/CHANGES b/contrib/bzip2/CHANGES
index d9b4c05c52ce6..30afead2586b6 100644
--- a/contrib/bzip2/CHANGES
+++ b/contrib/bzip2/CHANGES
@@ -2,8 +2,8 @@
  This file is part of bzip2/libbzip2, a program and library for
  lossless, block-sorting data compression.
 
- bzip2/libbzip2 version 1.0.7 of 27 June 2019
- Copyright (C) 1996-2010 Julian Seward <jseward@acm.org>
+ bzip2/libbzip2 version 1.0.8 of 13 July 2019
+ Copyright (C) 1996-2019 Julian Seward <jseward@acm.org>
 
  Please read the WARNING, DISCLAIMER and PATENTS sections in the 
  README file.
@@ -338,3 +338,19 @@ Security fix only.  Fixes CERT-FI 20469 as it applies to bzip2.
 * bzip2recover: Fix use after free issue with outFile (CVE-2016-3189)
 
 * Make sure nSelectors is not out of range (CVE-2019-12900)
+
+1.0.8 (13 Jul 19)
+~~~~~~~~~~~~~~~~~
+
+* Accept as many selectors as the file format allows.
+  This relaxes the fix for CVE-2019-12900 from 1.0.7
+  so that bzip2 allows decompression of bz2 files that
+  use (too) many selectors again.
+
+* Fix handling of large (> 4GB) files on Windows.
+
+* Cleanup of bzdiff and bzgrep scripts so they don't use
+  any bash extensions and handle multiple archives correctly.
+
+* There is now a bz2-files testsuite at
+  https://sourceware.org/git/bzip2-tests.git
diff --git a/contrib/bzip2/LICENSE b/contrib/bzip2/LICENSE
index 95f9598f847d8..81a37eab7a5be 100644
--- a/contrib/bzip2/LICENSE
+++ b/contrib/bzip2/LICENSE
@@ -2,7 +2,7 @@
 --------------------------------------------------------------------------
 
 This program, "bzip2", the associated library "libbzip2", and all
-documentation, are copyright (C) 1996-2010 Julian R Seward.  All
+documentation, are copyright (C) 1996-2019 Julian R Seward.  All
 rights reserved.
 
 Redistribution and use in source and binary forms, with or without
@@ -37,6 +37,6 @@ NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
 SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 
 Julian Seward, jseward@acm.org
-bzip2/libbzip2 version 1.0.7 of 27 June 2019
+bzip2/libbzip2 version 1.0.8 of 13 July 2019
 
 --------------------------------------------------------------------------
diff --git a/contrib/bzip2/README b/contrib/bzip2/README
index 64873f9826d85..b9c6099fd1c98 100644
--- a/contrib/bzip2/README
+++ b/contrib/bzip2/README
@@ -6,8 +6,8 @@ This version is fully compatible with the previous public releases.
 This file is part of bzip2/libbzip2, a program and library for
 lossless, block-sorting data compression.
 
-bzip2/libbzip2 version 1.0.7 of 27 June 2019
-Copyright (C) 1996-2010 Julian Seward <jseward@acm.org>
+bzip2/libbzip2 version 1.0.8 of 13 July 2019
+Copyright (C) 1996-2019 Julian Seward <jseward@acm.org>
 
 Please read the WARNING, DISCLAIMER and PATENTS sections in this file.
 
@@ -165,8 +165,8 @@ WHAT'S NEW IN 1.0.x ?
 
    See the CHANGES file.
 
-I hope you find bzip2 useful.  Feel free to contact me at
-   jseward@acm.org
+I hope you find bzip2 useful.  Feel free to contact the developers at
+   bzip2-devel@sourceware.org
 if you have any suggestions or queries.  Many people mailed me with
 comments, suggestions and patches after the releases of bzip-0.15,
 bzip-0.21, and bzip2 versions 0.1pl2, 0.9.0, 0.9.5, 1.0.0, 1.0.1,
@@ -193,3 +193,4 @@ Cambridge, UK.
 10 December 2007 (bzip2, version 1.0.5)
  6     Sept 2010 (bzip2, version 1.0.6)
 27     June 2019 (bzip2, version 1.0.7)
+13     July 2019 (bzip2, version 1.0.8)
diff --git a/contrib/bzip2/README.COMPILATION.PROBLEMS b/contrib/bzip2/README.COMPILATION.PROBLEMS
index 9e6ac5a1c88e3..fa317a50c8142 100644
--- a/contrib/bzip2/README.COMPILATION.PROBLEMS
+++ b/contrib/bzip2/README.COMPILATION.PROBLEMS
@@ -2,8 +2,8 @@
 This file is part of bzip2/libbzip2, a program and library for
 lossless, block-sorting data compression.
 
-bzip2/libbzip2 version 1.0.7 of 27 June 2019
-Copyright (C) 1996-2010 Julian Seward <jseward@acm.org>
+bzip2/libbzip2 version 1.0.8 of 13 July 2019
+Copyright (C) 1996-2019 Julian Seward <jseward@acm.org>
 
 Please read the WARNING, DISCLAIMER and PATENTS sections in the 
 README file.
diff --git a/contrib/bzip2/blocksort.c b/contrib/bzip2/blocksort.c
index 26579017af107..92d81fe287eae 100644
--- a/contrib/bzip2/blocksort.c
+++ b/contrib/bzip2/blocksort.c
@@ -8,8 +8,8 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.7 of 27 June 2019
-   Copyright (C) 1996-2010 Julian Seward <jseward@acm.org>
+   bzip2/libbzip2 version 1.0.8 of 13 July 2019
+   Copyright (C) 1996-2019 Julian Seward <jseward@acm.org>
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
diff --git a/contrib/bzip2/bzip2.1 b/contrib/bzip2/bzip2.1
index 87c2ce8d9cbfb..a358e08cce23b 100644
--- a/contrib/bzip2/bzip2.1
+++ b/contrib/bzip2/bzip2.1
@@ -1,6 +1,6 @@
 .TH bzip2 1
 .SH NAME
-bzip2, bunzip2 \- a block-sorting file compressor, v1.0.7
+bzip2, bunzip2 \- a block-sorting file compressor, v1.0.8
 .br
 bzcat \- decompresses files to stdout
 .br
@@ -404,7 +404,7 @@ I/O error messages are not as helpful as they could be.
 tries hard to detect I/O errors and exit cleanly, but the details of
 what the problem is sometimes seem rather misleading.
 
-This manual page pertains to version 1.0.7 of
+This manual page pertains to version 1.0.8 of
 .I bzip2.  
 Compressed data created by this version is entirely forwards and
 backwards compatible with the previous public releases, versions
diff --git a/contrib/bzip2/bzip2.c b/contrib/bzip2/bzip2.c
index c732669429af5..6ec9871747176 100644
--- a/contrib/bzip2/bzip2.c
+++ b/contrib/bzip2/bzip2.c
@@ -7,8 +7,8 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.7 of 27 June 2019
-   Copyright (C) 1996-2010 Julian Seward <jseward@acm.org>
+   bzip2/libbzip2 version 1.0.8 of 13 July 2019
+   Copyright (C) 1996-2019 Julian Seward <jseward@acm.org>
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
@@ -128,12 +128,12 @@
 #if BZ_LCCWIN32
 #   include <io.h>
 #   include <fcntl.h>
-#   include <sys\stat.h>
+#   include <sys/stat.h>
 
 #   define NORETURN       /**/
 #   define PATH_SEP       '\\'
-#   define MY_LSTAT       _stat
-#   define MY_STAT        _stat
+#   define MY_LSTAT       _stati64
+#   define MY_STAT        _stati64
 #   define MY_S_ISREG(x)  ((x) & _S_IFREG)
 #   define MY_S_ISDIR(x)  ((x) & _S_IFDIR)
 
@@ -748,8 +748,8 @@ void panic ( const Char* s )
    fprintf ( stderr,
              "\n%s: PANIC -- internal consistency error:\n"
              "\t%s\n"
-             "\tThis is a BUG.  Please report it to me at:\n"
-             "\tjseward@acm.org\n",
+             "\tThis is a BUG.  Please report it to:\n"
+             "\tbzip2-devel@sourceware.org\n",
              progName, s );
    showFileNames();
    cleanUpAndFail( 3 );
@@ -829,7 +829,7 @@ void mySIGSEGVorSIGBUScatcher ( IntNative n )
       "   The user's manual, Section 4.3, has more info on (1) and (2).\n"
       "   \n"
       "   If you suspect this is a bug in bzip2, or are unsure about (1)\n"
-      "   or (2), feel free to report it to me at: jseward@acm.org.\n"
+      "   or (2), feel free to report it to: bzip2-devel@sourceware.org.\n"
       "   Section 4.3 of the user's manual describes the info a useful\n"
       "   bug report should have.  If the manual is available on your\n"
       "   system, please try and read it before mailing me.  If you don't\n"
@@ -852,7 +852,7 @@ void mySIGSEGVorSIGBUScatcher ( IntNative n )
       "   The user's manual, Section 4.3, has more info on (2) and (3).\n"
       "   \n"
       "   If you suspect this is a bug in bzip2, or are unsure about (2)\n"
-      "   or (3), feel free to report it to me at: jseward@acm.org.\n"
+      "   or (3), feel free to report it to: bzip2-devel@sourceware.org.\n"
       "   Section 4.3 of the user's manual describes the info a useful\n"
       "   bug report should have.  If the manual is available on your\n"
       "   system, please try and read it before mailing me.  If you don't\n"
@@ -1605,7 +1605,7 @@ void license ( void )
     "bzip2, a block-sorting file compressor.  "
     "Version %s.\n"
     "   \n"
-    "   Copyright (C) 1996-2010 by Julian Seward.\n"
+    "   Copyright (C) 1996-2019 by Julian Seward.\n"
     "   \n"
     "   This program is free software; you can redistribute it and/or modify\n"
     "   it under the terms set out in the LICENSE file, which is included\n"
diff --git a/contrib/bzip2/bzip2recover.c b/contrib/bzip2/bzip2recover.c
index 45de311275363..136c72a494c69 100644
--- a/contrib/bzip2/bzip2recover.c
+++ b/contrib/bzip2/bzip2recover.c
@@ -7,8 +7,8 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.7 of 27 June 2019
-   Copyright (C) 1996-2010 Julian Seward <jseward@acm.org>
+   bzip2/libbzip2 version 1.0.8 of 13 July 2019
+   Copyright (C) 1996-2019 Julian Seward <jseward@acm.org>
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
@@ -314,7 +314,7 @@ Int32 main ( Int32 argc, Char** argv )
    inFileName[0] = outFileName[0] = 0;
 
    fprintf ( stderr, 
-             "bzip2recover 1.0.7: extracts blocks from damaged .bz2 files.\n" );
+             "bzip2recover 1.0.8: extracts blocks from damaged .bz2 files.\n" );
 
    if (argc != 2) {
       fprintf ( stderr, "%s: usage is `%s damaged_file_name'.\n",
diff --git a/contrib/bzip2/bzlib.c b/contrib/bzip2/bzlib.c
index 82dd0bad65f73..9496d5abc9512 100644
--- a/contrib/bzip2/bzlib.c
+++ b/contrib/bzip2/bzlib.c
@@ -8,8 +8,8 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.7 of 27 June 2019
-   Copyright (C) 1996-2010 Julian Seward <jseward@acm.org>
+   bzip2/libbzip2 version 1.0.8 of 13 July 2019
+   Copyright (C) 1996-2019 Julian Seward <jseward@acm.org>
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
@@ -47,12 +47,12 @@ void BZ2_bz__AssertH__fail ( int errcode )
    fprintf(stderr, 
       "\n\nbzip2/libbzip2: internal error number %d.\n"
       "This is a bug in bzip2/libbzip2, %s.\n"
-      "Please report it to me at: jseward@acm.org.  If this happened\n"
+      "Please report it to: bzip2-devel@sourceware.org.  If this happened\n"
       "when you were using some program which uses libbzip2 as a\n"
       "component, you should also report this bug to the author(s)\n"
       "of that program.  Please make an effort to report this bug;\n"
       "timely and accurate bug reports eventually lead to higher\n"
-      "quality software.  Thanks.  Julian Seward, 10 December 2007.\n\n",
+      "quality software.  Thanks.\n\n",
       errcode,
       BZ2_bzlibVersion()
    );
diff --git a/contrib/bzip2/bzlib.h b/contrib/bzip2/bzlib.h
index 8cf07918f8bca..8966a6c5804e8 100644
--- a/contrib/bzip2/bzlib.h
+++ b/contrib/bzip2/bzlib.h
@@ -8,8 +8,8 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.7 of 27 June 2019
-   Copyright (C) 1996-2010 Julian Seward <jseward@acm.org>
+   bzip2/libbzip2 version 1.0.8 of 13 July 2019
+   Copyright (C) 1996-2019 Julian Seward <jseward@acm.org>
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
diff --git a/contrib/bzip2/bzlib_private.h b/contrib/bzip2/bzlib_private.h
index 797555222d525..3755a6f701ec1 100644
--- a/contrib/bzip2/bzlib_private.h
+++ b/contrib/bzip2/bzlib_private.h
@@ -8,8 +8,8 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.7 of 27 June 2019
-   Copyright (C) 1996-2010 Julian Seward <jseward@acm.org>
+   bzip2/libbzip2 version 1.0.8 of 13 July 2019
+   Copyright (C) 1996-2019 Julian Seward <jseward@acm.org>
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
@@ -36,7 +36,7 @@
 
 /*-- General stuff. --*/
 
-#define BZ_VERSION  "1.0.7, 27-Jun-2019"
+#define BZ_VERSION  "1.0.8, 13-Jul-2019"
 
 typedef char            Char;
 typedef unsigned char   Bool;
diff --git a/contrib/bzip2/compress.c b/contrib/bzip2/compress.c
index 237620d85891e..5dfa00231b093 100644
--- a/contrib/bzip2/compress.c
+++ b/contrib/bzip2/compress.c
@@ -8,8 +8,8 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.7 of 27 June 2019
-   Copyright (C) 1996-2010 Julian Seward <jseward@acm.org>
+   bzip2/libbzip2 version 1.0.8 of 13 July 2019
+   Copyright (C) 1996-2019 Julian Seward <jseward@acm.org>
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
@@ -454,7 +454,7 @@ void sendMTFValues ( EState* s )
 
    AssertH( nGroups < 8, 3002 );
    AssertH( nSelectors < 32768 &&
-            nSelectors <= (2 + (900000 / BZ_G_SIZE)),
+            nSelectors <= BZ_MAX_SELECTORS,
             3003 );
 
 
diff --git a/contrib/bzip2/crctable.c b/contrib/bzip2/crctable.c
index 746efac1cda90..2b33c2535338d 100644
--- a/contrib/bzip2/crctable.c
+++ b/contrib/bzip2/crctable.c
@@ -8,8 +8,8 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.7 of 27 June 2019
-   Copyright (C) 1996-2010 Julian Seward <jseward@acm.org>
+   bzip2/libbzip2 version 1.0.8 of 13 July 2019
+   Copyright (C) 1996-2019 Julian Seward <jseward@acm.org>
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
diff --git a/contrib/bzip2/decompress.c b/contrib/bzip2/decompress.c
index 20ce4936cdfb1..a1a0bac89227b 100644
--- a/contrib/bzip2/decompress.c
+++ b/contrib/bzip2/decompress.c
@@ -8,8 +8,8 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.7 of 27 June 2019
-   Copyright (C) 1996-2010 Julian Seward <jseward@acm.org>
+   bzip2/libbzip2 version 1.0.8 of 13 July 2019
+   Copyright (C) 1996-2019 Julian Seward <jseward@acm.org>
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
@@ -287,7 +287,7 @@ Int32 BZ2_decompress ( DState* s )
       GET_BITS(BZ_X_SELECTOR_1, nGroups, 3);
       if (nGroups < 2 || nGroups > BZ_N_GROUPS) RETURN(BZ_DATA_ERROR);
       GET_BITS(BZ_X_SELECTOR_2, nSelectors, 15);
-      if (nSelectors < 1 || nSelectors > BZ_MAX_SELECTORS) RETURN(BZ_DATA_ERROR);
+      if (nSelectors < 1) RETURN(BZ_DATA_ERROR);
       for (i = 0; i < nSelectors; i++) {
          j = 0;
          while (True) {
@@ -296,8 +296,14 @@ Int32 BZ2_decompress ( DState* s )
             j++;
             if (j >= nGroups) RETURN(BZ_DATA_ERROR);
          }
-         s->selectorMtf[i] = j;
+         /* Having more than BZ_MAX_SELECTORS doesn't make much sense
+            since they will never be used, but some implementations might
+            "round up" the number of selectors, so just ignore those. */
+         if (i < BZ_MAX_SELECTORS)
+           s->selectorMtf[i] = j;
       }
+      if (nSelectors > BZ_MAX_SELECTORS)
+        nSelectors = BZ_MAX_SELECTORS;
 
       /*--- Undo the MTF values for the selectors. ---*/
       {
diff --git a/contrib/bzip2/huffman.c b/contrib/bzip2/huffman.c
index 0fd6fd762caca..43a1899e4688e 100644
--- a/contrib/bzip2/huffman.c
+++ b/contrib/bzip2/huffman.c
@@ -8,8 +8,8 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.7 of 27 June 2019
-   Copyright (C) 1996-2010 Julian Seward <jseward@acm.org>
+   bzip2/libbzip2 version 1.0.8 of 13 July 2019
+   Copyright (C) 1996-2019 Julian Seward <jseward@acm.org>
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
diff --git a/contrib/bzip2/randtable.c b/contrib/bzip2/randtable.c
index 726d62f0ca0c5..bdc6d4a4cc9a6 100644
--- a/contrib/bzip2/randtable.c
+++ b/contrib/bzip2/randtable.c
@@ -8,8 +8,8 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.7 of 27 June 2019
-   Copyright (C) 1996-2010 Julian Seward <jseward@acm.org>
+   bzip2/libbzip2 version 1.0.8 of 13 July 2019
+   Copyright (C) 1996-2019 Julian Seward <jseward@acm.org>
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
diff --git a/contrib/bzip2/spewG.c b/contrib/bzip2/spewG.c
index 320dc854beb4a..65d24c89cbceb 100644
--- a/contrib/bzip2/spewG.c
+++ b/contrib/bzip2/spewG.c
@@ -13,8 +13,8 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.7 of 27 June 2019
-   Copyright (C) 1996-2010 Julian Seward <jseward@acm.org>
+   bzip2/libbzip2 version 1.0.8 of 13 July 2019
+   Copyright (C) 1996-2019 Julian Seward <jseward@acm.org>
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
diff --git a/contrib/bzip2/unzcrash.c b/contrib/bzip2/unzcrash.c
index 6e7c63198d07a..c68f93c56cbaa 100644
--- a/contrib/bzip2/unzcrash.c
+++ b/contrib/bzip2/unzcrash.c
@@ -17,8 +17,8 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.7 of 27 June 2019
-   Copyright (C) 1996-2010 Julian Seward <jseward@acm.org>
+   bzip2/libbzip2 version 1.0.8 of 13 July 2019
+   Copyright (C) 1996-2019 Julian Seward <jseward@acm.org>
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.