diff options
Diffstat (limited to 'programs/README.md')
| -rw-r--r-- | programs/README.md | 92 |
1 files changed, 50 insertions, 42 deletions
diff --git a/programs/README.md b/programs/README.md index 2da9a6db1261e..a308fccf9ea3b 100644 --- a/programs/README.md +++ b/programs/README.md @@ -11,61 +11,74 @@ There are however other Makefile targets that create different variations of CLI #### Compilation variables -`zstd` scope can be altered by modifying the following compilation variables : +`zstd` scope can be altered by modifying the following `make` variables : - __HAVE_THREAD__ : multithreading is automatically enabled when `pthread` is detected. - It's possible to disable multithread support, by setting HAVE_THREAD=0 . - Example : make zstd HAVE_THREAD=0 - It's also possible to force compilation with multithread support, using HAVE_THREAD=1. - In which case, linking stage will fail if `pthread` library cannot be found. - This might be useful to prevent silent feature disabling. + It's possible to disable multithread support, by setting `HAVE_THREAD=0`. + Example : `make zstd HAVE_THREAD=0` + It's also possible to force multithread support, using `HAVE_THREAD=1`. + In which case, linking stage will fail if neither `pthread` nor `windows.h` library can be found. + This is useful to ensure this feature is not silently disabled. + +- __ZSTD_LEGACY_SUPPORT__ : `zstd` can decompress files compressed by older versions of `zstd`. + Starting v0.8.0, all versions of `zstd` produce frames compliant with the [specification](../doc/zstd_compression_format.md), and are therefore compatible. + But older versions (< v0.8.0) produced different, incompatible, frames. + By default, `zstd` supports decoding legacy formats >= v0.4.0 (`ZSTD_LEGACY_SUPPORT=4`). + This can be altered by modifying this compilation variable. + `ZSTD_LEGACY_SUPPORT=1` means "support all formats >= v0.1.0". + `ZSTD_LEGACY_SUPPORT=2` means "support all formats >= v0.2.0", and so on. + `ZSTD_LEGACY_SUPPORT=0` means _DO NOT_ support any legacy format. + if `ZSTD_LEGACY_SUPPORT >= 8`, it's the same as `0`, since there is no legacy format after `7`. + Note : `zstd` only supports decoding older formats, and cannot generate any legacy format. - __HAVE_ZLIB__ : `zstd` can compress and decompress files in `.gz` format. This is ordered through command `--format=gzip`. Alternatively, symlinks named `gzip` or `gunzip` will mimic intended behavior. `.gz` support is automatically enabled when `zlib` library is detected at build time. - It's possible to disable `.gz` support, by setting HAVE_ZLIB=0. - Example : make zstd HAVE_ZLIB=0 - It's also possible to force compilation with zlib support, using HAVE_ZLIB=1. + It's possible to disable `.gz` support, by setting `HAVE_ZLIB=0`. + Example : `make zstd HAVE_ZLIB=0` + It's also possible to force compilation with zlib support, `using HAVE_ZLIB=1`. In which case, linking stage will fail if `zlib` library cannot be found. - This might be useful to prevent silent feature disabling. + This is useful to prevent silent feature disabling. - __HAVE_LZMA__ : `zstd` can compress and decompress files in `.xz` and `.lzma` formats. This is ordered through commands `--format=xz` and `--format=lzma` respectively. Alternatively, symlinks named `xz`, `unxz`, `lzma`, or `unlzma` will mimic intended behavior. `.xz` and `.lzma` support is automatically enabled when `lzma` library is detected at build time. - It's possible to disable `.xz` and `.lzma` support, by setting HAVE_LZMA=0 . - Example : make zstd HAVE_LZMA=0 - It's also possible to force compilation with lzma support, using HAVE_LZMA=1. + It's possible to disable `.xz` and `.lzma` support, by setting `HAVE_LZMA=0` . + Example : `make zstd HAVE_LZMA=0` + It's also possible to force compilation with lzma support, using `HAVE_LZMA=1`. In which case, linking stage will fail if `lzma` library cannot be found. - This might be useful to prevent silent feature disabling. + This is useful to prevent silent feature disabling. - __HAVE_LZ4__ : `zstd` can compress and decompress files in `.lz4` formats. This is ordered through commands `--format=lz4`. Alternatively, symlinks named `lz4`, or `unlz4` will mimic intended behavior. `.lz4` support is automatically enabled when `lz4` library is detected at build time. - It's possible to disable `.lz4` support, by setting HAVE_LZ4=0 . - Example : make zstd HAVE_LZ4=0 - It's also possible to force compilation with lz4 support, using HAVE_LZ4=1. + It's possible to disable `.lz4` support, by setting `HAVE_LZ4=0` . + Example : `make zstd HAVE_LZ4=0` + It's also possible to force compilation with lz4 support, using `HAVE_LZ4=1`. In which case, linking stage will fail if `lz4` library cannot be found. - This might be useful to prevent silent feature disabling. - -- __ZSTD_LEGACY_SUPPORT__ : `zstd` can decompress files compressed by older versions of `zstd`. - Starting v0.8.0, all versions of `zstd` produce frames compliant with the [specification](../doc/zstd_compression_format.md), and are therefore compatible. - But older versions (< v0.8.0) produced different, incompatible, frames. - By default, `zstd` supports decoding legacy formats >= v0.4.0 (`ZSTD_LEGACY_SUPPORT=4`). - This can be altered by modifying this compilation variable. - `ZSTD_LEGACY_SUPPORT=1` means "support all formats >= v0.1.0". - `ZSTD_LEGACY_SUPPORT=2` means "support all formats >= v0.2.0", and so on. - `ZSTD_LEGACY_SUPPORT=0` means _DO NOT_ support any legacy format. - if `ZSTD_LEGACY_SUPPORT >= 8`, it's the same as `0`, since there is no legacy format after `7`. - Note : `zstd` only supports decoding older formats, and cannot generate any legacy format. + This is useful to prevent silent feature disabling. #### Aggregation of parameters CLI supports aggregation of parameters i.e. `-b1`, `-e18`, and `-i1` can be joined into `-b1e18i1`. +#### Symlink shortcuts +It's possible to invoke `zstd` through a symlink. +When the name of the symlink has a specific value, it triggers an associated behavior. +- `zstdmt` : compress using all cores available on local system. +- `zcat` : will decompress and output target file using any of the supported formats. `gzcat` and `zstdcat` are also equivalent. +- `gzip` : if zlib support is enabled, will mimic `gzip` by compressing file using `.gz` format, removing source file by default (use `--keep` to preserve). If zlib is not supported, triggers an error. +- `xz` : if lzma support is enabled, will mimic `xz` by compressing file using `.xz` format, removing source file by default (use `--keep` to preserve). If xz is not supported, triggers an error. +- `lzma` : if lzma support is enabled, will mimic `lzma` by compressing file using `.lzma` format, removing source file by default (use `--keep` to preserve). If lzma is not supported, triggers an error. +- `lz4` : if lz4 support is enabled, will mimic `lz4` by compressing file using `.lz4` format. If lz4 is not supported, triggers an error. +- `unzstd` and `unlz4` will decompress any of the supported format. +- `ungz`, `unxz` and `unlzma` will do the same, and will also remove source file by default (use `--keep` to preserve). + + #### Dictionary builder in Command Line Interface Zstd offers a training mode, which can be used to tune the algorithm for a selected type of data, by providing it with a few samples. The result of the training is stored @@ -107,7 +120,7 @@ Usage : FILE : a filename with no FILE, or when FILE is - , read standard input Arguments : - -# : # compression level (1-19, default:3) + -# : # compression level (1-19, default: 3) -d : decompression -D file: use `file` as Dictionary -o file: result stored into `file` (only if 1 input file) @@ -125,13 +138,13 @@ Advanced arguments : --ultra : enable levels beyond 19, up to 22 (requires more memory) --long : enable long distance matching (requires more memory) --no-dictID : don't write dictID into header (dictionary compression) ---[no-]check : integrity check (default:enabled) +--[no-]check : integrity check (default: enabled) -r : operate recursively on directories --format=gzip : compress files to the .gz format --format=xz : compress files to the .xz format --format=lzma : compress files to the .lzma format --test : test compressed file integrity ---[no-]sparse : sparse mode (default:disabled) +--[no-]sparse : sparse mode (default: disabled) -M# : Set a memory usage limit for decompression -- : All arguments after "--" are treated as files @@ -140,13 +153,13 @@ Dictionary builder : --train-cover[=k=#,d=#,steps=#] : use the cover algorithm with optional args --train-legacy[=s=#] : use the legacy algorithm with selectivity (default: 9) -o file : `file` is dictionary name (default: dictionary) ---maxdict=# : limit dictionary to specified size (default : 112640) +--maxdict=# : limit dictionary to specified size (default: 112640) --dictID=# : force dictionary ID to specified value (default: random) Benchmark arguments : - -b# : benchmark file(s), using # compression level (default : 1) + -b# : benchmark file(s), using # compression level (default: 3) -e# : test all compression levels from -bX to # (default: 1) - -i# : minimum evaluation time in seconds (default : 3s) + -i# : minimum evaluation time in seconds (default: 3s) -B# : cut file into independent blocks of size # (default: no block) --priority=rt : set process priority to real-time ``` @@ -155,7 +168,7 @@ Benchmark arguments : #### Long distance matching mode The long distance matching mode, enabled with `--long`, is designed to improve the compression ratio for files with long matches at a large distance (up to the -maximum window size, `128 MiB`) while still maintaining compression speed. +maximum window size, `128 MiB`) while still maintaining compression speed. Enabling this mode sets the window size to `128 MiB` and thus increases the memory usage for both the compressor and decompressor. Performance in terms of speed is @@ -168,7 +181,7 @@ decompression speed with and without long distance matching on an ideal use case: a tar of four versions of clang (versions `3.4.1`, `3.4.2`, `3.5.0`, `3.5.1`) with a total size of `244889600 B`. This is an ideal use case as there are many long distance matches within the maximum window size of `128 MiB` (each -version is less than `128 MiB`). +version is less than `128 MiB`). Compression Speed vs Ratio | Decompression Speed ---------------------------|--------------------- @@ -202,8 +215,3 @@ The below table illustrates this on the [Silesia compression corpus]. | `zstd -5 --long` | `3.319` | `51.7 MB/s` | `371.9 MB/s` | | `zstd -10` | `3.523` | `16.4 MB/s` | `489.2 MB/s` | | `zstd -10 --long`| `3.566` | `16.2 MB/s` | `415.7 MB/s` | - - - - - |