diff options
Diffstat (limited to 'programs/zstd.1')
| -rw-r--r-- | programs/zstd.1 | 408 |
1 files changed, 408 insertions, 0 deletions
diff --git a/programs/zstd.1 b/programs/zstd.1 new file mode 100644 index 000000000000..684fb868aa30 --- /dev/null +++ b/programs/zstd.1 @@ -0,0 +1,408 @@ +\" +\" zstd.1: This is a manual page for 'zstd' program. This file is part of the +\" zstd <http://www.zstd.net/> project. +\" Author: Yann Collet +\" + +\" No hyphenation +.hy 0 +.nr HY 0 + +.TH zstd "1" "2015-08-22" "zstd" "User Commands" +.SH NAME +\fBzstd, unzstd, zstdcat\fR - Compress or decompress .zst files + +.SH SYNOPSIS +.TP 5 +\fBzstd\fR [\fBOPTIONS\fR] [-|INPUT-FILE] [-o <OUTPUT-FILE>] +.PP +.B unzstd +is equivalent to +.BR "zstd \-d" +.br +.B zstdcat +is equivalent to +.BR "zstd \-dcf" +.br + +.SH DESCRIPTION +.PP +\fBzstd\fR is a fast lossless compression algorithm +and data compression tool, +with command line syntax similar to \fB gzip (1) \fR and \fB xz (1) \fR . +It is based on the \fBLZ77\fR family, with further FSE & huff0 entropy stages. +\fBzstd\fR offers highly configurable compression speed, +with fast modes at > 200 MB/s per core, +and strong modes nearing lzma compression ratios. +It also features a very fast decoder, with speeds > 500 MB/s per core. + +\fBzstd\fR command line syntax is generally similar to gzip, +but features the following differences : + - Source files are preserved by default. + It's possible to remove them automatically by using \fB--rm\fR command. + - When compressing a single file, \fBzstd\fR displays progress notifications and result summary by default. + Use \fB-q\fR to turn them off + +.PP +.B zstd +compresses or decompresses each +.I file +according to the selected operation mode. +If no +.I files +are given or +.I file +is +.BR \- , +.B zstd +reads from standard input and writes the processed data +to standard output. +.B zstd +will refuse (display an error and skip the +.IR file ) +to write compressed data to standard output if it is a terminal. +Similarly, +.B zstd +will refuse to read compressed data +from standard input if it is a terminal. + +.PP +Unless +.B \-\-stdout +or +.B \-o +is specified, +.I files +are written to a new file whose name is derived from the source +.I file +name: +.IP \(bu 3 +When compressing, the suffix +.B .zst +is appended to the source filename to get the target filename. +.IP \(bu 3 +When decompressing, the +.B .zst +suffix is removed from the filename to get the target filename. + +.SS "Concatenation with .zst files" +It is possible to concatenate +.B .zst +files as is. +.B zstd +will decompress such files as if they were a single +.B .zst +file. + + + +.SH OPTIONS + +. +.SS "Integer suffixes and special values" +In most places where an integer argument is expected, +an optional suffix is supported to easily indicate large integers. +There must be no space between the integer and the suffix. +.TP +.B KiB +Multiply the integer by 1,024 (2^10). +.BR Ki , +.BR K , +and +.B KB +are accepted as synonyms for +.BR KiB . +.TP +.B MiB +Multiply the integer by 1,048,576 (2^20). +.BR Mi , +.BR M , +and +.B MB +are accepted as synonyms for +.BR MiB . + +. +.SS "Operation mode" +If multiple operation mode options are given, +the last one takes effect. +.TP +.BR \-z ", " \-\-compress +Compress. +This is the default operation mode when no operation mode option +is specified and no other operation mode is implied from +the command name (for example, +.B unzstd +implies +.BR \-\-decompress ). +.TP +.BR \-d ", " \-\-decompress ", " \-\-uncompress +Decompress. +.TP +.BR \-t ", " \-\-test +Test the integrity of compressed +.IR files . +This option is equivalent to +.B "\-\-decompress \-\-stdout" +except that the decompressed data is discarded instead of being +written to standard output. +No files are created or removed. +.TP +.B \-b# + benchmark file(s) using compression level # +.TP +.B \--train FILEs + use FILEs as training set to create a dictionary. The training set should contain a lot of small files (> 100). + +. +.SS "Operation modifiers" +.TP +.B \-# + # compression level [1-19] (default:3) +.TP +.BR \--ultra + unlocks high compression levels 20+ (maximum 22), using a lot more memory. +Note that decompression will also require more memory when using these levels. +.TP +.B \-D file + use `file` as Dictionary to compress or decompress FILE(s) +.TP +.BR \--no-dictID + do not store dictionary ID within frame header (dictionary compression). + The decoder will have to rely on implicit knowledge about which dictionary to use, +it won't be able to check if it's correct. +.TP +.B \-o file + save result into `file` (only possible with a single INPUT-FILE) +.TP +.BR \-f ", " --force + overwrite output without prompting +.TP +.BR \-c ", " --stdout + force write to standard output, even if it is the console +.TP +.BR \--[no-]sparse + enable / disable sparse FS support, to make files with many zeroes smaller on disk. + Creating sparse files may save disk space and speed up the decompression +by reducing the amount of disk I/O. + default : enabled when output is into a file, and disabled when output is stdout. + This setting overrides default and can force sparse mode over stdout. +.TP +.BR \--rm + remove source file(s) after successful compression or decompression +.TP +.BR \-k ", " --keep + keep source file(s) after successful compression or decompression. + This is the default behavior. +.TP +.BR \-r + operate recursively on directories +.TP +.BR \-h/\-H ", " --help + display help/long help and exit +.TP +.BR \-V ", " --version + display Version number and exit +.TP +.BR \-v ", " --verbose + verbose mode +.TP +.BR \-q ", " --quiet + suppress warnings, interactivity and notifications. + specify twice to suppress errors too. +.TP +.BR \-C ", " --[no-]check + add integrity check computed from uncompressed data (default : enabled) +.TP +.BR \-t ", " --test + Test the integrity of compressed files. This option is equivalent to \fB--decompress --stdout > /dev/null\fR. + No files are created or removed. +.TP +.BR -- + All arguments after -- are treated as files + + +.SH DICTIONARY BUILDER +.PP +\fBzstd\fR offers \fIdictionary\fR compression, useful for very small files and messages. +It's possible to train \fBzstd\fR with some samples, the result of which is saved into a file called `dictionary`. +Then during compression and decompression, make reference to the same dictionary. +It will improve compression ratio of small files. +Typical gains range from ~10% (at 64KB) to x5 better (at <1KB). +.TP +.B \--train FILEs + use FILEs as training set to create a dictionary. The training set should contain a lot of small files (> 100), +and weight typically 100x the target dictionary size (for example, 10 MB for a 100 KB dictionary) +.TP +.B \-o file + dictionary saved into `file` (default: dictionary) +.TP +.B \--maxdict # + limit dictionary to specified size (default : 112640) +.TP +.B \--dictID # + A dictionary ID is a locally unique ID that a decoder can use to verify it is using the right dictionary. + By default, zstd will create a 4-bytes random number ID. + It's possible to give a precise number instead. + Short numbers have an advantage : an ID < 256 will only need 1 byte in the compressed frame header, + and an ID < 65536 will only need 2 bytes. This compares favorably to 4 bytes default. + However, it's up to the dictionary manager to not assign twice the same ID to 2 different dictionaries. +.TP +.B \-s# + dictionary selectivity level (default: 9) + the smaller the value, the denser the dictionary, improving its efficiency but reducing its possible maximum size. +.TP +.B \--cover=k=#,d=# + Use alternate dictionary builder algorithm named cover with parameters \fIk\fR and \fId\fR with \fId\fR <= \fIk\fR. + Selects segments of size \fIk\fR with the highest score to put in the dictionary. + The score of a segment is computed by the sum of the frequencies of all the subsegments of of size \fId\fR. + Generally \fId\fR should be in the range [6, 24]. + Good values for \fIk\fR vary widely based on the input data, but a safe range is [32, 2048]. + Example: \fB--train --cover=k=64,d=8 FILEs\fR. +.TP +.B \--optimize-cover[=steps=#,k=#,d=#] + If \fIsteps\fR is not specified, the default value of 32 is used. + If \fIk\fR is not specified, \fIsteps\fR values in [16, 2048] are checked for each value of \fId\fR. + If \fId\fR is not specified, the values checked are [6, 8, ..., 16]. + + Runs the cover dictionary builder for each parameter set saves the optimal parameters and dictionary. + Prints the optimal parameters and writes the optimal dictionary to the output file. + Supports multithreading if \fBzstd\fR is compiled with threading support. + + The parameter \fIk\fR is more sensitve than \fId\fR, and is faster to optimize over. + Suggested use is to run with a \fIsteps\fR <= 32 with neither \fIk\fR nor \fId\fR set. + Once it completes, use the value of \fId\fR it selects with a higher \fIsteps\fR (in the range [256, 1024]). + \fBzstd --train --optimize-cover FILEs + \fBzstd --train --optimize-cover=d=d,steps=512 FILEs +.TP + +.SH BENCHMARK +.TP +.B \-b# + benchmark file(s) using compression level # +.TP +.B \-e# + benchmark file(s) using multiple compression levels, from -b# to -e# (included). +.TP +.B \-i# + minimum evaluation time, in seconds (default : 3s), benchmark mode only +.TP +.B \-B# + cut file into independent blocks of size # (default: no block) +.B \--priority=rt + set process priority to real-time + +.SH ADVANCED COMPRESSION OPTIONS +.TP +.B \--zstd[=\fIoptions\fR] +.PD +\fBzstd\fR provides 22 predefined compression levels. The selected or default predefined compression level can be changed with advanced compression options. +The \fIoptions\fR are provided as a comma-separated list. You may specify only the \fIoptions\fR you want to change and the rest will be taken from the selected or default compression level. +The list of available \fIoptions\fR: +.RS + +.TP +.BI strategy= strat +.PD 0 +.TP +.BI strat= strat +.PD +Specify a strategy used by a match finder. +.IP "" +There are 8 strategies numbered from 0 to 7, from faster to stronger: +0=ZSTD_fast, 1=ZSTD_dfast, 2=ZSTD_greedy, 3=ZSTD_lazy, 4=ZSTD_lazy2, 5=ZSTD_btlazy2, 6=ZSTD_btopt, 7=ZSTD_btopt2. +.IP "" + +.TP +.BI windowLog= wlog +.PD 0 +.TP +.BI wlog= wlog +.PD +Specify the maximum number of bits for a match distance. +.IP "" +The higher number of bits increases the chance to find a match what usually improves compression ratio. +It also increases memory requirements for compressor and decompressor. +.IP "" +The minimum \fIwlog\fR is 10 (1 KiB) and the maximum is 25 (32 MiB) for 32-bit compilation and 27 (128 MiB) for 64-bit compilation. +.IP "" + +.TP +.BI hashLog= hlog +.PD 0 +.TP +.BI hlog= hlog +.PD +Specify the maximum number of bits for a hash table. +.IP "" +The bigger hash table causes less collisions what usually make compression faster but requires more memory during compression. +.IP "" +The minimum \fIhlog\fR is 6 (64 B) and the maximum is 25 (32 MiB) for 32-bit compilation and 27 (128 MiB) for 64-bit compilation. + +.TP +.BI chainLog= clog +.PD 0 +.TP +.BI clog= clog +.PD +Specify the maximum number of bits for a hash chain or a binary tree. +.IP "" +The higher number of bits increases the chance to find a match what usually improves compression ratio. +It also slows down compression speed and increases memory requirements for compression. +This option is ignored for the ZSTD_fast strategy. +.IP "" +The minimum \fIclog\fR is 6 (64 B) and the maximum is 26 (64 MiB) for 32-bit compilation and 28 (256 MiB) for 64-bit compilation. +.IP "" + +.TP +.BI searchLog= slog +.PD 0 +.TP +.BI slog= slog +.PD +Specify the maximum number of searches in a hash chain or a binary tree using logarithmic scale. +.IP "" +The bigger number of searches increases the chance to find a match what usually improves compression ratio but decreases compression speed. +.IP "" +The minimum \fIslog\fR is 1 and the maximum is 24 for 32-bit compilation and 26 for 64-bit compilation. +.IP "" + +.TP +.BI searchLength= slen +.PD 0 +.TP +.BI slen= slen +.PD +Specify the minimum searched length of a match in a hash table. +.IP "" +The bigger search length usually decreases compression ratio but improves decompression speed. +.IP "" +The minimum \fIslen\fR is 3 and the maximum is 7. +.IP "" + +.TP +.BI targetLength= tlen +.PD 0 +.TP +.BI tlen= tlen +.PD +Specify the minimum match length that causes a match finder to interrupt searching of better matches. +.IP "" +The bigger minimum match length usually improves compression ratio but decreases compression speed. +This option is used only with ZSTD_btopt and ZSTD_btopt2 strategies. +.IP "" +The minimum \fItlen\fR is 4 and the maximum is 999. +.IP "" + +.PP +.B An example +.br +The following parameters sets advanced compression options to predefined level 19 for files bigger than 256 KB: +.IP "" +\fB--zstd=\fRwindowLog=23,chainLog=23,hashLog=22,searchLog=6,searchLength=3,targetLength=48,strategy=6 + +.SH BUGS +Report bugs at:- https://github.com/facebook/zstd/issues + +.SH AUTHOR +Yann Collet |