Zstandard

zstdFacebook Zstandard
Zstandard (or zstd) is a lossless data compression algorithm developed by Yann Collet at Facebook.wikipedia
50 Related Articles

Asymmetric numeral systems

Finite State EntropyrANSANS
Zstandard combines a dictionary-matching stage (LZ77) with a large search window and a fast entropy coding stage, using both Finite State Entropy (a fast tabled version of ANS, tANS, used for entries in the Sequences section), and Huffman coding (used for entries in the Literals section).
Among others, ANS is used in the Facebook Zstandard compressor (also used e.g. in Linux kernel, Android operating system, was published as RFC 8478 for MIME and HTTP ), in the Apple LZFSE compressor, Google Draco 3D compressor and PIK image compressor, in CRAM DNA compressor from SAMtools utilities, Dropbox DivANS compressor, and in JPEG XL next generation image compression standard.

Btrfs

Btrfs file systemBTRFS_*
The Linux kernel has included Zstandard since November 2017 (version 4.14) as a compression method for the btrfs and squashfs filesystems.

HTTP compression

compressedcompressingcompression
In 2018 the algorithm was published as RFC 8478, which also defines an associated media type "application/zstd", filename extension "zst", and HTTP content encoding "zstd".

SquashFS

Squash FS
The Linux kernel has included Zstandard since November 2017 (version 4.14) as a compression method for the btrfs and squashfs filesystems.
Support for Zstandard was added in Linux Kernel 4.14.

Arch Linux

pacmanArchArchLinux
Arch Linux added support for zstd as a package compression method in October 2019 with the release of the pacman 5.2 package manager.
It uses binary packages in the format or (for zstd compression), with placed before this to indicate that it is a pacman package (giving or ).

LZ4 (compression algorithm)

LZ4LZ4 compressionLZ4 compression algorithm
LZ4 is also implemented in newer Zstd archiver by Yann Collet.

Data compression

compressionvideo compressioncompressed
Zstandard (or zstd) is a lossless data compression algorithm developed by Yann Collet at Facebook.

Facebook

Facebook LiveFacebook.comFacebook, Inc.
Zstandard (or zstd) is a lossless data compression algorithm developed by Yann Collet at Facebook.

C (programming language)

CC programming languageC language
Zstd is the reference implementation in C.

Free software

freefree-softwarefreely
Version 1 of this implementation was released as free software on 2016-08-31.

DEFLATE

deflateddeflate-decodeDeflate64
Zstandard was designed to give a compression ratio comparable to that of the DEFLATE algorithm (developed in 1991 and used in the original ZIP and gzip programs), but faster, especially for decompression.

Zip (file format)

ZIPZIP file.zip
Zstandard was designed to give a compression ratio comparable to that of the DEFLATE algorithm (developed in 1991 and used in the original ZIP and gzip programs), but faster, especially for decompression.

Gzip

gunzipzcat.tgz
Zstandard was designed to give a compression ratio comparable to that of the DEFLATE algorithm (developed in 1991 and used in the original ZIP and gzip programs), but faster, especially for decompression.

Rzip

lrzip
Starting from version 1.3.2 (October 2017), zstd optionally implements very long range search and deduplication similar to rzip or lrzip.

Lempel–Ziv–Markov chain algorithm

LZMALZMA2Lempel-Ziv-Markov chain algorithm
Zstd at its maximum compression level gives a compression ratio close to lzma, lzham, and ppmx, and performs better than lza, or bzip2.

Prediction by partial matching

PPMdPPM compression algorithmPPM
lzham, and ppmx, and performs better than lza, or bzip2.

LHA (file format)

LHALZHLHA/LZH
lzham, and ppmx, and performs better than lza, or bzip2.

Bzip2

BZIPbunzip2BZ2
lzham, and ppmx, and performs better than lza, or bzip2.

Pareto efficiency

Pareto efficientPareto optimalPareto optimality
Zstandard reaches the current Pareto frontier, as it decompresses faster than any other currently-available algorithm with similar or better compression ratio.

Log file

log fileslogginglog
In particular, one dictionary can be loaded to process large sets of files with redundancy between files, but not necessarily within each file, e.g., log files.

LZ77 and LZ78

LZ77Lempel-ZivLempel–Ziv
Zstandard combines a dictionary-matching stage (LZ77) with a large search window and a fast entropy coding stage, using both Finite State Entropy (a fast tabled version of ANS, tANS, used for entries in the Sequences section), and Huffman coding (used for entries in the Literals section).

Entropy encoding

entropy codingentropy codedentropy coder
Zstandard combines a dictionary-matching stage (LZ77) with a large search window and a fast entropy coding stage, using both Finite State Entropy (a fast tabled version of ANS, tANS, used for entries in the Sequences section), and Huffman coding (used for entries in the Literals section).

Huffman coding

HuffmanHuffman codeHuffman encoding
Zstandard combines a dictionary-matching stage (LZ77) with a large search window and a fast entropy coding stage, using both Finite State Entropy (a fast tabled version of ANS, tANS, used for entries in the Sequences section), and Huffman coding (used for entries in the Literals section).

Linux kernel

LinuxLinux kernel mainlinekernel
The Linux kernel has included Zstandard since November 2017 (version 4.14) as a compression method for the btrfs and squashfs filesystems.