The XZ file (.xz, .txz)

An XZ file is one file compressed with the xz program from XZ Utils. Like gzip and bzip2, xz compresses a single file and doesn't do folders. A folder is first bundled with tar, giving a .tar.xz or .txz file, which is the most common way you'll meet the format.

xz uses LZMA2, a refined version of the LZMA method from 7-Zip, wrapped in a container that adds checksums, an index and room for extra filters. It compresses noticeably better than gzip and decompresses far faster than bzip2, at the cost of slow compression. That suits software distribution: compressed once, downloaded and unpacked many times. The GNU project, many Linux distributions' source and package archives, and the Linux kernel's own downloads have used .tar.xz.

Older relatives you might meet: .lzma files are raw LZMA from the earlier LZMA Utils, without the checks or index, and 7z archives use the same LZMA and LZMA2 compression inside a different, multi-file archive format.

LZMA2 works like gzip's Deflate in outline: find repeated byte sequences and replace them with references back to an earlier copy. Two things make it compress better:

• A much larger window. Deflate looks back 32 KB. xz's dictionary is 8 MiB at the default level and 64 MiB at -9, so a repeat far back in a large file still counts.
• Range coding with adaptive probabilities instead of Huffman codes. It can spend a fraction of a bit on a very predictable symbol, where Huffman needs at least one.

The memory needed to decompress depends on the level used to compress, which matters on small devices. Levels -7 to -9 only help with files bigger than their dictionaries (8, 16 and 32 MiB, says the manual); on our 11.7 MB test, -9 saved 44 bytes over -6. Adding -e (extreme) tries harder at the same level for a slightly smaller file.

Filters prepare data before LZMA2. The BCJ filters (x86, ARM, ARM64, RISC-V and others) turn relative jump addresses in machine code into absolute ones, which repeat more, so executables compress better. The delta filter helps with some audio and image data.

A single-threaded xz writes one big block. With threads (-T0 uses every core), xz splits the input into blocks and compresses them in parallel; each block then stores its sizes in its header. Since XZ Utils 5.4, such files can also be decompressed in parallel, and since 5.6 multi-threaded mode is the default.

• Smaller blocks compress slightly worse, because each block starts with an empty dictionary. Our sample-multi-block.xz uses 64 KiB blocks to show the structure; the default block size in threaded mode is three times the dictionary, 24 MiB at -6.
• Random access: thanks to the index, a reader can jump to the block holding a given position without decompressing earlier blocks. Most tools don't offer this, but libraries can.
• Same input, different bytes: single-threaded and multi-threaded xz produce different files from the same input. Both decompress to the same data, but checksums of the .xz files differ, which surprises people building reproducible packages.

Output size is a percentage of the original. ZIP compresses each file separately, so it was measured on the folder, not the tar. Your numbers will differ with other data and hardware, but the pattern is typical:

• xz made a file about 23% smaller than gzip's default and unpacked about two and a half times slower, but it was by far the slowest to compress at the default level.
• bzip2 came out slightly smaller than xz on this text-heavy data, but decompressed three times slower.
• Zstandard at its default level was the fastest at everything, with gzip-like size. At -19 it got close to xz's size and decompressed five times faster than xz. That's why Arch Linux and Fedora switched their packages from xz to Zstandard.
• gzip and ZIP sit in the middle: quick both ways, larger files, and readable everywhere.

On 29 March 2024, Andres Freund, a PostgreSQL developer at Microsoft, reported that XZ Utils versions 5.6.0 and 5.6.1 contained a deliberately hidden backdoor. It was rated CVSS 10.0, the maximum. He found it because SSH logins on a Debian test machine were using noticeably more CPU than before, and the memory checker Valgrind reported errors.

• Who: a contributor using the name Jia Tan, who had spent more than two years building trust in the project, with help from other accounts pressing the original maintainer, Lasse Collin, to share control.
• How it was hidden: the malicious code sat in two binary "test files" in the Git repository. A modified build script, build-to-host.m4, was only in the release tarballs, not in Git, so reading the source on GitHub showed nothing wrong.
• What it did: when built as a Debian or RPM package on x86-64 Linux with glibc, liblzma was altered to interfere with the OpenSSH server. Some distributions link sshd to libsystemd, which loads liblzma. Someone holding the attacker's private key could then run commands on the server remotely.
• Who got it: mainly development and rolling releases, such as Debian unstable, Fedora Rawhide and the Fedora 40 beta, openSUSE Tumbleweed and Kali Linux. Stable releases of the major distributions weren't affected. Distributions rolled back to 5.4.x within days, and the project released a clean 5.6.2 in May 2024.

• File format not recognized: the file doesn't start with the xz magic. It may be an old .lzma file (xz --format=lzma -d or unlzma handles those), or something else with the wrong name.
• Memory usage limit reached: a memory limit is set (with --memlimit or the XZ_DEFAULTS variable) below what the file needs. xz -lvv file.xz shows the memory needed; raise the limit for that file.
• Unexpected end of input: the file was cut short. With multi-block files, xz -dc still outputs the complete blocks before the break.
• xz deleted the original. Like gzip, xz replaces its input. Use -k to keep it.
• Compression is very slow: that's xz at -6 or above. Use -T0 for all cores, a lower level such as -3, or Zstandard if speed matters more than size.
• A plain .xz won't open on Windows or Mac by double-click: use 7-Zip on Windows. On macOS, tar -xf handles .tar.xz, but a single-file .xz needs xz from Homebrew or an app such as Keka or The Unarchiver.

• Linux: unxz file.xz (or xz -d), xz -k file to compress and keep the original, xzcat and xzless to read without unpacking, xz -l to list sizes and ratio. For tarballs, tar -xf file.tar.xz and tar -cJf archive.tar.xz folder/.
• macOS: tar -xf file.tar.xz in Terminal works out of the box. For plain .xz files, install XZ Utils with Homebrew (brew install xz).
• Windows: recent Windows 11 opens .tar.xz and .txz archives in File Explorer. Otherwise use 7-Zip, which reads and writes .xz.
• Programming: Python's lzma module, liblzma for C, Go and Rust libraries, and Apache Commons Compress with XZ for Java.
• Convert: xzcat file.tar.xz | gzip > file.tar.gz, or xzcat file.tar.xz | zstd -19 > file.tar.zst. The tar inside stays the same.