bzip3: A Practical Guide to Modern File Compression

Picture a directory full of source archives: each file is a little suitcase, and your disk is already out of shelf space. The older bzip2 compressor still squeezes well, xz is known for high compression ratios, while Zstandard (zstd) and LZ4 usually favor speed. bzip3 takes a different route: it combines a prediction-heavy pipeline with independent compression blocks, so it can chase a small output without making every operation single-threaded.

bzip3 is a command-line compressor and a library that C programs can link to, not a replacement for an archive tool such as tar, a utility that bundles files together. It is designed especially for text and source code. At the time of writing, its latest tagged release is 1.5.3, published August 13, 2025, while newer maintenance work continues in unreleased code. What is bzip3, and when does it make sense beside bzip2, xz, or zstd? The answer starts with understanding what happens inside one block.

bzip3 is a compressor, not an archive

An archive collects several files into one container; a compressor transforms bytes into a smaller representation. bzip3 normally reads one file or a byte stream and writes a .bz3 file. Its format is not compatible with bzip2, despite the familiar name.

A first test looks like this:

printf 'The same sentence appears again and again.\n%.0s' {1..1000} > notes.txt
bzip3 -e notes.txt
bzip3 -dc notes.txt.bz3 > restored.txt

The -e flag makes compression explicit, although compression is the default action. The -d flag decodes, while -c sends the result to standard output, the stream a shell can pipe into another command. bzip3 keeps the original file unless told otherwise, so this example writes the restored data to a different filename instead of trying to overwrite notes.txt.

The compression pipeline does several jobs

Compression here is less like one clever trick and more like a workshop with several stations. Each stage changes the data in a way that helps the next one:

1. Lempel-Ziv Prediction (LZP) looks at the bytes around the current position and predicts what should come next. Repeated phrases can then be represented as matches instead of being copied in full.
2. Run-Length Encoding (RLE) replaces a run of identical values with a value and a count. A long stretch of the same byte becomes much cheaper to store.
3. The Burrows-Wheeler Transform (BWT) rearranges a block so similar bytes sit near one another. BWT does not shrink the data by itself; it makes patterns easier for later coding stages to see.
4. An entropy coder turns statistical predictability into fewer bits. bzip3 combines arithmetic coding with a context-mixing model, which blends predictions from different recent contexts.

That combination explains why bzip3 tends to shine on text, programming-language source, and other data with repeated structure. Already-compressed files such as JPEG images, video, or another compressed archive offer far fewer patterns to find.

Blocks are the part you feel in practice

A block is a self-contained piece of the input. bzip3 can work on separate blocks at the same time, using a worker thread, an independent execution lane for a piece of work, for each active job.

At the command line, -b chooses the block size in mebibytes (MiB, 1,048,576 bytes). The default is 16 MiB, and the command-line range is 1 to 511 MiB. -j chooses the number of worker threads:

bzip3 -e -b 64 -j 4 dataset.tar

This asks for 64 MiB blocks and four workers. The trade-off is memory. A useful planning estimate is roughly six times the block size for compression or decompression, so a 64 MiB setting can imply about 384 MiB of working memory before extra overhead; parallel workers can push the total higher. Larger blocks often improve the compression ratio, but the benefit falls off quickly. More subtly, the decompressor must support the block size chosen when the file was created.

For a laptop, the 16 MiB default is a sensible baseline. Increase it when storage matters more than memory, and increase -j only when the machine has both spare CPU cores and enough memory.

A .bz3 file has a small, deliberate structure

A command-line bzip3 file begins with the five-byte signature BZ3v1, followed by the maximum decompressed block size. The file header is nine bytes. The library's frame format, a self-contained buffer format used by its API calls, adds a block-count field and uses a 13-byte header. After that come chunks, each recording compressed and original sizes.

Tiny blocks under 64 bytes are stored literally instead of being forced through the full compressor. Larger blocks carry a CRC32 checksum, a checksum used to detect accidental corruption, along with a Burrows-Wheeler index and model flags that record whether LZP or RLE helped. The format also keeps random, incompressible data from growing much: expansion is generally under 0.8%.

Those details explain why bzip3 can validate output, why block size affects memory during decompression, and why different blocks can choose different filters.

Use bzip3 with tar, pipes, or libbz3

To compress a directory, create an archive first with tar, then compress that archive:

tar -cf project.tar project/
bzip3 -e project.tar
bzip3 -dc project.tar.bz3 | tar -xf -

This keeps the responsibilities clear: tar groups the files, and bzip3 performs the file compression. The same standard-output behavior makes bzip3 useful in shell pipelines, and bz3cat is available as a shorthand for common decode-to-output operations.

Applications can connect to libbz3, the project's compression library. Its high-level application programming interface (API) exposes functions such as bz3_bound, bz3_compress, and bz3_decompress. A minimal compression call looks like this after the input and output buffers have been prepared:

size_t output_size = bz3_bound(input_size);
uint8_t *output = malloc(output_size);

int rc = bz3_compress(
 16 * 1024 * 1024, input, output, input_size, &output_size
);

The high-level call does not create parallel work by itself. Programs that need more control can use lower-level block functions, while bz3_min_memory_needed helps estimate the memory budget. The library is released under the GNU Lesser General Public License version 3 (LGPLv3), which matters when linking it into a larger product.

Where bzip3 fits

bzip3 is a strong candidate for source snapshots, text-heavy datasets, and long-lived backups where every saved byte matters. On a large Perl-source corpus, large bzip3 blocks produce very aggressive size results, but that is one workload, one machine, and one set of settings rather than a universal ranking.

For tiny files, memory-constrained containers, or latency-sensitive network traffic, a faster compressor such as zstd or gzip may be a better operational choice. bzip3 is newer, so some older tools and hosted services will not recognize .bz3 without an added package.

The useful mental model is this: bzip3 is a block-oriented compressor that layers prediction, run detection, data reordering, and statistical coding, then lets several blocks move through the machine at once. Start with the defaults, measure your own files, and treat block size as both a compression setting and a memory commitment.