Regressive JPEGs: When “Later” Scans Paint Over “Earlier” Pixels

Most of the time, a progressive JPEG feels polite.

You start downloading an image, and the browser gives you a quick, blurry preview. Then each additional chunk of data refines the picture: details sharpen, colors settle, and eventually you get the full-quality image. It’s the same idea as watching a photo come into focus as your camera shutter stays open longer.

But what if the file behaved like the opposite of refinement? What if the later bytes didn’t just add information, but actively rewrote what was already shown?

That’s the weird little idea behind regressive JPEGs: crafting a JPEG so that subsequent scans can cause a decoder to “switch frames” while it’s still receiving one HTTP response.

So what is a regressive JPEG, exactly?

A regressive JPEG is a JPEG file that’s constructed so that, as additional progressive JPEG scans arrive, they effectively correspond to different images rather than higher quality of the same image. Because many decoders incrementally render each scan, the displayed pixels appear to jump backward and forward between states.

A quick refresher: JPEG scans and why progressive images can display early

JPEG is built on block-based frequency transforms. Each 8×8 block is converted into DCT coefficients (DCT stands for Discrete Cosine Transform, which turns an image patch into frequency components). In an 8×8 block you get 64 coefficients: one “low-frequency-ish” one called DC and the remaining ones called AC (alternating frequencies around it).

A progressive JPEG doesn’t store all coefficients in one go. Instead, it divides the entropy-coded stream into multiple segments called scans. Each scan includes only some coefficient information, and decoders can show an intermediate image after receiving just the first scan(s). The goal is faster “first view” on slow networks. (help.accusoft.com)

To decide what each scan contains, progressive JPEGs use parameters in the scan header:

• Spectral selection: chooses coefficient index ranges using Ss (start) and Se (end). The possible indices run from 0 to 63; 0 is the DC coefficient; 1..63 are AC coefficients. (help.accusoft.com)
• Successive approximation: chooses precision refinement for coefficients using bit ranges (HBit and LBit). This controls which bits of a coefficient appear in a given scan. (help.accusoft.com)

In other words, a progressive JPEG is a script: “Here are the bits that make the image look like version A; next scan adds bits for version B; after scan N you have the final image.” (help.accusoft.com)

The key constraint: progressive scans aren’t free-form

The concept sounds simple: concatenate multiple JPEG images and let the decoder “keep going.”

But progressive JPEGs are constrained in how scans may be defined.

A big one: the DC coefficient cannot be encoded in the same scan as AC coefficients. Put differently, the rules require that DC-only scans are separate from scans that include AC information. (help.accusoft.com)

Also, decoders expect the progressive order to make sense per component. One scan might cover DC for multiple components, while later scans handle AC for a single component at a time. (help.accusoft.com)

These constraints don’t stop regressive JPEGs outright, but they shape what kinds of “frame switching” are possible.

How regressive JPEGs exploit a decoder’s incremental rendering

Here’s the intuition that makes everything click.

A progressive decoder doesn’t wait for the end of the file. When it finishes parsing a scan, it may update a partially decoded image buffer and then run rendering steps (color conversion, inverse transforms, etc.) for the pixels it has enough data to reconstruct.

If the scan boundaries in the JPEG bitstream line up with different “versions” of image content, the visible output jumps.

That means the regressive construction needs to satisfy two conditions:

1. Multiple images must share the same overall JPEG geometry: same sampling factors, same image dimensions, and compatible quantization/Huffman behavior so that later scan data can be parsed at all.
2. Scan data must correspond to different frames in a way the decoder accepts.

A straightforward (and very memey) way to build this is:

• Take several progressive JPEGs of the same size.
• Strip out the wrapper markers that would make each one a separate standalone file (start/end of image and start/end of frame).
• Concatenate the remaining entropy-coded scan segments so they form a single larger progressive stream.

The result is still “one JPEG” to the decoder. But as it keeps reading scans, the new scan bytes can correspond to the next image’s coefficient data. If the decoder renders after each scan, the browser appears to cycle frames.

This is less like traditional “video inside a file” and more like “scan-level time slicing.” Network delay becomes the clock.

What goes wrong in practice: decoders bail out after some number of scans

Even if the bitstream is valid, decoders tend to be defensive.

Progressive JPEGs can include many scans, and each scan can trigger work. That makes it a potential target for denial-of-service behavior if an attacker sends a JPEG with an enormous scan count and incomplete data. Many tools therefore implement guardrails.

For example, jpegtran (a libjpeg-turbo tool for lossless transformations) includes a -maxscans option that aborts if an input image contains more than N scans, explicitly motivated by denial-of-service risks from malicious progressive structures and “EOB runs” (runs of zeros encoded efficiently). (man.archlinux.org)

Browsers and other decoders often use similar ideas: they may cap the number of scans they’ll process/render for safety or performance reasons.

That’s why naive regressive JPEGs tend to work for a limited number of “frames” and then freeze or stop showing updates.

Why “baseline progressive tricks” hit an AC/DC wall

A tempting workaround is: use baseline JPEGs so there’s only one scan per image.

But baseline and progressive behave differently, and many decoders stop after the first scan when treating the file as baseline-style content.

The better attempt is: create the smallest possible progressive image so each “frame” consumes only a tiny scan budget.

Here’s the snag: progressive JPEGs can’t mix DC and AC inside a single scan. (help.accusoft.com)

So the smallest compliant progressive scan is effectively DC-only.

DC-only means only coefficient index 0 is present (no AC refinement). Visually, that’s not a full-resolution photo. Because DCT runs on 8×8 blocks, DC-only reconstruction yields an image where each 8×8 block shares the same approximation from its DC term, producing a blocky “chunked” look.

And yet… regressive construction doesn’t need each intermediate state to be pretty. It needs each intermediate state to be different enough that the sequence feels like motion.

Building a “DC-only frame” pipeline (and why it becomes a video surrogate)

In a regressive setup, every “frame” corresponds to one progressive scan chunk.

To maximize how many frames can fit before a decoder gives up, each frame should take the smallest possible progressive footprint. DC-only progressive frames are one way to do that.

With a scan script approach, tools like jpegtran can emit progressive JPEGs using a scan specification file. The -scans file option selects a scan script, which is exactly how you’d request DC-only in a single-scan progressive output. (man.archlinux.org)

So the pipeline becomes conceptually:

• Generate DC-only progressive JPEGs for each frame of a sequence.
• Concatenate their scan payloads so the decoder reads “frame 0 scan, frame 1 scan, frame 2 scan …” as it streams through one big JPEG.

No timing metadata exists inside JPEG in the way a real video codec has. The “playback rate” is whatever timing naturally arises from network chunking and scan processing speed. That makes it chaotic, but also kind of magical.

Why progressive JPEG “precision refinement” matters for the illusion

One reason progressive JPEGs are so good at incremental display is that each scan can either:

• introduce new coefficient ranges (spectral selection), or
• increase coefficient precision (successive approximation).

The scan header controls both. (help.accusoft.com)

In a regressive JPEG, the illusion depends on the fact that decoding updates an internal coefficient buffer and may redraw based on what’s currently present.

Normal progressive images exploit this by making each scan a refinement toward the same final picture. Regressive JPEGs exploit the same machinery by making each scan correspond to a different target.

It’s not that JPEG itself understands animation. It’s that decoders often behave like an animation player when they decide to re-render after each scan.

The fun conclusion: “video” from an entropy stream, not a media format

Regressive JPEGs are an odd corner of the JPEG spec and the decoder implementations around it. They take something designed for progressive rendering and stretch it into scan-level state switching.

And the most important lesson is practical: when you’re working with progressive formats, you’re not only dealing with compression. You’re dealing with control flow in the decoder, especially how it chooses when to render.

That’s why the idea works at all.

A standard progressive JPEG rearranges data into scans so a decoder can display a low-quality image quickly and improve as more scans arrive. (bitmiracle.github.io)

A regressive JPEG weaponizes the same feature: later scan data can be prepared to make what was “already drawn” look like something else.

It’s not a new media format. It’s a stunt built out of parsing rules, incremental rendering, and the uncomfortable fact that browsers optimize for user-perceived speed.

Sources (for deeper reading)

• Accusoft ImageGear documentation on progressive JPEG scan structure and restrictions (DC/AC separation, spectral selection, successive approximation).
• libjpeg.net overview of progressive JPEG behavior and incremental display via scan progression.
• libjpeg-turbo / Independent JPEG Group jpegtran manual for -scans and -maxscans safety behavior around scan counts.