Bluesky Protocol Services: Jetstream v2 Network Replay for gapless history
The moment you realize a “live” stream isn’t enough
Picture the first time a feed-ingestion job goes wrong. The WebSocket is flowing, your consumer is parsing JSON events, and everything looks healthy—until you notice you missed a slice of time. Maybe your process restarted, maybe your network hiccuped, maybe the deploy took longer than expected. Either way, “live” delivery gives you momentum, but it doesn’t automatically give you the past.
That gap is exactly what Jetstream v2: Network Replay is designed to close. With it, Bluesky’s open-network infrastructure can serve history archives that you can replay from an earlier point and then merge into the ongoing live stream without an observable gap.
So what changed with Bluesky Protocol Services? Not just branding and docs. The ecosystem now has clearer “service contracts,” a more consistent development experience, and a set of SDKs that try to make the hard parts of streaming less error-prone.
Quick mental model: WebSocket firehose vs. replayable history
Before diving into the new pieces, it helps to name the actors.
- Jetstream is Bluesky’s way for developers to consume data at scale. Practically, it’s a streaming service that emits plain JSON events.
- A WebSocket is a long-lived network connection where the server can push messages to the client as events happen.
- A cursor is a pointer that tells you where you are in the stream. With live streaming, your cursor typically advances as new events arrive.
Historically, Jetstream-style consumption is “tail-first”: you connect and keep going. That’s fantastic for real-time experiences, but it doesn’t automatically solve this recurring problem:
Why is it hard to get records that already happened before your consumer started?
The old approach was often “backfill yourself”: figure out what time range you missed, download data via separate mechanisms, and then cut over to live—hoping you didn’t overlap too much or leave a hole.
Jetstream v2 Network Replay changes the shape of that workflow.
Jetstream v2 Network Replay: “plan, download sealed segments, then tail”
Network Replay introduces an archive pipeline that sits alongside the existing live tail.
At a high level, the workflow looks like this:
- Plan a snapshot by submitting your filters.
- Download replay data as a set of sealed archive pieces (called segments) over plain HTTP.
- Connect to the live WebSocket once you reach the tip of the archive, so your stream continues without a gap.
The important design detail is what “stateless on the server” means.
Stateless server: why it matters for reliability and cost
A stateless server doesn’t keep per-consumer progress in server-side memory. Instead of tracking “where you are” for each subscriber, the system produces archive artifacts based only on the plan request and then lets the client stitch the playback together.
In practice, that means Network Replay can be operated in a more predictable way under load:
- No per-consumer cursor to maintain on the server
- No subscription registration state to clean up
- The archive acts like your buffer
You still manage your cursor on the client side, but you don’t force the server to remember your exact consumption path.
“Snapshot only” mode also exists
Network Replay isn’t just about catching up and tailing. There’s also a snapshot-only variant where you download a point-in-time archive without doing the live WebSocket handoff.
That’s useful for batch analysis, forensics, and data recovery scenarios where “good enough for the present moment” beats “always-on streaming.”
Archive access now requires a token (live tail stays open)
Serving archive data is bandwidth-intensive. To keep the service dependable and cost-controlled, Network Replay archive requests now require a dedicated API token.
The key nuance is operational: the live tail remains open and unauthenticated. In other words, you still get frictionless real-time streaming, while the heavier “download the past” pathway is gated.
From a system-design perspective, this split matches what you’d expect:
- Live streaming: frequent, smaller messages, continuous connection
- Replay/snapshot archives: fewer requests, but potentially huge downloads
Where to connect: Jetstream v2 instances and a clean migration story
Jetstream v2 runs on separate WebSocket endpoints for different regions. Meanwhile, older Jetstream v1 instances continue running, and the live tail behavior is consistent enough that you’re not forced into an immediate migration.
The practical takeaway is that you can adopt replay features without rewriting every existing streaming pipeline at once.
Why SDKs still matter when the wire format is “just JSON”
Jetstream emits JSON, so you might think “an SDK is optional.” That’s true in the strictest sense.
But the moment you write production code, you end up needing a bunch of glue logic:
- reconnecting strategies when WebSockets drop
- deduplication when events reappear
- cursor management and “where am I?” bookkeeping
- decoding events into typed records so you stop indexing into random object shapes
That’s exactly why the new Jetstream SDKs exist.
Jetstream SDK in TypeScript: typed events with async iteration
The TypeScript SDK centers the experience around a Jetstream object and then uses for await to consume events in order.
The core idea is readable: construct a client, specify your filters (for example, a collection like app.bsky.feed.post), then iterate decoded events.
Here’s the shape of it, with environment variables standing in for actual host configuration:
import { Jetstream } from '@bsky/jetstream'
import { app } from '@bsky/sdk/lexicons'
const baseUrl = process.env.JETSTREAM_BASE_URL!
const js = new Jetstream(baseUrl)
for await (const evt of js.live({ collections: [app.bsky.feed.post] })) {
if (evt.kind === 'commit' && evt.commit.operation === 'create') {
console.log(evt.commit.collection, evt.commit.record.text)
}
}
Two beginner-friendly notes:
for awaitmeans the loop can consume an asynchronous stream of events without manually wiring callbacks.- The SDK is doing the “typed decoding” part for you, so you can access fields like
evt.commit.record.textwithout guessing what the JSON looks like.
The Bluesky TypeScript SDK is rebased on lex (typed end-to-end)
The other major shift under the Bluesky Protocol Services umbrella is the rebuilt Bluesky TypeScript SDK.
To understand what “rebased on lex” means, you need one new concept.
What is lex (Lexicons as a code generator)?
Lexicons are schema definitions: structured descriptions of what records look like in the AT Protocol ecosystem. A code generator can read those schemas and produce strongly typed clients and validators.
@atproto/lex is the lex tooling that builds a typed workflow end-to-end: from protocol layer definitions all the way to specific app.bsky records.
That’s why examples on the new docs can stay consistent: the type system is anchored in the lexicon source of truth.
Why this reduces “legacy technical debt”
If a project evolves over time, older SDK helpers can accumulate. Those helpers tend to get deprecated, but code examples can still keep recommending them—especially across AI-generated snippets.
By rebuilding the Bluesky TypeScript SDK on top of the lex toolchain, the ecosystem can move away from legacy code paths and toward a consistent “one way to do it” story.
And importantly for existing developers: if you’re still using the older @atproto/api-style code, it can continue to work, while the new guides act as a migration reference.
Endpoint reference: Jetstream v2 surface area is now documented together
A common pain with infrastructure services is discovery: you have to hunt across multiple sites to learn what exists.
Under Bluesky Protocol Services, the HTTP reference documentation has been updated so that Jetstream v2’s HTTP and WebSocket endpoints live in one place.
That includes the replay planning and archive distribution mechanisms (the planning step and the segment listing/downloading workflow), alongside the WebSocket endpoints used for live tails.
For developers, the “single reference” aspect matters because replay workflows often involve two phases:
- HTTP calls to fetch archive artifacts
- WebSocket calls to continue consuming as events happen
When those are split across unrelated docs sites, it’s easy to miss a method name, a request body shape, or the data model used in the next step.
A replay-focused implementation sketch (conceptual, not copy-paste)
Even without tying yourself to a specific SDK method, the replay story stays consistent:
- Ask the server for a plan based on your filters and desired historical window.
- Use the plan to enumerate which sealed segments exist.
- Download the segments over HTTP and process them locally.
- When you reach the planned “tip,” switch to the live WebSocket connection.
The tricky part—deduplication and cutover correctness—is exactly where typed event decoding and cursor-aware processing help.
In other words, the server can be stateless and still support gapless transitions because the client can treat replay as an ordered buffer, then continue forward with live updates.
What to use first if you’re new
The new doc structure is arranged to match how a developer’s mental model grows:
- First, understand the overall data flow between lexicons, records, and streaming.
- Next, pick the consumption mode: live tail, replay, or snapshot-only.
- Finally, use the SDKs so you spend time building features, not fighting connection edge cases.
Closing thought: replay makes “streaming systems” feel less fragile
Streaming systems always carry a small dread: a restart can turn into a silent data hole.
Jetstream v2 Network Replay attacks that dread at the infrastructure layer by offering a clear, bounded workflow: plan an archive, download sealed segments, then hand off to live streaming at the tip. Pair that with typed SDKs rebased on lexicon schemas, and the overall developer experience becomes more predictable.
It’s not just new branding. It’s an attempt to make the hard parts of distributed data consumption less mysterious—so you can focus on what your app actually does with the posts once they arrive.
Comments (0)
No comments yet. Be the first to respond!
Leave a Comment
Your comment will be visible after review.