PostgreSQL

Postgres LISTEN/NOTIFY Actually Scales (When You Treat It Like a Wake-Up, Not a Queue)

Postgres LISTEN/NOTIFY Actually Scales (When You Treat It Like a Wake-Up, Not a Queue)

The first time LISTEN/NOTIFY felt “magical”

Picture a reader waiting for the next chunk of work: a chat token, an event, a stream message. Polling works, but it’s noisy and wasteful. Poll too slowly and latency feels laggy; poll too fast and the database starts feeling it.

So we reach for Postgres LISTEN/NOTIFY. The vibe is perfect: the writer inserts data, fires a notification, and the reader wakes up immediately.

Why does Postgres LISTEN/NOTIFY get a bad reputation? Because the naïve pattern makes Postgres serialize work at commit time.

And yet: that doesn’t mean LISTEN/NOTIFY can’t scale. It means the shape of your design matters.


A mental model: LISTEN/NOTIFY is “a poke,” not the data

At a high level:

  • LISTEN channel registers your session as waiting on a notification channel.
  • NOTIFY channel, payload sends a notification to every session listening on that channel.
  • Notifications are delivered in commit order, and LISTEN/NOTIFY take effect around transaction commit. (postgresql.org)

The important subtlety is this: Postgres notifications are not the durable source of truth. They’re a wake-up signal. The durable truth should live in a table.

That’s exactly how stream readers should operate. A notification tells the reader “check the table now.” The table tells the reader what actually happened.

This separation turns out to be the lever that makes scaling possible.


The pattern that breaks: NOTIFY on every row insert

The classic “stream with low latency” approach looks like this:

  1. Create a streams table.
  2. Every new chunk is inserted as a new row.
  3. A trigger fires a NOTIFY for each insert.
  4. Readers LISTEN and wake up, then query for unseen rows.

This gives you great latency in development.

But throughput collapses at scale. The database looks calm—CPU, memory, and I/O don’t spike—yet transaction commit speed caps out.

That symptom matches what’s happening inside Postgres async notification internals: adding notifications to the shared queue requires an exclusive lock.


Under the hood: a single commit-ordered notification queue + an exclusive lock

Postgres keeps a shared notification queue so it can deliver notifications in commit order.

The source code comment sums it up: a single queue exists so notifications are received in commit order. (doxygen.postgresql.org)

More importantly for performance, when a transaction has pending notifications, Postgres adds them to that global queue using NotifyQueueLock in LW_EXCLUSIVE mode. (doxygen.postgresql.org)

So if every stream chunk insert triggers its own NOTIFY, you force every transaction to contend on the same exclusive lock during the commit path.

Even worse, this happens exactly where databases are usually trying to help each other: group commit (batching multiple transactions into fewer durable flushes). If your workload insists on serializing the commit path, you lose those efficiencies.

The result is a “looks idle” bottleneck that is really a coordination bottleneck.


Another truth: NOTIFY isn’t meant to survive crashes

One reason LISTEN/NOTIFY is appealing is also why it needs careful application design.

Postgres documentation explains that there is a queue for pending notifications and that NOTIFY ordering rules apply, but notifications are not a crash-proof delivery mechanism in the way a durable job queue is. The async implementation also notes that notifications are not expected to survive database crashes, so there’s no need for WAL support or fsync specifically for the notification mechanism. (doxygen.postgresql.org)

This isn’t a bug; it’s a design choice. It means: if you treat notifications as the data, you’ll eventually lose things.

If you treat notifications as “wake up and re-read,” you can recover safely.


How to make LISTEN/NOTIFY scale: batch the pokes

Here’s the key optimization idea:

  • Notifications don’t have to be 1:1 with stream rows.
  • Notifications only need to prompt readers to check the table.
  • Therefore, you can buffer notifications in memory and send them in batches.

Instead of firing NOTIFY from a trigger on every insert, you change the contract:

  • Writers insert stream chunks into the streams table normally.
  • A separate “notifier” component (often inside the app) buffers which stream(s) have new data.
  • Every so often (or after N chunks), it sends one NOTIFY per stream (or per group of streams) in a single batched transaction.

Why this helps: NotifyQueueLock contention drops because far fewer transactions execute the NOTIFY path. Readers still get low latency because the notifier flushes the wake-up signal frequently enough.

Meanwhile, writers keep the normal transaction shape that Postgres handles well, including group commit behavior.


But buffering introduces a new failure mode

Buffering means a crash can discard the buffered wake-up signals.

If a notification never arrives, the reader might sleep forever—unless the system has a fallback.

So the scalable design includes a safety net:

  • Readers wake up on notifications and immediately read the table.
  • Readers also periodically poll for progress (a “heartbeat” check).

That polling interval can be tuned to keep interactive latency tight while guaranteeing eventual delivery even when notifications are lost.

This is the same pattern used in many production systems: notifications are the fast path; polling is the correctness backstop.


A concrete design that stays durable

A workable stream schema often looks like this (simplified):

CREATE TABLE stream_chunks (
  stream_id     bigint not null,
  chunk_id      bigint generated always as identity,
  payload       text not null,
  created_at    timestamptz default now(),
  primary key (stream_id, chunk_id)
);

-- Optional: track consumer progress
CREATE TABLE stream_consumers (
  stream_id bigint not null,
  consumer  text not null,
  last_chunk_id bigint not null default 0,
  primary key (stream_id, consumer)
);

Writer behavior

  • Insert chunk rows in a regular transaction.
  • Record “this stream has new data” in an in-memory buffer.
  • A background flusher sends NOTIFY for that stream in batches.

Reader behavior

  • LISTEN on a channel like stream_updates.
  • When woken, query:
SELECT chunk_id, payload
FROM stream_chunks
WHERE stream_id = $1
  AND chunk_id > $last_chunk_id
ORDER BY chunk_id
LIMIT $batch;
  • Commit the updated last_chunk_id once processed.

Fallback polling

  • Every T milliseconds, do the same “read unseen rows” query even if no notification arrived.

This makes the stream durable by construction: the table is truth, notifications are a nudge.


What about PostgreSQL improvements over time?

Postgres has been improving NOTIFY performance for large listener sets. For example, PostgreSQL 19 includes an enhancement that improves NOTIFY so it only wakes backends that are listening to specified notifications (reducing unnecessary wakeups). (postgresql.org)

That’s helpful for some workloads, but it doesn’t change the broader scaling lesson: a trigger-driven NOTIFY per row can hammer the commit path.

The batching approach still matters because it reduces the number of transactions that pay the expensive coordination cost.


Conclusion: scale comes from treating the lock like a cost

Postgres LISTEN/NOTIFY scales when the system treats notifications like they are meant to be: lightweight wake-ups, not a durable queue.

Once the design stops issuing NOTIFY on every single write, and instead batches wake-ups, the contention that throttles throughput drops dramatically. Readers still get millisecond-level latency because they re-read the durable table immediately after being poked.

The “gotcha” is real. The fix is also real. The trick is remembering that the database can be both fast and correct—if the architecture respects the cost centers inside the engine.

ahsan

ahsan

Hello! I am Mr Ahsan, the writer of the Website. I am from Netherland. I like to write about technology and the news around it.

Comments (0)

No comments yet. Be the first to respond!

Leave a Comment

Your comment will be visible after review.