MediumSystem Design11 min read

What is the publish-subscribe pattern?

Q: What is the publish-subscribe pattern?

**Target Audience:** Beginners to system design who have written services that call each other; no messaging background assumed. **Difficulty:** Medium **Verified with:** Node 22.22 and Python 3.12 (both print identical output). Redis and Kafka behaviour is cited from their documentation. > **How to read this doc:** Each idea is explained in plain language first, then tagged with `📌 Interview term:` so you also learn the words an interviewer expects. Behaviour claims come from the example; the **Verified** section quotes the full output. ## 1. Why This Even Matters — A Story First A bakery takes an order. Without pub/sub, the cashier has to walk to the kitchen and tell the baker, then phone the delivery driver, then update the loyalty-points book, then tell the accountant. Every new department means the cashier learns a new step, and if the accountant is at lunch, the cashier waits. With pub/sub, the cashier pins the order slip to a board marked **Orders** and goes back to the que

872 views

16 of 106

The problem

Question presented to candidate: "Explain the publish-subscribe pattern. How is it different from just calling the other services, or from a work queue? And what changes when the broker is Redis Pub/Sub versus something like Kafka?"

What a strong answer should cover:

Publishers send messages to a topic without knowing who listens; the broker delivers a copy to every subscriber (fan-out), so producers and consumers are decoupled in identity, number and often time
The contrast with a point-to-point queue, where each message goes to one consumer, and how consumer groups combine both (fan-out across groups, load sharing within a group)
Ephemeral subscriptions (Redis Pub/Sub: only connected subscribers receive, at-most-once) versus durable ones (a retained log with per-subscriber offsets: late or offline subscribers catch up)
Delivery guarantees: durable systems are usually at-least-once, so subscribers must be idempotent; ordering is typically per partition or per key, not global
Costs: harder to trace a flow, schema evolution of events, and no reply path — pub/sub is for notifications, not for questions that need an answer

Code / implementation expected: Optional — the concepts matter most; the example builds an ephemeral channel and a durable partitioned log side by side.

pub-sub

Try it yourself

Sketch your approach before you read the solution. That is what the interview tests.

Stuck? Get a hint.

A short nudge toward the approach. It will not give the answer away.

Solution

Target Audience: Beginners to system design who have written services that call each other; no messaging background assumed. Difficulty: Medium Verified with: Node 22.22 and Python 3.12 (both print identical output). Redis and Kafka behaviour is cited from their documentation.

How to read this doc: Each idea is explained in plain language first, then tagged with 📌 Interview term: so you also learn the words an interviewer expects. Behaviour claims come from the example; the Verified section quotes the full output.

1. Why This Even Matters — A Story First

A bakery takes an order. Without pub/sub, the cashier has to walk to the kitchen and tell the baker, then phone the delivery driver, then update the loyalty-points book, then tell the accountant. Every new department means the cashier learns a new step, and if the accountant is at lunch, the cashier waits.

The solution is hidden so you can try first

11 min read · 9 sections

Read the code

Pub/sub two ways: an ephemeral channel vs a durable log with consumer groups and redelivery

Open in playground

// Publish-subscribe two ways, in one process: an ephemeral channel (Redis Pub/Sub style)
// and a durable partitioned log with consumer groups (Kafka style). No real broker.

// ---- 1. Ephemeral: deliver to whoever is subscribed right now, keep nothing ----
const channels = {};
const subscribe = (ch, name, inbox) => ((channels[ch] ||= {})[name] = inbox);
const unsubscribe = (ch, name) => delete channels[ch][name];
const publishNow = (ch, msg) => {
  const subs = Object.values(channels[ch] || {});
  subs.forEach((inbox) => inbox.push(msg));
  return subs.length; // like Redis PUBLISH, the reply is how many subscribers got it
};
const d1 = [], d2 = [];
console.log("Part 1: ephemeral channel 'prices'");
console.log("  publish p0 -> delivered to " + publishNow("prices", "p0") + " subscribers (lost)");
subscribe("prices", "dash-1", d1);
publishNow("prices", "p1"); publishNow("prices", "p2");
subscribe("prices", "dash-2", d2);
publishNow("prices", "p3");
unsubscribe("prices", "dash-2"); // network blip
publishNow("prices", "p4");
subscribe("prices", "dash-2", d2);
console.log("  dash-1 got " + d1.join(",") + "; dash-2 got " + d2.join(",") + " (joined late, missed p4 while away)");

// ---- 2. Durable log: every group gets every message; members of a group split partitions ----
const PARTITIONS = 2;
const log = Array.from({ length: PARTITIONS }, () => []);
const partitionOf = (key) => [...key].reduce((s, c) => s + c.charCodeAt(0), 0) % PARTITIONS;
const append = (key, value) => log[partitionOf(key)].push({ key, value });
const committed = {}; // group -> [offset per partition]
function poll(group, partitions, handler, crashAfter = -1) {
  const offs = (committed[group] ||= Array(PARTITIONS).fill(0));
  let handled = 0;
  for (const p of partitions) {
    while (offs[p] < log[p].length) {
      handler(log[p][offs[p]], p, offs[p]);
      handled++;
      if (handled === crashAfter) return "crashed before commit";
      offs[p]++; // commit after processing: at-least-once
    }
  }
  return "ok";
}
const orders = [["ann", "o1", 30], ["bea", "o2", 15], ["ann", "o3", 20], ["cy", "o4", 50], ["bea", "o5", 25], ["ann", "o6", 10]];
orders.slice(0, 4).forEach(([k, id, amt]) => append(k, { id, amt }));
console.log("\nPart 2: durable topic 'orders', " + PARTITIONS + " partitions keyed by customer");
for (let p = 0; p < PARTITIONS; p++) console.log("  partition " + p + ": " + log[p].map((m) => m.key + ":" + m.value.id).join(" "));
const emailSeen = { "email-a": [], "email-b": [] };
poll("email", [0], (m) => emailSeen["email-a"].push(m.value.id));
poll("email", [1], (m) => emailSeen["email-b"].push(m.value.id));
console.log("  group email, member a owns p0: " + emailSeen["email-a"].join(",") + "; member b owns p1: " + emailSeen["email-b"].join(","));
console.log("  group analytics is offline; nothing is lost, its offsets are still " + JSON.stringify(committed.analytics || [0, 0]));
orders.slice(4).forEach(([k, id, amt]) => append(k, { id, amt }));

// ---- 3. At-least-once redelivery and an idempotent subscriber ----
console.log("\nPart 3: analytics comes back, crashes once before committing, restarts");
let naive = 0, safe = 0;
const processed = new Set();
const handle = (m, p, off) => {
  naive += m.value.amt;
  if (!processed.has(m.value.id)) { processed.add(m.value.id); safe += m.value.amt; }
  console.log("  p" + p + "@" + off + " " + m.value.id + " amt " + m.value.amt);
};
console.log("  run 1: " + poll("analytics", [0, 1], handle, 3));
console.log("  run 2: " + poll("analytics", [0, 1], handle));
const truth = orders.reduce((s, o) => s + o[2], 0);
console.log("  revenue: true " + truth + ", naive counter " + naive + ", idempotent counter " + safe);
console.log("  per-key order for ann: " + log[partitionOf("ann")].filter((m) => m.key === "ann").map((m) => m.value.id).join(" -> "));

Discussion

Other approaches, follow-ups and how it went in real interviews.

No comments yet. Share how you would answer it.

Question 16 of 106 in System design

Keep your momentum going.

Mark this question solved to track your progress, then take the next one while it is fresh.

All System design questions

Up nextMedium→

Token bucket vs sliding window vs fixed window — which rate limiter when?

Continue