Skip to the solution
MediumSystem Design16 min read

What is two-phase commit and its drawbacks?

649 views

The problem

Question presented to candidate: "An order has to be written to the orders database, the payments database and the inventory database, and either all three changes happen or none do. Explain how two-phase commit achieves that, walk me through what happens when a participant or the coordinator fails, and tell me why many teams avoid it between microservices."

What a strong answer should cover:

  • The goal is atomic commit across separate databases (participants), driven by a coordinator
  • Phase 1 (prepare / vote): each participant makes its changes durable, keeps its locks and votes YES or NO; a YES is a promise it can no longer break on its own
  • Phase 2 (commit / abort): the coordinator forces its decision to its own log first, then tells everyone; any NO or timeout before the decision means abort
  • The core drawback is blocking: a participant that voted YES and lost the coordinator is in doubt and holds its locks until the coordinator recovers
  • Cost: two round trips to the slowest participant, extra forced log writes, locks held across the network; heuristic decisions by operators break atomicity
  • Alternatives and fixes: replicate the coordinator with consensus (Spanner style), 3PC (not safe under partitions), or avoid distributed transactions with a saga or an outbox

Clarifying questions expected:

  • Are all participants databases we control that support a prepare step (XA, PREPARE TRANSACTION), or are some of them third-party APIs?
  • How long can the affected rows stay locked, and is temporary unavailability acceptable in exchange for atomicity?

Code / implementation expected: Optional โ€” a clear walk-through of both phases and the failure cases; the example simulates the protocol on a fake clock.

  • two-phase-commit

Try it yourself

Sketch your approach before you read the solution. That is what the interview tests.

Stuck? Get a hint.

A short nudge toward the approach. It will not give the answer away.

Solution

The solution is hidden so you can try first

16 min read ยท 12 sections

Read the code

Two-phase commit on a fake clock: happy path, a NO vote, a coordinator crash and a heuristic abort
// Two-phase commit on a fake clock: one coordinator, three participants.
// Each participant is a separate database; "rtt" is the round trip to it in ms.
// A forced log write (fsync) costs 1 ms. No real network or disk is used.
const FSYNC = 1;

function makeParts(votes) {
  return [
    { name: "orders", rtt: 2, vote: votes[0], state: "init" },
    { name: "payments", rtt: 4, vote: votes[1], state: "init" },
    { name: "inventory", rtt: 60, vote: votes[2], state: "init" },
  ];
}

// Phase 1: send PREPARE, each participant forces a PREPARED (or ABORT) record and votes.
function phase1(parts) {
  let votesAt = 0;
  for (const p of parts) {
    p.lockedAt = p.rtt / 2; // PREPARE arrives; rows stay locked from here on
    p.state = p.vote === "YES" ? "prepared" : "aborted";
    if (p.vote === "NO") p.releasedAt = p.lockedAt + FSYNC; // a NO voter rolls back at once
    votesAt = Math.max(votesAt, p.rtt + FSYNC);
  }
  return votesAt;
}

// Phase 2: coordinator forces the decision, then tells everyone.
function phase2(parts, decision, decidedAt) {
  for (const p of parts) {
    if (p.releasedAt !== undefined) continue;
    if (p.state === "prepared") p.state = decision === "COMMIT" ? "committed" : "aborted";
    p.releasedAt = decidedAt + p.rtt / 2 + FSYNC;
  }
}

function show(parts) {
  for (const p of parts) {
    const held = p.releasedAt === undefined ? "still locked" : `locks held ${p.releasedAt - p.lockedAt} ms`;
    console.log(`  ${p.name.padEnd(9)} rtt ${String(p.rtt).padStart(2)} ms  vote ${p.vote.padEnd(3)} -> ${p.state.toUpperCase().padEnd(9)} ${held}`);
  }
}

console.log("== 1. Happy path: all three vote YES");
let parts = makeParts(["YES", "YES", "YES"]);
let votesAt = phase1(parts);
let decidedAt = votesAt + FSYNC;
phase2(parts, "COMMIT", decidedAt);
show(parts);
console.log(`  votes in at t=${votesAt} ms, COMMIT forced at t=${decidedAt} ms, client told OK at t=${decidedAt} ms`);
console.log(`  messages ${parts.length * 4}, forced log writes ${parts.length * 2 + 1} (a single-database commit: 1 write, ${FSYNC} ms)`);

console.log("\n== 2. One NO vote: payments cannot reserve funds");
parts = makeParts(["YES", "NO", "YES"]);
phase1(parts);
const noAt = parts[1].rtt + FSYNC; // the first NO is enough to decide
decidedAt = noAt + FSYNC;
phase2(parts, "ABORT", decidedAt);
show(parts);
console.log(`  NO arrives at t=${noAt} ms, ABORT forced at t=${decidedAt} ms without waiting for inventory`);

console.log("\n== 3. Coordinator crashes after collecting YES votes, before deciding");
parts = makeParts(["YES", "YES", "YES"]);
votesAt = phase1(parts);
show(parts);
const outageMs = 30000;
const hotRowRps = 50;
console.log(`  every participant is IN DOUBT: it voted YES, so it may not abort on its own`);
const asked = parts.map((p) => `${p.name}=${p.state}`).join(", ");
console.log(`  termination protocol: ask peers -> ${asked} -> nobody knows the outcome`);
console.log(`  coordinator down ${outageMs / 1000} s; ${hotRowRps} req/s need the locked stock row -> ${(outageMs / 1000) * hotRowRps} requests blocked`);
const recoveredAt = votesAt + outageMs;
phase2(parts, "ABORT", recoveredAt); // log has no decision record -> presumed abort
console.log(`  recovery at t=${recoveredAt} ms: no decision in the coordinator log -> presumed ABORT`);
show(parts);

console.log("\n== 4. COMMIT forced, coordinator crashes before sending it; payments gives up after 5 s");
parts = makeParts(["YES", "YES", "YES"]);
votesAt = phase1(parts);
decidedAt = votesAt + FSYNC;
parts[1].state = "aborted"; // heuristic decision: an operator or timeout rolled it back
parts[1].releasedAt = parts[1].lockedAt + 5000;
const restartAt = decidedAt + outageMs;
for (const p of parts) {
  if (p.state === "prepared") { p.state = "committed"; p.releasedAt = restartAt + p.rtt / 2 + FSYNC; }
}
show(parts);
const outcomes = new Set(parts.map((p) => p.state));
console.log(`  outcomes: ${[...outcomes].join(" + ")} -> atomicity broken (heuristic hazard)`);

Discussion

Other approaches, follow-ups and how it went in real interviews.

Sign in to join the discussion.

No comments yet. Share how you would answer it.

Question 28 of 106 in System design

Keep your momentum going.

Mark this question solved to track your progress, then take the next one while it is fresh.

Up nextMedium

Head vs tail sampling in OpenTelemetry โ€” when does each break?

Continue