Skip to the solution
MediumLow-Level Design7 min read

How do you consume a readable stream with for await...of versus the data event?

284 views

The problem

Question presented to candidate: "You are processing each chunk of a stream with an async operation that takes real time. Should you use the 'data' event or for await...of, and does it matter?"

What a strong answer should cover:

  • Attaching a "data" listener switches a Readable into flowing mode immediately β€” the stream starts emitting data as fast as it can produce it, regardless of how long your handler takes to process each chunk, because emitting an event does not wait for an async listener's returned promise.
  • for await...of iterates a Readable using its async iterator β€” each iteration explicitly requests the next chunk only once the current loop body (including anything it awaits) has finished. This is a pull-based model, not a push-based one.
  • The practical consequence: with the "data" event, an async handler that is slower than the data arrives will fall behind with no built-in signal to slow production down β€” you would need to manually pause()/resume() the stream to get that back. With for await...of, the pacing is naturally tied to how fast your loop body actually finishes.
  • A precise, honest nuance worth stating rather than glossing over: for await...of pacing the consumer's own request rate to the chunks is not automatically identical to the underlying source slowing its own production β€” a source can still fill its internal buffer up to its highWaterMark ahead of what the consumer has asked for yet. True end-to-end backpressure to a slow/expensive source depends on that buffer filling up, which a small or fast source may never actually trigger.
  • for await...of correctly propagates a thrown error out of the loop as a normal exception (usable with try/catch), whereas the "data" event requires a separate "error" listener on the stream.
  • for await...of is generally the more modern, more correct default for anything where a slow, real async operation happens per chunk; the "data" event remains reasonable for a synchronous, fast handler with no meaningful per-chunk async work.

Clarifying questions expected:

  • "Is the per-chunk processing synchronous and fast, or does it involve a real async operation (a DB write, a network call)?" β€” that is the deciding factor.
  • "Does anything downstream need explicit pause()/resume() backpressure control if the 'data' event is used?" β€” a good answer flags this as the added complexity the event-based approach brings.

Code / implementation expected: Yes β€” timing the actual delivery of chunks under both approaches with an artificial async delay is the clearest way to make the difference concrete rather than assert it.

  • streams
  • async-iteration
  • for-await-of
  • backpressure

Try it yourself

Sketch your approach before you read the solution. That is what the interview tests.

Stuck? Get a hint.

A short nudge toward the approach. It will not give the answer away.

Solution

The solution is hidden so you can try first

7 min read Β· 8 sections

Read the code

JavaScriptIdentical object-mode stream, identical 50ms consumer delay, measured under both consumption styles
import { Readable } from "node:stream";

function makeCounter() {
  let n = 0;
  return new Readable({
    objectMode: true,
    read() {
      n++;
      if (n > 5) { this.push(null); return; }
      this.push(String(n));
    },
  });
}

// "data" event: does NOT wait for the async handler.
await new Promise((resolve) => {
  const r = makeCounter();
  r.on("data", async (chunk) => {
    console.log("data event delivered", chunk, "at", Date.now());
    await new Promise((res) => setTimeout(res, 50)); // ignored by the stream
  });
  r.on("end", resolve);
});
// All 5 timestamps land within ~1ms of each other.

// for await...of: genuinely paced by the consumer.
const r2 = makeCounter();
for await (const chunk of r2) {
  console.log("for-await received", chunk, "at", Date.now());
  await new Promise((res) => setTimeout(res, 50)); // the NEXT read genuinely waits
}
// Consecutive timestamps land ~50-58ms apart, matching the delay.

Discussion

Other approaches, follow-ups and how it went in real interviews.

Sign in to join the discussion.

No comments yet. Share how you would answer it.

Question 101 of 152 in Node.js

Keep your momentum going.

Mark this question solved to track your progress, then take the next one while it is fresh.

All Node.js questions
Up nextMedium

Describe the Node.js module system. What are CommonJS and ES Modules?

Continue