Skip to the solution
HardDSA8 min read

How would you implement an async generator function to stream paginated API results, and how does a consumer use it with for await...of?

802 views

The problem

Question presented to candidate: "Say you are calling a paginated API that returns a page of items plus a cursor for the next page. Write an async generator function that streams every item across every page to the caller one at a time, so the caller can use for await...of and start processing before every page has loaded. How do you prove this is actually lazy, one page at a time, and not secretly loading everything up front?"

What a strong answer should cover:

  • async function* combines a generator (pausable, resumable via yield) with an async function (can await inside), which is exactly the shape needed to await a network call between yields.
  • The generator holds a loop: await the current page, yield each of its items one at a time, then move to the next page using the cursor the API handed back, stopping when there is no next page.
  • for await...of on the consuming side automatically calls the async generator's .next() repeatedly, awaiting each result, and unwraps { value, done } for you β€” no manual iterator-protocol code needed.
  • Laziness is the key selling point over "fetch everything into an array first": only one page fetch is ever in flight at a time, and the NEXT page is not requested until the CURRENT page's items have all been consumed.
  • Breaking out of the for await...of loop early (a break, a return, or an uncaught error) triggers the generator's implicit cleanup and genuinely stops it from fetching any further pages.
  • This is testable, not just assertable β€” instrument the fetch function with timestamps or a counter and show the real call pattern rather than describing it from memory.

Clarifying questions expected:

  • "Should errors from a single page fetch stop the whole stream, or should the consumer be able to catch and continue?" β€” a real async generator propagates a thrown error out of the current for await iteration, ending the loop, unless the generator body itself catches it.
  • "Does the API give a full cursor, or just a page number, and can pages be fetched out of order?" β€” determines whether the loop can be parallelized at all, or whether it is fundamentally sequential.

Code / implementation expected: Yes β€” a full, runnable async generator plus a real, timestamped test proving only one page fetch is ever in flight, and that an early break genuinely stops further fetching.

  • async-generator
  • generators
  • async-iteration

Try it yourself

Sketch your approach before you read the solution. That is what the interview tests.

Stuck? Get a hint.

A short nudge toward the approach. It will not give the answer away.

Solution

The solution is hidden so you can try first

8 min read Β· 7 sections

Run the code

Edit the code and run it here. Each console line with an expected value becomes a test.

JavaScriptAsync generator streaming a paginated API plus a real, timestamped test proving one-page-ahead laziness and early-break behavior (run directly)
JavaScriptReference: the eager (broken) alternative this doc's pitfalls are based on β€” awaits every page up front, losing the streaming property (run directly to see the contrast)
const DB = [["a1", "a2"], ["b1", "b2"], ["c1", "c2"]];
let concurrentFetches = 0;
let maxConcurrent = 0;

async function fetchPage(pageIndex) {
  concurrentFetches++;
  maxConcurrent = Math.max(maxConcurrent, concurrentFetches);
  await new Promise((r) => setTimeout(r, 20));
  concurrentFetches--;
  return DB[pageIndex];
}

// BUG: this is a regular async function returning an array, not an async
// generator -- it awaits every page BEFORE returning anything, so the
// "streaming" claim is false: the whole dataset loads into memory up front.
async function eagerPaginate() {
  const allPages = await Promise.all(DB.map((_, i) => fetchPage(i)));
  return allPages.flat();
}

(async () => {
  const items = await eagerPaginate();
  console.log("eager result:", items);
  console.log("max CONCURRENT fetches in flight (all pages fired at once):", maxConcurrent);
  console.log("this defeats streaming: nothing is handed to the consumer until ALL pages have loaded");
})();

Discussion

Other approaches, follow-ups and how it went in real interviews.

Sign in to join the discussion.

No comments yet. Share how you would answer it.

Question 134 of 165 in JavaScript

Keep your momentum going.

Mark this question solved to track your progress, then take the next one while it is fresh.

Up nextHard

In what order do console.log, setTimeout, and Promise.then run?

Continue