What is a Large Language Model and how does it generate text?

Q: What is a Large Language Model and how does it generate text?

**Target Audience:** Beginners to AI engineering — you can write code and call an API, but you have never looked inside a language model. **Difficulty:** Easy **Verified with:** Node 22.22.0 and Python 3.12.3 (the toy model in the example; no real model is called). Product and paper facts are cited inline with the date they were checked (October 2026). > **How to read this doc:** Every idea is explained in plain language first, then tagged with `📌 Interview term:` so you also learn the exact words an interviewer expects. All program output shown was produced by actually running the example; the **Verified** section quotes it in full. ## 1. Why This Even Matters — A Story First Think of the autocomplete on your phone keyboard. You type "see you" and it offers "tomorrow", "soon", "later". It has no idea what you are planning; it has simply seen a lot of messages and learned which word tends to come next. A large language model is that same idea, scaled up until something surprising

The problem

Question presented to candidate: "Explain to me, as if I were a new engineer on the team, what a large language model actually is. When I send it a prompt, what happens step by step until text comes back?"

What a strong answer should cover:

An LLM is a neural network (almost always a decoder-only transformer) trained to predict the next token given all previous tokens; its knowledge lives in billions of learned weights, not in a database
Generation is autoregressive: tokenize the prompt, run a forward pass, get logits over the whole vocabulary, turn them into probabilities with softmax, pick one token, append it, repeat until a stop token or a length limit
The picking step is the decoding strategy: greedy, or sampling shaped by temperature and top-p; this is why the same prompt can give different answers
Training stages: pretraining on a huge text corpus with a next-token (cross-entropy) loss, then post-training (instruction tuning, preference tuning) that turns a text continuer into an assistant
Practical consequences: output tokens are produced one at a time (latency grows with output length), the model only sees what fits in its context window, and fluent output is not the same as correct output (hallucination)

Code / implementation expected: Optional — a toy next-token model and a generation loop are a strong way to show the mechanism; no real model call is needed.

llm
transformer
fundamentals
next-token

Try it yourself

Sketch your approach before you read the solution. That is what the interview tests.

Stuck? Get a hint.

A short nudge toward the approach. It will not give the answer away.

Solution

Target Audience: Beginners to AI engineering — you can write code and call an API, but you have never looked inside a language model. Difficulty: Easy Verified with: Node 22.22.0 and Python 3.12.3 (the toy model in the example; no real model is called). Product and paper facts are cited inline with the date they were checked (October 2026).

How to read this doc: Every idea is explained in plain language first, then tagged with 📌 Interview term: so you also learn the exact words an interviewer expects. All program output shown was produced by actually running the example; the Verified section quotes it in full.

1. Why This Even Matters — A Story First

Think of the autocomplete on your phone keyboard. You type "see you" and it offers "tomorrow", "soon", "later". It has no idea what you are planning; it has simply seen a lot of messages and learned which word

The solution is hidden so you can try first

16 min read · 11 sections

Read the code

A toy next-token model: probabilities, temperature, greedy decoding and sampling

Open in playground

// A toy language model: count which word follows which, turn counts into
// probabilities, then generate text one token at a time (autoregressive).
const corpus =
  "the cat sat on the mat . the dog sat on the rug . the cat ate the fish . " +
  "the dog ate the bone . a cat sat on a mat . the cat sat on the rug .";
const tokens = corpus.split(" ");

// "Training": count next-token frequencies for every token (a bigram model).
const counts = {};
for (let i = 0; i + 1 < tokens.length; i++) {
  const cur = tokens[i], nxt = tokens[i + 1];
  counts[cur] = counts[cur] || {};
  counts[cur][nxt] = (counts[cur][nxt] || 0) + 1;
}

// Logits are log-counts; softmax with a temperature turns them into probabilities.
function nextDistribution(prev, temperature) {
  const words = Object.keys(counts[prev]).sort();
  const logits = words.map((w) => Math.log(counts[prev][w]) / temperature);
  const max = Math.max(...logits);
  const exps = logits.map((l) => Math.exp(l - max));
  const sum = exps.reduce((a, b) => a + b, 0);
  return words.map((w, i) => [w, exps[i] / sum]);
}

const fmt = (dist) => dist.map(([w, p]) => w + "=" + p.toFixed(3)).join(" ");
console.log("training tokens:", tokens.length, "| vocabulary:", new Set(tokens).size);
console.log("P(next | the)   T=1.0:", fmt(nextDistribution("the", 1.0)));
console.log("P(next | the)   T=0.5:", fmt(nextDistribution("the", 0.5)));
console.log("P(next | the)   T=2.0:", fmt(nextDistribution("the", 2.0)));

// Greedy decoding: always take the most likely token (ties -> alphabetical).
function greedy(prompt, maxNew) {
  const out = prompt.split(" ");
  let logProb = 0;
  for (let step = 1; step <= maxNew; step++) {
    const dist = nextDistribution(out[out.length - 1], 1.0);
    let best = dist[0];
    for (const d of dist) if (d[1] > best[1]) best = d;
    logProb += Math.log(best[1]);
    out.push(best[0]);
    console.log("  step " + step + ": context=[" + out.slice(0, -1).join(" ") + "] -> " + best[0] + " (p=" + best[1].toFixed(3) + ")");
    if (best[0] === ".") break;
  }
  return { text: out.join(" "), logProb };
}
console.log("greedy from 'the dog':");
const g = greedy("the dog", 8);
console.log("result:", g.text, "| log-prob of new tokens:", g.logProb.toFixed(3));

// Sampling: draw from the distribution with a seeded generator (Park-Miller).
let seed = 123456789;
const rand = () => { seed = (seed * 48271) % 2147483647; return seed / 2147483647; };
function sample(prompt, temperature, maxNew) {
  const out = prompt.split(" ");
  for (let step = 0; step < maxNew; step++) {
    const dist = nextDistribution(out[out.length - 1], temperature);
    const r = rand();
    let acc = 0, pick = dist[dist.length - 1][0];
    for (const [w, p] of dist) { acc += p; if (r < acc) { pick = w; break; } }
    out.push(pick);
    if (pick === ".") break;
  }
  return out.join(" ");
}
for (const t of [0.5, 1.0, 2.0]) {
  seed = 123456789;
  const tally = {};
  for (let i = 0; i < 1000; i++) { const w = sample("the", t, 1).split(" ")[1]; tally[w] = (tally[w] || 0) + 1; }
  const shares = Object.keys(tally).sort().map((w) => w + ":" + tally[w]).join(" ");
  console.log("T=" + t.toFixed(1) + " 1000 draws after 'the' -> " + shares + " | sentence: " + sample("the", t, 8));
}

Discussion

Other approaches, follow-ups and how it went in real interviews.

Sign in to join the discussion.

No comments yet. Share how you would answer it.

The solution is hidden so you can try first

Keep your momentum going.