When should you still use multiprocessing vs free-threaded threads in Python 3.12+?

Q: When should you still use multiprocessing vs free-threaded threads in Python 3.12+?

**Target Audience:** Developers who know threads, processes and the GIL at a basic level and must choose a concurrency model for CPU-bound Python work. **Difficulty:** Medium **Python version:** Example 1 runs in the page on Pyodide 0.25.1 (CPython 3.11.3) and was checked on CPython 3.11. Example 2 starts threads and processes, so it was run on CPython 3.13 (default build). **We did not have a free-threaded interpreter**, so performance claims about it come from the CPython docs and PEPs cited inline, not from our runs. > **How to read this doc:** Every idea comes in plain language first. Words an interviewer expects you to use are tagged `📌 Interview term:` and defined on the spot; all of them are repeated in the Quick Glossary at the end. Every output shown was printed by a real run. ## 1. Why This Even Matters — A Story First A bakery needs to frost 1,000 cakes. Two options: - **Hire more bakers in the same kitchen** (threads). They share the fridge, the frosting bowl and the o

The problem

Question presented to candidate: "Now that CPython has a free-threaded build, threads can finally run Python code on several cores. Is there still a reason to reach for multiprocessing? How would you decide between processes and free-threaded threads for a CPU-heavy job?"

What a strong answer should cover:

Version reality first: there is no free-threaded build in 3.12; it is experimental in 3.13 and supported but optional in 3.14. On the default build, CPU-bound pure-Python work still needs processes.
What threads win on: no pickling, cheap start-up, shared objects by reference, low memory — best when tasks share large in-memory state or exchange data often.
What processes still win on: they work on every build and with every C extension (one extension without free-threading support re-enables the GIL), they give memory isolation (no data races by default) and crash isolation (a segfault kills one worker, not the app), and the OS can kill or limit them.
The costs of processes: pickling every argument and result, start-up time (especially spawn), duplicated memory; mitigate with chunking, initializer, and shared_memory.
Measure: single-threaded slowdown of the free-threaded build, the size of data crossing the boundary, and real wall-clock time on the target machine.

Clarifying questions expected:

Which Python version and build will production run, and do all compiled dependencies support free threading?
How much state do the workers share, and how often do they exchange data?

Code / implementation expected: Optional — the same function run on a ThreadPoolExecutor and a ProcessPoolExecutor, with the trade-offs explained.

gil
threading
performance

Try it yourself

Sketch your approach before you read the solution. That is what the interview tests.

Stuck? Get a hint.

A short nudge toward the approach. It will not give the answer away.

Solution

Target Audience: Developers who know threads, processes and the GIL at a basic level and must choose a concurrency model for CPU-bound Python work. Difficulty: Medium Python version: Example 1 runs in the page on Pyodide 0.25.1 (CPython 3.11.3) and was checked on CPython 3.11. Example 2 starts threads and processes, so it was run on CPython 3.13 (default build). We did not have a free-threaded interpreter, so performance claims about it come from the CPython docs and PEPs cited inline, not from our runs.

How to read this doc: Every idea comes in plain language first. Words an interviewer expects you to use are tagged 📌 Interview term: and defined on the spot; all of them are repeated in the Quick Glossary at the end. Every output shown was printed by a real run.

1. Why This Even Matters — A Story First

A bakery needs to frost 1,000 cakes. Two options:

**

The solution is hidden so you can try first

11 min read · 12 sections

Run the code

Edit the code and run it here. Each console line with an expected value becomes a test.

PythonThe process tax: bytes pickled per design, plus a decision helper

Practice

PythonMemory and crash isolation: what only processes give you

# Two reasons processes still win even when threads can run in parallel:
# memory isolation and crash isolation. Runs on CPython (not the browser).
import multiprocessing as mp
import os
import sys
import sysconfig
from concurrent.futures import ProcessPoolExecutor, ThreadPoolExecutor
from concurrent.futures.process import BrokenProcessPool

CACHE = {"hits": 0}


def touch_cache(_):
    CACHE["hits"] += 1          # threads: one shared CACHE; processes: one each
    return CACHE["hits"]


def crunch(n):
    total = 0
    for i in range(n):
        total += i * i % 7
    return total


def crash(n):
    if n == 2:
        os._exit(1)             # simulates a segfault in a C extension
    return n


if __name__ == "__main__":
    print("free-threaded build:", bool(sysconfig.get_config_var("Py_GIL_DISABLED")))
    print("GIL enabled:", sys._is_gil_enabled())
    ctx = mp.get_context("spawn")

    # Same function, same answers, either executor
    with ThreadPoolExecutor(4) as tp, ProcessPoolExecutor(4, mp_context=ctx) as pp:
        t = list(tp.map(crunch, [100_000] * 4))
        p = list(pp.map(crunch, [100_000] * 4))
    print("threads == processes:", t == p)

    # Memory isolation: threads share module state, processes do not
    with ThreadPoolExecutor(2) as tp:
        list(tp.map(touch_cache, range(6)))
    print("parent CACHE after threads:", CACHE["hits"])
    with ProcessPoolExecutor(2, mp_context=ctx) as pp:
        list(pp.map(touch_cache, range(6)))
    print("parent CACHE after processes:", CACHE["hits"])

    # Crash isolation: a dying worker breaks the pool, not the parent
    try:
        with ProcessPoolExecutor(2, mp_context=ctx) as pp:
            print(list(pp.map(crash, range(4))))
    except BrokenProcessPool as exc:
        print("parent survived:", type(exc).__name__)
    print("parent still running, pid is an int:", isinstance(os.getpid(), int))

Discussion

Other approaches, follow-ups and how it went in real interviews.

Sign in to join the discussion.

No comments yet. Share how you would answer it.

The solution is hidden so you can try first

Keep your momentum going.