Question presented to candidate:
"[x * x for x in range(n)] and (x * x for x in range(n)) differ by one pair of brackets. What is the actual memory difference between them, how would you measure it, and are there cases where the generator expression saves nothing?"
What a strong answer should cover:
- A list comprehension runs to completion immediately and stores every result: a pointer array of about 8 bytes per item on 64-bit CPython (plus over-allocation) and every element object it points to; memory is O(n)
- A generator expression creates a small generator object (about 200 bytes on 64-bit CPython 3.11) holding a paused frame and the source iterator; items are produced one at a time and memory is O(1) in n
- Measurement:
sys.getsizeofis shallow (counts the list pointer array, not the ints), so usetracemallocpeak memory, or add up element sizes, to see the real cost - The saving disappears when the consumer materialises anyway:
sorted(),list(), andstr.join()(which builds a sequence internally) peak the same either way; it only pays off with streaming consumers likesum,any,max,for - Behavioural differences that come with it: the generator is lazy (only the first iterable is evaluated at creation), single-pass, has no
len()or indexing, and raises errors at consumption time; speed is roughly similar and depends on size - CPython detail: both get their own code object on 3.11; PEP 709 inlined list, dict and set comprehensions in 3.12, but generator expressions still create a generator
Code / implementation expected: Optional β measuring both with sys.getsizeof and tracemalloc, and noting why getsizeof alone under-reports.