Question presented to candidate: "Design an object storage service like Amazon S3. Customers create buckets and PUT, GET, LIST and DELETE objects from a few bytes to many terabytes. It has to be extremely durable, survive losing a whole data centre, and scale to hundreds of petabytes. Walk me through the API, how metadata and data are stored, how you keep data safe, and what breaks first as it grows."
What a strong answer should cover:
- Split into a metadata plane (bucket and key to object record: size, checksum, version, shard locations) and a data plane (storage nodes holding immutable chunks or shards), with stateless front ends in between
- Durability with erasure coding instead of plain replication: RS(6,3) stores 150% of the data and survives 3 lost shards, versus 300% for 3 copies; repair reads k shards, so repair traffic is the cost
- Placement across failure domains: shards spread so that losing one availability zone loses at most m shards (RS(6,3) as 3/3/3 survives; RS(10,4) over 3 zones does not)
- The write path and consistency: write and acknowledge shards first, then commit the metadata record as the single commit point; that gives strong read-after-write and atomic overwrites, plus conditional writes
- Metadata at scale: range-partitioned by bucket and key so LIST by prefix is a range scan, split hot partitions, versioning and delete markers, garbage collection of unreferenced data
- Capacity numbers: objects, bytes, disks, metadata size, throughput, and what breaks first at 10x (metadata hot spots, repair bandwidth, small-object overhead)
Clarifying questions expected:
- Object size distribution, request rates, and read/write ratio?
- Durability and availability targets, and must it survive a full AZ or region loss?
- Which features are in scope: versioning, listing, multipart, lifecycle, storage classes?
Code / implementation expected: Optional โ architecture and numbers come first; the example implements Reed-Solomon over a small prime field to show encode, loss and rebuild, and computes the overhead and loss odds.