Name the shape when something breaks

Interview decks love pattern bingo. Production teams already ship the same shapes under boring product names: edge gateways, on-device caches, fan-out search, attachment pipelines. Ricky Ho’s 2010 tour of scalable system design patterns still maps cleanly — if you skip the résumé rename and ask which failure you are fixing.

The point of naming a pattern is not decoration. It is so you can argue the tradeoff out loud: what this shape buys, what it costs, and which nearby shape is the wrong borrow. Start from a concrete break.

Failure 1 — Stampede after a purge

Symptom: After a deploy or CDN purge, cold open suddenly cliffs. Thousands of clients miss together, hit origin, and amplify each other. For example, in a mail app, thread metadata that usually lives in an API/CDN cache plus a Room store on device all miss at once — open-inbox feels like a thundering herd.

Right shape: result cache with disciplined invalidation (cache-aside, jittered TTL, write-through on the “I just wrote this” path). Before expensive work, look up a prior answer. Invalidation is the tax; TTL alone is not a strategy when avatars and thread metadata change mid-scroll. Stampede mitigations — soft TTL, single-flight / request coalescing, jitter — belong to this family because the problem is repeated reads of the same key, not “we need more servers.”

Wrong borrow: map-reduce or a heavier orchestrator. Nightly batch aggregation will not save an interactive open. Adding a workflow engine so every cold open “orchestrates” a refresh turns a cache miss into a distributed saga. If a user is waiting on a spinner, you wanted a warm result or a partial stale read with a background refresh — not a batch job.

Load balancers sit adjacent but do not fix the stampede by themselves. Spreading identical origin hits across replicas still multiplies origin work when every client asks for the same miss. Stateless replicas help after you stop treating every open as unique work.

Failure 2 — One slow shard holds the whole UI

Symptom: Global search or a unified typeahead fans out to many backends. Nineteen shards answer in 40 ms; one straggler holds the suggestion list for 800 ms. Completeness looks virtuous in a design doc and feels broken in the product.

flowchart LR
  Client[Client / BFF] --> Disp[Dispatcher]
  Disp --> W1[Shard A]
  Disp --> W2[Shard B]
  Disp --> W3[Shard C]
  W1 --> Agg[Aggregator]
  W2 --> Agg
  W3 --> Agg
  Agg --> Client

Figure 1. Scatter-gather: parallel workers, one merged response (timeouts drop stragglers).

Right shape: scatter-gather with explicit timeouts. Unlike a load balancer (pick one identical worker), scatter-gather sends work to many workers that may hold different shards or facets, then aggregates. The budget decision is completeness vs latency: drop late shards; show partial results with a clear “still loading” affordance. That is a client UX contract as much as a server one. A BFF that fans out to mail + calendar + contacts for typeahead is the same shape — name it so you can argue for partial merge instead of waiting on the worst facet.

Wrong borrow: pipe-and-filter in series. Sequencing shard calls “for simplicity” turns parallelism into the sum of latencies. Pipe-and-filter is the right family when stages have different owners and SLAs (upload → scan → transcode → publish); it is the wrong borrow when the work is embarrassingly parallel facets of one user-visible answer.

Failure 3 — Send waits on the whole CDN pipe

Symptom: The user taps send and the UI blocks until virus scan, thumbnail, and CDN publish all finish. Trust requirements may need some of that inline; most of it does not. The product mistake is conflating “message is durable” with “every filter finished.”

Right shape: pipe-and-filter with an early ack, plus an orchestrator when retries matter. Data flows through sequential stages; each filter owns one transformation. Ack when the message is durable; let thumbnail/CDN finish asynchronously unless policy requires inline completion. On a compose screen, that means separate UI states for “message exists” vs “preview ready.” When step three fails and must retry without the phone chaining five fragile calls, a server-side orchestrator (dumb workers, smart schedule) owns the DAG — persist → upload attachments → commit send → fan out push/index.

Wrong borrow: scatter-gather for ordered trust stages. Fan-out does not help if stage B must not run before stage A’s scan clears. Parallelism here creates races, not speed. Also wrong: pretending interactive send is map-reduce. Map-reduce earns its keep when disk or scan cost dwarfs coordination — nightly “large attachments” analytics, one-shot reindex of a search partition — not when a human is staring at a progress spinner.

Argue the tradeoff, then stop renaming

You already ship these under product names. Naming the shape lets you reject the wrong borrow in a design review — not pad a résumé. If a pattern does not map to a failure you have felt, leave it off the whiteboard.

References

• Ricky Ho — Scalable System Design Patterns (2010)
• Will Larson — Introduction to architecting systems for scale
• Gateway Aggregation — fan-in many backends for one client call
• donnemartin/system-design-primer