AI design patterns cheat sheet

Start with the simplest pattern that passes your tests, and add the next one only when a measured problem calls for it. Most real systems combine three or four.

1. 1

   UserInput check

   Question

2. 2

   Input checkRetriever

   Safe question + user ID

3. 3

   RetrieverModel

   Top chunks the user may see

4. 4

   ModelOutput check

   Draft answer with sources

5. 5

   Output checkUser

   Checked answer

Retrieval-augmented generation

• What: look up relevant passages first, then ask the LLM to answer from them. RAG splits documents into chunks and stores an embedding of each for search.
• Use when: answers depend on private or changing facts, need citations, or depend on who is asking.
• Watch out for: most bad answers start with bad retrieval. Check that the right chunk came back before blaming the model, and filter by permissions inside the search.

Hybrid search + reranking

• What: run keyword search and meaning-based search together, merge the results, then let a reranker rescore the top few dozen.
• Use when: users type exact names, codes or error messages, or plain RAG returns near misses.
• Watch out for: the reranker adds a model call to every query. Rerank a short list, and drop it if your tests show no gain.

Router

• What: a cheap first step that decides where a request goes: which prompt, tool, data source or model. A router can be rules, a small classifier or a small model.
• Use when: requests fall into clear types that need different handling, such as billing, technical help and small talk.
• Watch out for: a wrong route fails quietly. Log every decision, keep a sensible default route, and test the router on its own.

Model cascade

• What: try a small, cheap model first and pass the request up to a stronger one only when the answer fails a check. A model cascade differs from a router by deciding after an attempt, not before.
• Use when: most requests are easy, and you have a reliable way to tell a good answer from a weak one.
• Watch out for: escalated requests pay for both models and wait longer. If most requests escalate, skip the cascade.

Semantic cache

• What: store past answers by meaning, and reuse one when a new question is close enough to an old one. A semantic cache sits in front of the model.
• Use when: many people ask the same few questions in different words, and the answers aren't personal.
• Watch out for: "close enough" can be wrong ("cancel my order" vs "don't cancel my order"). Set a strict similarity threshold, keep caches per tenant, and expire entries when the source changes.

Guardrail sandwich

• What: checks on both sides of the model: screen input for prompt injection, abuse and personal data; screen output for leaks, unsafe content and unsupported claims. Each layer is a guardrail.
• Use when: the system faces the public, reads untrusted text, or can take actions.
• Watch out for: checks add latency and block good requests too. Measure both missed attacks and false blocks, and enforce permissions in code, not in the prompt.

Structured output with validation

• What: ask for structured output matching a schema, check it in code, and on failure retry once with the error message before falling back.
• Use when: the reply feeds other software: forms, database rows, tool arguments, labels.
• Watch out for: a valid shape can still hold wrong values. Add business rules (totals add up, dates are real) on top of schema validation.

Evaluator-optimiser loop

• What: one model drafts, another (or the same one with a rubric) critiques, and the draft is revised until it passes or runs out of rounds. This is the evaluator-optimiser pattern.
• Use when: quality can be judged against clear criteria, such as translations, code that must pass tests or summaries that must cover set points.
• Watch out for: cost multiplies with each round, and a judge can reward the wrong thing. Cap the rounds and prefer real checks (tests, rules) over opinion.

Tool-using agent

• What: the model works in a loop: decide, make a tool call, read the result, repeat until done. Your code runs each tool with the user's own permissions.
• Use when: the steps can't be known in advance, such as research, debugging or multi-system lookups.
• Watch out for: loops, wrong tools and runaway cost. Set a step budget, keep the tool list short and clearly named, and apply least privilege.

Orchestrator-workers

• What: an orchestrator splits a big task, hands parts to workers that each start with a clean context, and combines their short reports.
• Use when: the parts are truly independent and each needs a lot of reading, such as researching ten companies or changing many files.
• Watch out for: workers can't see each other, so tightly linked work goes wrong. Limit to one level of delegation and a small number of workers.

Human-in-the-loop approval

• What: the agent prepares an action and pauses; a person sees exactly what will happen and approves or rejects it. This is human-in-the-loop control.
• Use when: an action costs money, can't be undone, or reaches outside the company (payments, emails, deletions).
• Watch out for: people approve without reading when asked too often. Ask only for risky actions, show a clear summary, and tie the approval to that exact action so it can't be reused.

Background generation jobs

• What: accept the request, return a job ID at once, do the slow work from a queue, and notify the user or let them check back.
• Use when: work takes longer than a person will wait on a page: long reports, video, large document batches.
• Watch out for: retries can run a job twice. Use idempotency keys, save progress as you go, and show honest progress.

Fallback chain across providers

• What: on timeout or error, try the next model in a fixed list, ideally from a different provider, behind a circuit breaker so a failing provider is skipped quickly.
• Use when: downtime is costly and one provider's outage would stop the product.
• Watch out for: a fallback model behaves differently. Run it through the same evals, adjust prompts per model, and check refusals that arrive as normal-looking replies.

Map-reduce over long documents

• What: split a long input into pieces, process each piece in parallel (map), then combine the partial results (reduce). This is map-reduce applied to model calls.
• Use when: a document or collection is too long for one context window, or a single long prompt misses details.
• Watch out for: facts that span two pieces get lost, and the combine step can drop detail. Overlap pieces slightly and keep references back to the source.

For experienced engineers

Patterns stack in a typical order: a router at the front, a semantic cache before the model, RAG for knowledge, guardrails around it, structured output for anything software reads, and a fallback chain underneath. Agents come last, once a fixed workflow has clearly run out of road.