Real client decisions
Practice this question| Client requirement | Minimum useful design | Why | What would justify more? |
|---|---|---|---|
| Show claim status | Authenticated API lookup + template | Existing exact facts; no reasoning needed | Natural-language UI may add classification/explanation |
| Explain policy wording | Applicable-source retrieval + LLM + citations | Private knowledge and language simplification | Multi-hop for endorsements if evaluations show need |
| Help file a claim | Extraction/clarification + deterministic draft/review/submit | Varied language but fixed submission rules | Graph for multi-turn state and durable pauses |
| Summarize uploaded invoice | Parser/OCR + schema extraction + exact checks | Bounded document task | Python/local model only for a concrete capability need |
| Automate missing-document reminders | Durable schedule + rules + reviewed/template message | Known events and recipients | LLM drafting only if wording variation matters |
| Research public products | Bounded search + cited synthesis + review | Variable evidence gathering | Specialists only if measurable comparison quality improves |
| Shared tools for several AI hosts | MCP facade over secure services | Standardized reuse and discovery | More capabilities only after review and scoped need |
Worked decision: “Automatically approve claims from uploaded photos.” First separate observation from adjudication. Vision can identify proposed observations, OCR can extract document fields, and RAG can retrieve applicable terms. Approval requires authoritative business rules, evidence sufficiency, and authorized review. A model's plausible photo description is not sufficient basis for a financial commitment.
Worked decision: “Send a daily email about incomplete claims.” If incomplete status and recipients are structured, a scheduled deterministic job and template may be enough. Add model drafting only for a demonstrated communication need; enforce consent, current status, deduplication, and safe recipients regardless.
Worked decision: “An internal assistant should answer from all company documents.” Start with identity and source ACL synchronization before generation. Design ingestion/freshness/deletion and authorized source links. Add hybrid retrieval, reranking, or agentic search only when the evaluation set reveals the relevant gap.