State machines, DAGs, and event-driven workflows

🟠 SHOULD KNOW · These are backend engineering concepts applied to AI workloads.

Overview

• A state machine models valid states and transitions.
• A DAG, directed acyclic graph, models dependencies without loops.
• Event-driven workflows react to events such as document.uploaded or review.approved instead of doing all work inside one HTTP request.

Why this matters

AI pipelines often involve dependent steps, parallel work, and user events that arrive later.

An everyday example

A claim moves through stages, document processing has dependencies, and an approval message triggers the next step.

A technical example

Parse and OCR jobs finish before embeddings are generated; approval is an external event correlated to a draft ID.

How it works

1. Persist request/job
2. Emit event
3. Worker validates dependency/state
4. Execute step
5. Record outcome
6. Emit next event.

Architecture and flow

Durable event → state/dependency check → worker → persisted result → next event.

The insurance assistant example

Claim intake launches document analysis and fraud checks independently, then a review step waits for both required outcomes.

When to use it

Background pipelines and multi-step processes with clear dependencies.

When to avoid it

Avoid assuming every graph is a DAG: agent loops are cyclic. Do not build event infrastructure for a trivial synchronous lookup.

Interview answer

“I use state machines for lifecycle rules, DAGs for acyclic dependencies, and events for decoupled asynchronous progress.”

Benefits, trade-offs, and production considerations

• Define event schemas, correlation IDs, ordering assumptions, deduplication, and ownership.
• Queue delivery is commonly at least once; consumers must tolerate repeats.