AI Engineering
Learn LLMs, RAG, agents, secure tools, and production AI through practical insurance examples.
Why isn't normal unit testing enough?
Model outputs vary and can be plausible but wrong. I still unit-test deterministic code, but also evaluate representative tasks, support from evidence, tool behavior, and safe failure using versioned datasets and rubrics.
Faithfulness versus correctness?
Faithfulness asks whether the answer follows its evidence. Correctness asks whether it is the right answer for the applicable task and facts. An answer can faithfully repeat an outdated or inapplicable policy and remain wrong.
How do you trust an LLM judge?
I calibrate it against human-labeled cases, test adversarial outputs, track disagreement, and version its rubric/model. I combine it with deterministic checks and never make it the sole permission or financial gate.
A new prompt raises average accuracy but causes one tenant leak. Ship?
No. Severe boundary violations are release blockers regardless of the average. I investigate the cause, fix the enforced access boundary, add the case to the security suite, and rerun relevant regression and quality checks.
Difficulty is an editorial estimate. Completion means you chose to mark a question as understood.