AI Engineering
Learn LLMs, RAG, agents, secure tools, and production AI through practical insurance examples.
AI, machine learning, deep learning, and generative AI
Artificial intelligence, or AI, is the broad field of making software perform tasks such as recognizing speech, finding patterns, or understanding language. Machine learning, or ML, learns patterns from examples instead of requiring a rule for every case.…
LLMs and how they work
A large language model, or LLM, is a model trained on large amounts of data to process and generate language. It learns patterns in how text is arranged and can use those patterns for summarization, extraction, explanation, and tool selection. It does not…
Tokens and token usage
Tokens are the small units of input and output processed by a model. A token might be a word, part of a word, punctuation, or another encoded unit. Token counts depend on the tokenizer, language, and content; characters and words are only rough estimates.
Context window and context construction
A context window is the amount of model input and generated content a model can handle within its documented limits. Context is the particular information you provide for one call: instructions, messages, evidence, and tool results. A large window gives…
Temperature and top-p
Temperature changes how sharply a model favors its higher-scoring next-token choices. Lower values generally reduce variation; higher values allow more variation. Top-p, also called nucleus sampling, restricts selection to a set of tokens covering a chosen…
System, user, assistant, and tool messages
Message roles identify the purpose and source of information. System or developer instructions describe application behavior, user messages contain requests, assistant messages contain model responses, and tool results supply external observations. Exact role…
Prompt engineering, zero-shot, and few-shot prompting
Prompt engineering is writing clear task instructions, inputs, constraints, and output expectations. Zero-shot prompting gives instructions without examples; few-shot prompting adds a small number of input/output examples. A prompt is part of the application…
Structured output and JSON output
JSON output is machine-readable data, such as an object with named fields. Structured output adds a declared schema so the response follows expected fields and types when the provider supports schema-constrained generation. Valid JSON, valid schema, and valid…
Function calling and tool calling
Function calling lets a model propose a named function and structured arguments. Tool calling is the broader pattern that includes functions and capabilities such as search or code execution. A proposed tool call is an untrusted request that your application…
Streaming
Streaming delivers response events as work happens instead of waiting for the entire response. Text events may contain small pieces called deltas. The browser can show progress and partial text while the server continues processing.
Multimodal models, vision, and audio
Multimodal models process more than one kind of input or output, such as text with images or audio. Vision means interpreting visual content; audio understanding includes spoken words and sometimes sounds or vocal context. Support differs by model and…
Model selection, routing, fallback, latency, and cost
Model selection chooses a model for a task. Routing chooses among models or processing paths per request, while fallback handles a failed or unavailable preferred path. Latency is waiting time; cost includes tokens, tools, retrieval, infrastructure, and…
Provider ecosystems: OpenAI, Anthropic, Gemini, and Azure OpenAI
Providers supply hosted model inference and related capabilities. OpenAI, Anthropic, and Google offer their own model ecosystems; Azure OpenAI provides access to supported OpenAI models within Azure's service and identity environment. Common concepts…
Difficulty is an editorial estimate. Completion means you chose to mark a question as understood.