AI Engineering
Learn LLMs, RAG, agents, secure tools, and production AI through practical insurance examples.
Local LLMs, API models, and self-hosting
Local or self-hosted models run on infrastructure you control instead of a provider's shared inference API. An API model outsources much of inference operation. Self-hosting can mean a laptop, on-premises servers, or controlled cloud GPU infrastructure.
Ollama, Hugging Face, and Transformers
Ollama simplifies running supported models locally through a runtime and API. Hugging Face hosts models, datasets, and libraries; Transformers is its widely used library for loading and using supported model architectures. A model repository is a distribution…
vLLM and GPU serving
vLLM is an inference-serving system designed for efficient model execution. Serving techniques such as batching and attention-memory management help multiple requests share expensive hardware. GPU optimization is matching workload, memory, scheduling, and…
Quantization, GGUF, and local embeddings
Quantization represents model values with fewer bits to reduce memory and sometimes improve speed. GGUF is a model file format used by compatible local inference runtimes; it is not itself a model or a guarantee of quantization level. Local embeddings run an…
Difficulty is an editorial estimate. Completion means you chose to mark a question as understood.