Deployment, Scaling, Versioning, and Operations

Local versus remote

Production checklist

1. Run the container as a non-root user with minimal filesystem and network access.
2. Enable HTTPS/TLS and strict Host/Origin handling.
3. Make health checks dependency-aware without leaking details.
4. Prefer stateless handlers when session state is unnecessary.
5. Design session state, an event bus, or resumability storage before horizontal scaling.
6. Configure timeouts, payload limits, concurrency limits, and graceful shutdown.
7. Run type checking, tests, security checks, and container scanning in CI.
8. Test protocol and SDK changes against supported clients.
9. Provide feature flags and a rollback path.
10. Maintain a runbook that defines the alert owner, incident response, and credential rotation.

Versioning

A tool schema is a contract. Breaking renames or removals can affect clients and models. Introduce additive fields carefully, maintain a deprecation period for old tools, publish a changelog, and run contract tests.

Cost drivers

Compute, database/API usage, model sampling, egress, logs, and always-on connections. Cache only permission-safe reads and never mix data between tenants.