LLMOps
Also called: MLOps for LLMs, AI operations
The operational discipline for language-model systems — evaluation in CI, tracing, cost control and versioning — because these systems degrade quietly rather than failing loudly.
Infrastructure monitoring tells you the service is up. It cannot tell you the answers got worse after last Tuesday's prompt change.
That gap is what LLMOps exists to close. AI systems fail by degrading, not by erroring — accuracy slides from 94% to 81% across a quarter because the input mix shifted, a provider updated a model, or three prompt tweaks each helped one case and broke four others.
The four things it actually means
Evaluation in CI. An evaluation set that runs on every pull request, so a change that drops accuracy fails the build.
Tracing with cost attached. One view per request: prompt, retrieved context, tool calls, output, latency and money spent. Without it, debugging is guesswork with a large bill.
Cost control. Per-feature token budgets, caching of stable prefixes, routing simple steps to cheaper models. Measured as cost per successful task, not cost per token — the latter hides the retry rate.
Versioning and rollback. Prompts, models and retrieval configs are versioned artefacts you can roll back in minutes, because at some point you will need to.
The metric that changes conversations
Cost per successful task. It makes the trade-off between a cheaper model and a higher retry rate visible, and it is the only figure a finance team can use.
Governance comes free
The same tracing that makes debugging possible produces the audit trail your risk function will eventually ask for. Building it in costs almost nothing; retrofitting it under audit pressure is a project. See MLOps & LLMOps.