What's the first thing to build in week 1?

An eval set for a feature you own or a workflow from this site. Twenty real inputs, expected outputs, one scoring function. Everything else — tracing, guardrails, CI gates — hangs off knowing what you're measuring.

Do I need Kubernetes?

Not on day one. Docker + a managed platform (Fly, Railway, Cloud Run) is enough for most LLM services. Graduate to K8s when you have multi-tenant scaling requirements, not because the roadmap said so.

Langfuse, Helicone, or roll my own?

Start managed (Langfuse or Helicone) — ship tracing in an afternoon. Roll your own only when compliance or cost at scale demands it. The patterns are the same either way; see [LLM Observability](/learn/llm-observability).

Learning Path · Advanced

MLOps / LLMOps Engineer Roadmap

Take LLM features from prototype to production — evals, observability, deployment, guardrails, and cost control. A 10-week path for engineers owning AI in prod.

Advanced8–10 weeks10 stages

Outcome

You can deploy an LLM service with tracing, eval gates in CI, output validation, cost attribution, and an incident playbook — the full LLMOps loop, not just a demo API.

Edited by The AIKnowHub team · Editorial team

Key takeaways

1Python or TypeScript — comfortable shipping and debugging backend services
2Basic ML literacy — you understand inference even if you don't train models
3Familiarity with Docker, CI/CD, and one cloud provider
4Comfort reading logs and traces when things break at 2am

Recommended tools

Claude
Ollama
Langfuse
OpenTelemetry

Projects you'll build

The roadmap

01
1. LLMOps vs MLOps — scope the job
Week 1
LLMOps is not the same as training pipelines. Learn what production LLM ownership actually includes in 2026.
02
2. Evals as your foundation
Week 2
Build eval sets before you build dashboards. Every LLMOps practice flows from knowing what 'good' means.
03
3. Observability and tracing
Week 3
Trace every LLM call — prompts, tokens, latency, cost, retrieval steps, tool calls. You can't debug prod without it.
04
4. Guardrails and output validation
Week 4
Layer input filtering, structured outputs, schema validation, and policy checks before responses reach users or downstream systems.
05
5. RAG in production
Week 5
Chunking, embedding, retrieval, reranking — the pipeline that breaks silently when your demo worked fine.
06
6. Deployment patterns
Week 6
API gateways, model routing, caching, batch inference, and local vs hosted inference — ship the service, not just the notebook.
07
7. Agents and tool orchestration
Week 7
Production agents need timeouts, retry budgets, permission scopes, and kill switches — not just a ReAct loop in a notebook.
08
8. CI/CD and eval gates
Week 8
Block deploys that regress eval scores. Version prompts, pin models, and treat LLM config as code.
09
9. Incidents, drift, and iteration
Week 9
Write the playbook for bad answers, cost spikes, and model deprecations. Sample production traces back into your eval set.
10
10. Capstone — production-ready service
Week 10
Deploy one LLM feature with tracing, eval gate, guardrails, cost dashboard, and runbook. This is the portfolio piece.

Frequently asked questions

MLOps traditionally covers training pipelines, feature stores, and model registries. LLMOps covers prompt versioning, evals, retrieval pipelines, agent orchestration, and API inference. Most 'MLOps Engineer' job posts in 2026 want LLMOps skills — read the JD for evals and RAG vs Kubeflow.

Key takeaways

The roadmap

1. LLMOps vs MLOps — scope the job

2. Evals as your foundation

3. Observability and tracing

4. Guardrails and output validation

5. RAG in production

6. Deployment patterns

7. Agents and tool orchestration

8. CI/CD and eval gates

9. Incidents, drift, and iteration

10. Capstone — production-ready service

Frequently asked questions

Related on AIKnowHub