Observability

Concept

LLM Observability in Production

You can't debug what you can't see. LLM observability means tracing every prompt, token, latency spike, and failure — here's the production stack.

Tool Guide

Braintrust Guide

The definitive guide to Braintrust for AI evals and observability — traces to experiments, GitHub Action CI, scoring UX, and when to pick Langfuse instead.

Tool Guide

Langfuse Guide

The definitive guide to Langfuse for LLM observability — traces, prompt versions, evals, cost tracking, self-host setup, and when to pick it over Braintrust or Phoenix.

Comparison

Langfuse vs Braintrust vs Phoenix

Three observability options for production LLM apps — open-source tracing, eval-first SaaS, and RAG-focused OSS from Arize.

Directory

Arize AI

AI observability and evaluation platform for production LLM and ML systems; maker of the open-source Phoenix library.

Directory

Braintrust

AI evaluation and observability platform — traces feed directly into eval datasets, regression tests, and prompt experiments.

Directory

Giskard

Open-source testing framework from a French startup that scans LLM apps and ML models for bias, errors, and vulnerabilities.

Directory

Helicone

Open-source LLM observability platform offering proxy-based logging, cost tracking, caching, and prompt experiments.

Directory

Langfuse

Open-source LLM observability — traces, evals, prompt versioning, and cost dashboards for production AI apps.

Directory

LangSmith

LangChain's platform for tracing, evaluating, and monitoring LLM applications, usable outside LangChain via SDK.

Directory

Neptune.ai

Experiment tracker built in Poland for ML teams, logging metrics, artifacts, and metadata at foundation-model training scale.

Directory

Phoenix (Arize)

Open-source LLM tracing and evaluation from Arize — inspect spans, embeddings, and retrieval quality locally.

Directory

Run:ai

GPU orchestration platform from Israel, acquired by NVIDIA, that schedules and shares GPU clusters for AI workloads.

Directory

Weights & Biases

ML experiment tracking platform whose Weave product adds tracing and evaluations for LLM applications.