- LLM observability tools at a glance
- Eleven LLM observability tools to evaluate
- Which observability tool suits your stack
- Security risks show up in your traces
- Tracing versus evaluation
- Questions to ask before you choose an LLM observability tool
- Signs your AI app needs observability
- My plain advice on choosing an LLM observability tool
- Frequently asked questions
The best LLM observability tools record what your AI app or agent did on every request, from the prompt to each tool call and the final answer, so you can debug problems and measure quality. Some are open source projects you can host yourself, some combine a gateway with tracing, and others focus on evaluation. Eleven options follow in no particular order.
An AI feature that works perfectly in testing can behave very differently in front of real customers. A prompt that looked fine starts producing odd answers, an agent loops on the same tool call, or response times creep up and nobody notices until someone complains.
The eleven LLM observability tools below are listed alphabetically and in no particular order, and every description comes from the vendor's own website, read this week. If you're still deciding what to build, my explainer on why AI agents hallucinate shows the kind of failure these tools help you catch.
LLM observability tools at a glance
I checked the free plan column on each tool's own website, as of Oct 2026. Yes means the site names a free plan, a free version, a free allowance or an open source version. Trial only means a time-limited trial. Not stated means the pages I read named neither, even where there's a button inviting you to start for free, so check the vendor's page before you sign up.
| Tool | Type | Good for | Free plan |
|---|---|---|---|
| Arize Phoenix | Open source LLM observability and evals | Self-hosted tracing and evaluation | Yes |
| Braintrust | Observability platform for agents | Linking traces to evaluations | Not stated |
| Datadog LLM Observability | Agent and LLM monitoring in Datadog | Teams already monitoring with Datadog | Trial only |
| Galileo | AI observability and evaluation platform | Turning evals into production guardrails | Not stated |
| Helicone | AI gateway with LLM observability | Routing and debugging model calls | Trial only |
| HoneyHive | Agent observability and evaluation | Inspecting every prompt and tool call | Not stated |
| Laminar | Open source observability for AI agents | Catching agent failures automatically | Yes |
| Langfuse | Open source LLM engineering platform | Tracing, prompt management and evals | Yes |
| LangSmith | Agent and LLM observability platform | Apps built with LangChain | Not stated |
| Opik | Open source AI observability from Comet | An open source option with a free tier | Yes |
| Pydantic Logfire | AI observability for LLMs, apps and RAG | Python teams wanting a monthly free allowance | Yes |
Eleven LLM observability tools to evaluate
Arize Phoenix
Best for: AI engineers who want open source observability and evaluation they can run themselves, with a managed platform from the same company.
Braintrust
Best for: Product teams who want to trace everything their agents do and test what ships, with evaluation built into the same workflow.
Datadog LLM Observability
Best for: Engineering teams already using Datadog for infrastructure who want agent and LLM observability in the same place.
Galileo
Best for: AI teams who want offline evaluations to become guardrails in production, so failures are stopped rather than only logged.
Helicone
Best for: Startups who want to route, debug and analyse their AI applications through one gateway with observability built in.
HoneyHive
Best for: Teams building agents who want to inspect every prompt, tool call, output and handoff, then score quality on live traffic.
Laminar
Best for: Agent builders who want an open source observability platform that automatically catches agent failures in their traces.
Langfuse
Best for: Engineers who want open source tracing, prompt management and evaluation, with open standards and data portability.
LangSmith
Best for: Teams building with LangChain who want to see exactly what their agents are doing and score their performance.
Opik
Best for: Machine learning teams who want an open source observability and evaluation platform with a hosted free tier from Comet.
Pydantic Logfire
Best for: Python developers who want AI observability across LLMs, apps and RAG pipelines, with a monthly free allowance of spans, logs and metrics.
Want AI doing the heavy lifting in your marketing?
I build the systems that handle the boring 80 percent, so you get your week back. Done properly, with the human kept in.
Which observability tool suits your stack
- Open source you can host yourself: Langfuse, Arize Phoenix or Laminar.
- An open source option with a hosted free tier: Opik.
- Apps built with LangChain: LangSmith.
- Teams already on Datadog: Datadog LLM Observability.
- A gateway with tracing built in: Helicone.
- Evaluations that become guardrails: Galileo or Braintrust.
- Inspecting every agent step: HoneyHive.
- Python teams wanting a free monthly allowance: Pydantic Logfire.
Security risks show up in your traces
The OWASP Top 10 for LLM Applications lists prompt injection first, as LLM01:2025, describing it as a vulnerability that occurs when user prompts alter the model's behaviour. Sensitive information disclosure comes second on the same list.
You can't fix what you can't see. Traces show you the exact prompts users sent and how the model responded, which is often the only way to spot an injection attempt or a leak after it's happened, and to test that your fix holds.
Tracing versus evaluation
Tracing records what happened: every prompt, response, tool call and handoff, with timings. It's what you reach for when something has gone wrong and you need to see why.
Evaluation measures how good the answers are, either with test sets before release or by scoring live traffic. Most tools now do both, but each leans one way, so decide which matters more to your team today.
Questions to ask before you choose an LLM observability tool
- Does it support the models and frameworks we use?
- Can we host it ourselves if our data must stay in-house?
- Does it trace multi-step agents, not only single prompts?
- Can we run evaluations on live traffic?
- How does it handle personal data in prompts?
- Does it fit with our existing monitoring?
Signs your AI app needs observability
- Customers report strange answers you can't reproduce. Start tracing.
- Responses are getting slower. Measure each step.
- An agent repeats the same action. Inspect the tool calls.
- You change prompts without testing. Add evaluations.
Do you make an LLM observability or AI evaluation tool? Tool makers can ask to be considered for a featured place on this list.
My plain advice on choosing an LLM observability tool
Add tracing before you launch, not after the first complaint. Even a simple setup that logs prompts, responses and errors gives you something to work with when a customer says the chatbot gave them a strange answer.
Agree upfront what you'll do with personal data that turns up in prompts, as traces can hold more customer information than you expect. My prompt injection defence checklist is a useful companion while you set it up.
Frequently asked questions
What is LLM observability?
LLM observability means recording and analysing what an AI application does on each request, including prompts, responses, tool calls, errors and timings.
Are there free LLM observability tools?
Several are open source, and some offer free tiers or allowances. The free plan column above shows what each vendor's site says.
What's the difference between LLM observability and regular monitoring?
Regular monitoring tracks servers and errors. LLM observability also captures prompts, model responses and quality scores.
Can I self-host an LLM observability tool?
Yes. Several tools are open source and can run on your own infrastructure.
Do these tools work with AI agents?
Most now trace multi-step agents, including tool calls and handoffs between steps.
Can observability help with prompt injection?
It helps you spot attempts by showing the exact prompts and responses, though you still need defences in the app itself.
