Start with how much of your code you're willing to change. Full observability platforms trace and evaluate everything, prompt-first tools suit teams editing prompts daily, and a gateway sees every call because traffic flows through it. Run one real feature through two options and keep the one your engineers find easiest.
LLM observability means seeing every model call your app makes: what went in, what came out, and whether the answer was any good. Evaluation is the other half, checking quality before and after you ship. If you're leaving Galileo, here are seven tools that cover some or all of that, in no particular order.
Three ways these tools approach the job
Full observability and evaluation platforms
Arize, Braintrust and Langfuse cover tracing, evals and experiments together, which is closest to the all-in-one approach you're leaving. One of the three is open source, which matters if you need to host it yourself. When you compare them, look closely at how each handles experiments, because that's where day-to-day work happens once tracing is set up.
Prompt-first tools
PromptLayer and Lunary start from prompt management, then add tracing and monitoring. They suit teams where prompts change daily and several people edit them, including people who don't write code. If non-engineers write prompts in your team, test the editing screen with one of them, not only with a developer.
Reliability loops and gateways
Traceloop links evals and monitors into a loop around each release. Portkey works as a gateway in front of your model calls, so it's a different shape of tool: it sees traffic because everything flows through it. A gateway can also sit alongside an evaluation tool rather than replace it, so you don't have to pick one shape for everything.
The question that decides the switch
Galileo's own site calls it an observability and evaluation platform for AI. Most of the alternatives say something similar, so the headline won't help you choose. What will is the effort it takes to start: how calls are traced, how they're scored, and how much of your code you'll change to get there.
Some tools ask you to wrap calls with their SDK. Some read standard tracing data. A gateway asks you to send traffic through it. Each route has trade-offs for speed of setup, lock-in and what you can see. Tools here split by who they're built for and how much you set up yourself, so put the same real feature through each before you commit.
Data handling is the other check. Traces can hold user messages, documents and personal details, so ask where each tool stores them, whether you can mask sensitive fields, and whether there's a self-hosted option if your data has to stay in-house. Rules on this vary by country and sector, so check with whoever leads data protection before you send production traffic anywhere new.
Seven tools from A to Z
Alphabetical, in no particular order, each described in line with the vendor's own words.
Arize
Best for: AI teams wanting observability, evaluation, tracing and experiments in one place to keep improving their agents, which is how the vendor frames its platform.
Braintrust
Best for: Teams shipping agents at scale who want production tracing, evals and regression checks together; the vendor's angle is catching problems before users see them.
Langfuse
Best for: Engineers who prefer open source and want to trace, evaluate and improve agents from production data, with a shared space to work on fixes, says the vendor.
Lunary
Best for: Smaller teams building LLM apps who want observability and prompt management together; the vendor's pitch is building agents with confidence.
Portkey
Best for: Organisations putting generative AI into production across many teams; the vendor says the product now runs under the PRISMA AIRS AI Gateway name, and a free plan is named.
PromptLayer
Best for: AI engineering teams that want to version prompts, run evals and monitor agents with tracing, logs and regression sets, as the vendor describes it.
Want AI doing the heavy lifting in your marketing?
I build the systems that handle the boring 80 percent, so you get your week back. Done properly, with the human kept in.
Traceloop
Best for: Developers who want evals and monitors linked so each release improves on the last, which is the vendor's main claim.
Side-by-side summary
| Tool | Approach | Good for |
|---|---|---|
| Arize | Agent observability and evals | Tracing, evaluating and experimenting on AI agents |
| Braintrust | Observability for agents | Catching regressions before they reach users |
| Langfuse | Open source tracing and evals | Open source tracing using production data |
| Lunary | LLM observability and prompts | Monitoring and prompt management for LLM apps |
| Portkey | AI gateway | Routing model calls through one production gateway |
| PromptLayer | Prompt management and evals | Versioning prompts and running evals as a team |
| Traceloop | LLM reliability platform | Turning evals and monitors into a feedback loop |
Advice for AI teams changing tools
Keep your evaluation data portable. Your test sets, scoring rules and labelled examples took effort to build, and they're worth more than any dashboard. Before you sign up anywhere, check that you can export them and bring them somewhere else later.
If you're a small team shipping one product, a lighter tool with prompt management may cover everything. If several teams run agents in production, a full platform or a gateway gives you one place to see it all.
Bring the people who'll use it into the trial. Engineers care about setup and SDKs, product managers want to see quality trends, and whoever handles support needs to look up a single bad answer quickly. A tool that only one of those groups likes won't stick. For more on the field, see my guide to LLM evaluation tools and my list of prompt management tools.
Instrument one feature first
- Choose one live feature that makes model calls and has known problem cases.
- Add tracing for that feature in two shortlisted tools, noting how much code each one needed.
- Run your existing test set through both and compare how scores and failures are shown.
- Ship a small prompt change and check whether each tool flags any drop in quality.
- Roll out the tool your engineers found easiest to live with, one feature at a time.
Do you make LLM observability tools? Tool makers can ask to be considered for a featured place on this list.
Want your tool on this list? Featured spots and reviews here are paid partnerships. Here's how to partner with me.
Frequently asked questions
What is LLM observability?
It's the practice of recording each model call in an AI app, with its inputs, outputs, timing and errors, so you can see what happened when something goes wrong.
What's the difference between tracing and evaluation?
Tracing records what happened. Evaluation judges whether it was good, using test sets, scoring rules, human review or another model acting as a judge.
Do I need an observability tool for a small AI feature?
Not on day one, but you'll want one once real users rely on the feature. Without it, bad answers are hard to spot and harder to explain.
Is open source a good option for LLM observability?
It can be, especially if your data must stay on your own servers. You'll take on hosting and updates, so weigh that against a managed service.
Will switching tools break my app?
It shouldn't, if you move one feature at a time. Keep the old tracing in place until the new tool shows the same calls, then remove it.
Does AI monitoring help with regulation?
Records of model behaviour can support compliance work, but rules on AI vary by country and sector. Check what your regulator expects.
