If you're replacing your current platform, start with what you need to keep running: prompt versions, evaluations and production tracing. Prompt managers, evaluation platforms and AI gateways each cover a different slice, so match the slice to your work, check how much code changes to get started, and move the most used prompts first.
Humanloop's homepage now carries one message above everything else: the company is joining Anthropic to speed up the safe adoption of AI. For teams who built prompt workflows and evaluations on it, that raises a practical question about what to use next. This page lists eight LLM observability and evaluation tools, groups them by the job they do, and ends with a sensible order for moving.
What the move means for your team
When a platform's homepage leads with an acquisition rather than its product, it's sensible to plan your next step rather than wait and see. That doesn't mean panic. It means getting clear on what you rely on today, so you can judge every alternative against the same list.
The thing to test in any replacement is how model calls are traced and evaluated, and how much code you change to start. Some tools ask you to wrap calls in their own library; others sit in front of your model provider as a gateway; a few lean on open standards so you can switch again later with less work. Tools here split by who they're built for and how much you set up yourself, so run the same real task through each before you decide.
Gather these before you shortlist
- Every prompt in production, with its current version and the reason for its last change.
- Your evaluation datasets: the inputs, the expected outputs and any human labels.
- The scoring rules you use to judge an answer as good or bad.
- A short list of the failures that worried you most, so you can check a new tool would have caught them.
- Who uses the platform: engineers only, or product and subject experts too.
Eight tools by the job they do
Prompt management first
PromptLayer and Lunary both pair prompt management with monitoring. If your team spends most of its time versioning and reviewing prompts, these are the closest in spirit to what you're leaving.
Evaluation and observability platforms
Arize, Braintrust, Galileo and Traceloop focus on tracing, evals and catching regressions. Their homepages differ in tone: some talk about experiments, some about stopping failures, one about a continuous feedback loop between evals and monitors.
A gateway in front of your models
Portkey sits in front of model calls for a whole organisation, and its homepage notes a new product name. A gateway suits companies with many teams calling models in different ways.
Open source and self-hosted
Agenta describes an open source workspace for building and improving agents, which you can run yourself or in the cloud.
The list, A to Z
In no particular order, sorted alphabetically. Every line reflects the vendor's own description.
Agenta
Best for: Teams that want an open source workspace to build agents, improve them with feedback and share them across the company, self-hosted or in the cloud, per the vendor.
Arize
Best for: AI teams improving agents over time with observability, evaluation, tracing and experiments in one platform, which is how the vendor describes it.
Braintrust
Best for: Engineering teams shipping agents who want production tracing, evals and regression checks before changes reach users, as the vendor puts it.
Galileo
Best for: Teams who want evaluation and observability aimed at stopping AI failures rather than only monitoring them, in line with the vendor's own pitch.
Lunary
Best for: Builders of LLM-based apps who want observability and prompt management side by side, in what the vendor calls a platform for building agents with confidence.
Portkey
Best for: Organisations routing generative AI across many teams through one gateway; the vendor's homepage says the product is now PRISMA AIRS AI Gateway.
Want AI doing the heavy lifting in your marketing?
I build the systems that handle the boring 80 percent, so you get your week back. Done properly, with the human kept in.
PromptLayer
Best for: AI engineering teams that version prompts, run evals and monitor agents in production with tracing, logs and regression sets, per the vendor.
Traceloop
Best for: Developers who want evals and monitors joined into one feedback loop so each release improves on the last, which is the vendor's main promise.
Side-by-side summary
| Tool | Main focus | Good for |
|---|---|---|
| Agenta | Open source agent workspace | Self-hosted or cloud agent building with team feedback |
| Arize | Agent observability and evaluation | Tracing, evaluating and experimenting on AI agents |
| Braintrust | AI observability and evals | Tracing production and catching regressions early |
| Galileo | AI observability and evaluation | Spotting and stopping AI failures, not just logging them |
| Lunary | LLM observability and prompts | Monitoring LLM apps and managing prompts together |
| Portkey | AI gateway | One gateway for model calls across a company |
| PromptLayer | Prompt management and evals | Versioning prompts and monitoring agents in production |
| Traceloop | Evals and monitoring loop | Turning evals and monitors into release feedback |
Advice on picking an LLM eval tool
Don't choose on dashboards. Every tool in this category has a good-looking trace view, and screenshots tell you little about daily use. Choose on two things: how quickly you can get one real production flow traced, and whether your existing evaluation set runs in the new tool and gives you results you trust.
Involve the people who write and review prompts, not just engineers. If subject experts helped judge outputs on your old platform, they need a way to do the same here. For deeper lists, see my guides to the best LLM observability tools, LLM evaluation tools and prompt management tools.
A sensible migration order
- Pick one production flow that matters and has a clear evaluation set.
- Trace it in two shortlisted tools for a few days, using a free plan or trial where one exists.
- Rerun your evaluation set in both and compare results with what your current scores say.
- Move prompt versions for that flow and confirm your team can review and approve changes.
- Only then move the remaining flows, one at a time, keeping your old exports as a backup.
Do you make LLM observability or evaluation software? Tool makers can ask to be considered for a featured place on this list.
Want your tool on this list? Featured spots and reviews here are paid partnerships. Here's how to partner with me.
Frequently asked questions
What is LLM observability?
It means recording what happens inside an AI app: the prompts sent, the model's replies, the tools called and how long each step took. That record helps you debug problems and spot quality drops.
What's the difference between evals and monitoring?
Evals test a model or prompt against a set of examples before or after a change. Monitoring watches live traffic in production. Most teams need both, joined up.
Do I need an AI gateway?
Not always. A gateway helps when many teams call different models and you want one place for routing, logging and controls. A small team with one app can often skip it.
Can I self-host an LLM observability tool?
Some vendors offer open source or self-hosted options, which suit teams with strict data rules. Check each vendor's own site for what's available and how it's supported.
How do I keep prompts and test data safe when switching?
Export prompts, datasets and labels to your own storage before you start, and limit who can access them. Data rules vary by country and sector, so check your regulator's guidance.
How much code changes when I switch?
It depends on the tool. Some need a small wrapper around your model calls, gateways need a change of endpoint, and others use open standards. Test with one flow before committing.
