Choose a replacement by the layer you need most. If you mainly debug model calls, a tracing tool comes first. If quality before each release worries you more, look at evaluation platforms, and if you want every model request to pass through one control point, a gateway fits. Seven options follow, in no particular order.
There are three layers to this category, and most teams leaving PromptLayer only need one or two of them. PromptLayer's homepage covers all three: versioning prompts, running evaluations and monitoring agents in production with tracing and logs. The alternatives below tend to lead with one layer, so it helps to know which one hurts most in your current set-up before you start comparing.
The seven at a glance
| Tool | Focus | Good for |
|---|---|---|
| Agenta | Open source agent workspace | Building and sharing agents with the whole team |
| Braintrust | AI observability and evals | Catching regressions before users see them |
| Galileo | Observability and evaluation | Stopping AI failures, not only logging them |
| Langfuse | Open source tracing and evals | Tracing and evaluating agents on an open platform |
| Lunary | Observability and prompt management | Prompt management and monitoring for LLM apps |
| Portkey | AI gateway | Routing an organisation's model traffic through one gateway |
| Traceloop | Evals and monitoring loop | Turning evals and monitors into release feedback |
How much code changes on day one
The thing to test in any replacement is how calls are traced and evaluated, and how much of your own code you'll change to get started. Some tools wrap your model client or plug in through an SDK, so a few lines change and traces start flowing. Others sit in front of your model provider as a gateway, which can mean changing endpoints and keys across every service that calls a model.
Open source matters here too. Langfuse and Agenta both describe themselves as open source, and the second can be self-hosted, which helps if prompts or user data can't leave your environment. Hosted platforms such as Braintrust, Galileo and Traceloop focus on catching quality problems before release, each with its own angle on evaluation. Lunary pairs prompt management with monitoring for teams that want both in one place.
Two names may surprise you. Humanloop's homepage now says it's joining Anthropic, so it isn't included as an option here, and Portkey's site says the product is now called PRISMA AIRS AI Gateway. Tools in this category split by who they're built for and how much you set up yourself, so run the same real task through each before you decide.
Seven tools, sorted A to Z
Alphabetical and in no particular order of preference. Every line reflects the vendor's own description.
Agenta
Best for: Teams building agents through chat and improving them with feedback; the vendor says it's open source and runs self-hosted or in the cloud.
Braintrust
Best for: Teams shipping agents to production who want tracing, evals and regression checks, which the vendor says catch problems before they reach users.
Galileo
Best for: Organisations that want evaluation alongside monitoring; the vendor's pitch is stopping AI failures rather than just watching them.
Langfuse
Best for: Developers who prefer an open source platform to trace and evaluate agents using production data, according to the vendor.
Lunary
Best for: Builders of LLM-based apps who want observability and prompt management in one product, as the vendor describes it.
Portkey
Best for: Organisations putting generative AI into production across teams; the vendor's site says the product now goes by PRISMA AIRS AI Gateway.
Traceloop
Best for: Engineering teams who want evaluations and production monitors feeding each other so each release improves, per the vendor.
Advice for AI teams weighing a new platform
Don't choose from the dashboard screenshots. Pick one feature that already runs in production, ideally one with a known failure you've had to debug, and wire it up to two shortlisted tools. You'll learn more from how each one shows that failure than from any demo.
Want AI doing the heavy lifting in your marketing?
I build the systems that handle the boring 80 percent, so you get your week back. Done properly, with the human kept in.
Think about who'll use the tool, too. Engineers want traces and quick debugging; product managers and domain experts often want to edit prompts and review outputs without touching code. Some tools serve both groups well and some are built mostly for one. A shared prompt workspace, like the chat-based agent building one vendor here describes, helps non-engineers join in; a tracing-first tool keeps engineers happy but may leave everyone else reading exports. Decide which group you're choosing for, and ask the other group to sit in on the trial anyway, because they'll spot gaps you won't. Check where data is stored, how long it's kept and whether personal data in prompts is handled the way your policies require; rules vary by country, so check with your regulator if you're unsure. If your team is still building its prompt skills, my guides to prompt engineering basics and writing better ChatGPT prompts are a good starting point.
Before you rip out your prompt registry
Prompts are easy to lose in a move. Export every prompt version you rely on, with the model and settings each one was tested against, and keep a copy outside any vendor's system. Rebuild your evaluation sets in the new tool before switching traffic, so you can confirm that results match what you saw before.
Run old and new tracing side by side for a short period, compare a sample of traces from each, and only then remove the old SDK. It's slower than a clean cut, but it means nobody discovers a gap in your logs on the day something goes wrong in production.
Do you make LLM observability or evaluation tools? Tool makers can ask to be considered for a featured place on this list.
Want your tool on this list? Featured spots and reviews here are paid partnerships. Here's how to partner with me.
Frequently asked questions
What is LLM observability?
It's the practice of logging and tracing every call to a language model, with inputs, outputs, timing and errors, so you can debug problems and see how an AI feature behaves in production.
What's the difference between tracing and evals?
Tracing records what happened on each call. Evals score outputs against tests or criteria, often before a release, so you catch quality problems before users do.
Do I need an AI gateway?
A gateway helps if many teams call several model providers and you want one place for routing, keys and controls. A single app calling one model may not need one.
Can I self-host an LLM observability tool?
Some open source tools can be self-hosted, which keeps prompts and user data inside your own environment. Check the docs for what the self-hosted version includes.
Is it safe to log prompts that contain customer data?
Only if your policies allow it. Check where logs are stored, how long they're kept and whether personal data can be masked, as requirements vary by country.
How do I compare these tools fairly?
Connect the same production feature to two tools and compare how each shows a known failure. Homepage demos won't tell you how a tool handles your own traffic.
