- Where the tracker sits decides what it can attribute
- Following one customer's spend through the stack
- Mistakes that make the spend figures lie
- What LLM cost tracking costs
- Gateways and dashboards for engineering teams
- Metering and FinOps platforms that tie spend to products and customers
- Which kind to pick
Put a gateway in the request path if you need to cap LLM spend before it happens and split it by customer or feature. If finance only needs the monthly bill carved up by team, a FinOps platform that imports provider and cloud bills will do, and if you resell AI to your own customers, a usage-metering tool ties each call to the account that pays for it.
A gateway suits engineering teams who want hard caps and per-request detail; a billing-side FinOps platform suits finance teams reconciling the invoice. Product companies that charge customers for AI usage need a metering layer, because margin per account is the number they'll be asked for.
Why does the position matter so much? A tracker can only attribute what it can see. Your provider's invoice knows the API key, the model and the day. It has no idea that Tuesday's spike came from one customer's agent stuck in a retry loop. Whatever sits closest to the request sees the most, and whatever reads the bill afterwards sees the least, however good its charts look.
Where the tracker sits decides what it can attribute
There are three places to put cost tracking, and most teams that get serious end up running two of them side by side.
| Where it sits | What it can see | What it can't see | Can it stop spend? |
|---|---|---|---|
| Gateway in the request path | Every call as it passes: model, tokens in and out, latency, and any customer or feature tags your code attaches | Calls that skip it, such as a script holding a provider key directly | Yes, it can refuse or reroute a call once a budget is hit |
| SDK or log import | Whatever your app records next to each call, often the richest metadata because your own code writes it | Anything you forgot to log, plus discounts and credits that only appear on the invoice | Only if your code checks a budget before it calls the model |
| Billing layer (provider or cloud bill import) | The real invoiced amount by key, project, model and day, beside the rest of your cloud spend | Which customer, feature or prompt drove a line, unless keys or projects were split that way upfront | No, it reports after the money's gone |
So the billing side is your record of what you paid, and the gateway or SDK is your record of why. Teams with only the bill spend month-end meetings guessing. Teams with only a gateway find its total and the invoice disagree and can't say which to trust. Picking the routing layer is a decision in its own right, and the comparison of AI gateway tools covers it.
Following one customer's spend through the stack
Take an invoicing app with three AI features. Payment reminder emails are drafted on a small, cheap model. Uploaded contracts are reviewed on a large one. A help chat answers product questions. The provider invoice arrives as one figure per model, and finance wants to know whether a single big account, call it Northwind, is still profitable.
The work starts in code, before anyone buys a tool. Every request carries two tags: the customer ID and the feature name. Through a gateway they usually travel as request metadata or sit on a virtual key issued per customer; with an SDK they're fields on the log line. Skip this and nothing downstream can split the bill, whatever the dashboard promises.
With tags in place, the tracker prices each call: input tokens at the model's input rate, output tokens at its output rate, and cached input at whatever discount the provider applies. Grouped by customer and feature over a month, Northwind looks lopsided. Reminder emails make up most of its requests and a sliver of its cost, since the prompts are short and the model is cheap. Contract review is a small share of requests yet more than half its spend, because each call ships a long document to the large model. The help chat lands in between.
That's something you can act on. You could cap contract reviews per account, send the first pass to a smaller model, or turn contract review into a paid add-on. None of those choices would ever surface from the invoice alone.
Before anyone trusts the split, reconcile it. Add up every tagged call for the month and set the total beside the invoice. The gap is untagged traffic: staging keys, evaluation runs, background jobs and retries that never reached your logging. A small gap means the per-customer view holds. A wide one means fixing the tagging comes before any pricing decision.
Mistakes that make the spend figures lie
Quoting the gateway's estimate as the bill is the commonest one. Request-path trackers price calls from list rates, while negotiated discounts, batch pricing and credits only land on the invoice, so the two rarely match to the cent. Agree upfront which number finance repeats.
A provider key that lives outside the gateway is the next leak. One notebook or scheduled job calling the model directly is invisible to anything in the request path, and it's often the job that runs all weekend. Revoke direct keys, or give them a project of their own so the billing layer can at least isolate them.
Tag by feature, too, and not only by user. A per-customer total tells you who's expensive; it won't tell you what to change. Budgets need an owner as well: an alert dropped into a shared channel at midnight gets ignored, so tie each limit to a team or a virtual key with a named person behind it.
Ownership deserves a question during any trial. Several observability products in this space now sit inside larger companies, according to their own sites, and one gateway now goes by a different product name entirely. Ask who'll run the product next year and how you'd export your history. If you also need prompts, outputs and traces per call, that's a wider job, and the guide to LLM observability tools is the better starting point.
What LLM cost tracking costs
The pricing models differ more than the price tags. A hosted gateway can add a percentage on top of what you pay the model provider, which feels painless early and grows with every token. Open-source gateways are free to run yourself, with paid enterprise tiers sized to request capacity rather than tokens, so the true cost is the engineer who keeps the thing patched. Metering tools sell transactions in blocks. FinOps platforms charge a flat monthly tier, a yearly fee pegged to your cloud spend, or a quote.
Where a tracker's pricing page gives a figure, the spread runs from nothing for a self-hosted open-source gateway to the high hundreds of dollars for each extra block of metered transactions.
For Bifrost, AI Cost Board, Ascerta, Stigg, CloudZero and Finout, the pages checked gave no price.
Do the markup sum before the subscription sum. At volume, a percentage on top of a large model bill can outgrow a flat fee, and if the bill itself is the problem, comparing LLM API providers on token rates may save more than any tracker will.
Want AI doing the heavy lifting in your marketing?
I build the systems that handle the boring 80 percent, so you get your week back. Done properly, with the human kept in.
Gateways and dashboards for engineering teams
Requesty
A hosted gateway across hundreds of models, where spend tracking, spend limits and budget caps come with the routing rather than as a separate product.
LiteLLM
Teams who'd rather run their own gateway reach for this open-source option, which issues virtual keys and tracks spend and budgets across more than a hundred providers.
Bifrost
Maxim AI's open-source gateway, a fit for engineering orgs that want spending limits set per team or per virtual key.
AI Cost Board
If one hosted dashboard showing tokens, costs, latency and errors across the major providers is all you're after, this covers that ground.
Metering and FinOps platforms that tie spend to products and customers
Revenium
Built for metering, it attributes AI spend to the team, feature and customer behind each call and raises alerts when budget thresholds are crossed.
Stigg
For AI products that meter what customers use, this usage and entitlement layer applies limits, credits and budgets inside each agent run.
Ascerta
An enterprise platform aimed at measuring the return on generative AI and isolating wasteful spending.
Vantage
Multi-cloud cost management with LLM token allocation, so model spend lands in the same reports as the rest of your cloud bill.
CloudZero
Finance and product leads use this allocation platform to tie model and provider costs to the products and customer segments they serve.
Finout
Covering cloud and AI spend together, this FinOps platform adds virtual tags for allocation and anomaly detection for unexpected jumps.
Which kind to pick
A small team shipping its first AI feature should start with an open-source gateway in front of every call and tag customer and feature from day one; it's the cheapest route to attribution and hard caps together. Already on a hosted router? Use its built-in spend limits before adding another tool.
Do you make LLM cost trackers, AI gateways or AI FinOps tools? Tool makers can ask to be considered for a featured place on this list.
Selling AI usage to your own customers changes the answer. A metering or entitlement layer earns its place there, because it connects what each account burns to what you charge it.
When the question comes from finance and the AI bill sits beside a large cloud bill, a FinOps platform with LLM allocation gives one report for both, provided someone has split keys or projects by team. Once spend is large enough to argue about, run both: a gateway that explains why, and a billing-side platform that confirms what was paid.
Want your tool on this list? Featured spots and reviews here are paid partnerships. Here's how to partner with me.
