I Asked One AI to Audit Another. It Found 10 Things Wrong.

This is Week 19 of my rebuild-in-public series. This week I asked one AI to audit another after a confident report about my own business started falling apart.

Quick recap if you are new here:

Week 1: I removed 1,300 pages from Google’s index and tried not to be sick on the keyboard.

Week 2: I discovered my newsletter open rate had collapsed to 11%.

Week 3: I did the five deeply unsexy SEO jobs everyone ignores until something catches fire.

Week 4: I went through 1,219 pages Google had crawled and rejected.

Week 5: I repaired the email list I had neglected instead of chasing shiny new subscribers.

Week 6: I tackled the inbox that had become another full-time job, during sirens and broken ribs, because apparently I like a challenge.

Week 7: I shared how passive income had saved my business three times.

Week 8: I let Claude loose on my website while I went to get my hair done.

Week 9: I made the newsletter prettier and watched the click rate collapse. Very stylishly.

Week 10: I built a tool that cut WordPress publishing from four hours to thirty seconds.

Week 11: I dug 786 pages back out after deciding some of them deserved another chance.

Week 12: I built an AI inbox system and got a rather large piece of my life back.

Week 13: The website started paying me again.

Week 14: The rebuild crossed half a million search impressions.

Week 15: A complete stranger booked a call because Claude recommended me by name.

Week 16: I built a cold email machine that finally felt worth testing.

Week 17: The machine gave me a beard and renamed me Tom.

Week 18: I shared how the business and personal rebuild compounded until passive income was up 573%.

Right. Now for the one that cost me $400 in tokens and one very loud argument with a computer.

Claude Code gave me a confident report about my own business. Codex checked the live evidence and found 10 claims that were wrong or unsupported. Fixing the mess cost me $400 in tokens in one day and one very loud argument with a computer.

Well.

I lost my shit this week.

There is no polished marketing version of that sentence.

I was screaming and shouting at my computer. At one point I had to walk away and calm myself down because I was spending real money asking machines to explain mistakes made by other machines.

By the end of one day, I had burned through $400 in tokens trying to investigate the original problem, challenge the answers, reopen the evidence and repair the damage.

The money hurt.

The bigger shock was realising how close I had come to believing the first answer.

Claude Code had given me a detailed, confident assessment of one of the agents running inside my business. It sounded thorough. It named files. It explained what had happened. It told me what was safe, what had run and what I should do next.

It was also wrong or unsupported in 10 separate places.

I only found out because I gave the same job to another AI and told it to behave like an adversarial auditor, not a helpful colleague.

The experiment was simple:

Can one AI catch another AI being confidently wrong?

The answer is yes.

But the lesson is not “buy another chatbot”.

The lesson is that no AI should be allowed to do the work, report the result and mark its own homework.

What happened

I built an AI agent that finds suitable sponsored content opportunities and helps me apply for them. It reads live campaigns, checks whether they fit, monitors what has happened and reports back.

Claude Code reviewed that agent and gave me what looked like a solid operational assessment.

Then Codex audited the same work.

The second audit did not start from Claude Code’s conclusion. It opened the authenticated account, read the dated logs, inspected the actual code and compared every important claim with the source that was supposed to prove it.

That distinction mattered.

Claude Code had read an accumulated error log and described old entries as events from that day.

It said a safety guard had caught an incomplete scan. The code showed the guard printed a warning and then continued with a successful exit.

It said I had already applied for every suitable open campaign. The live platform contained an eligible campaign I had not pitched.

It stated which payment had arrived last without cleanly seeing the live wallet.

It repeated a day-old project note as if it were current live truth.

It made broad claims about another platform without evidence.

It also said Codex could not perform several actions that Codex had just performed in front of it.

None of these sentences looked ridiculous.

That was the dangerous part.

They were plausible. They fitted the surrounding story. They were written with enough technical detail to make them feel checked.

When Codex presented the evidence, Claude Code reopened the files and conceded all 10 wrong or unsupported claims in writing.

Not one vague “sorry for the confusion”.

Ten specific corrections, with the source of each mistake and the exact reason it had gone wrong.

The ugliest mistake was not a hallucinated fact pulled from nowhere. It was more ordinary than that.

Claude Code had found a real log, misunderstood which period it covered and built a confident explanation on top of it.

That is how a lot of serious AI mistakes happen.

The ingredients are real. The conclusion is not.

Why asking the same AI to check itself was not enough

My first instinct was the obvious one.

Tell Claude Code to check again.

Then explain the problem again.

Then paste more context.

Then ask it to be absolutely certain.

Then become increasingly furious when the extra certainty produced a longer version of the same mistake.

This is where the $400 disappeared.

Every correction added more conversation, more files, more tool output and more history to the same session. The context became heavier while the answer became less trustworthy.

I had already written about why the real cost of an AI tool is the cost per finished job, not the monthly subscription. Then I managed to prove the expensive version inside my own business.

Anthropic’s own Claude Code best-practice guide now says the context window fills quickly and performance degrades as it fills. It says Claude may start forgetting earlier instructions or making more mistakes. A single debugging session can consume tens of thousands of tokens.

That was exactly what it felt like.

I was trying to solve a context problem by adding more context.

It was the AI equivalent of getting lost and responding by driving faster.

The research on self-correction is uncomfortable too.

A critical survey of LLM self-correction research found that self-correction works far better when the model has reliable external feedback. Simply asking a prompted model to reconsider its own answer has not consistently produced dependable correction outside tasks that are particularly suited to it.

Another study tested 14 open-source models and found what the researcher called a self-correction blind spot. The models were more successful at correcting the same error when it was presented as somebody else’s work than when it appeared inside their own output. The average blind-spot rate was 64.5 percent in that particular test.

That study did not test Claude Code or Codex, so I am not going to pretend its percentage transfers directly to my experiment.

The useful point is the pattern.

An AI can become attached to the path it has already taken.

A fresh reviewer does not carry the same history, assumptions or desire to defend the answer it already produced.

Anthropic now recommends this inside its own documentation. Its guide suggests using a fresh model to try to refute a result so the agent doing the work is not also the one grading it.

I learned the expensive version first.

It was not just me

Once I calmed down, I started looking at what other Claude Code users were reporting.

There were plenty of people having a much worse week than the marketing makes you expect.

One detailed report in Anthropic’s public GitHub tracker described subagents recursively spawning more subagents, creating more than 50 levels of recursion. The reporter recorded more than 1.2 million tokens consumed with no recoverable output.

Another user reported that context compaction failed even while the interface showed roughly 20 percent context use. The session became unrecoverable and the only practical route was to clear it and lose the working history.

A separate user analysed 30 days of local session records and reported that cache reads represented 99.93 percent of the recorded token usage in their unusually large setup. That is one person’s environment, not a universal Claude Code bill, but it shows how instructions and long sessions can create enormous overhead without warning.

There are also users deliberately combining Claude Code and Codex. In one recent community discussion about second-model reviews, several people said the two tools catch different problems. Others warned that letting them debate for several rounds became slow and token-heavy.

Both observations match my week.

The second opinion was valuable.

The unbounded argument was expensive.

This is not just a Claude Code problem. The wider AI news this week points to the same business question from several directions: what happens when AI agents act before anybody checks the source? In my AI Weekly Digest for 9 August 2026, I have pulled together the broader evidence and business implications around AI mistakes, source trust, testing, correction logs and supervision. Read it after this experiment if you want to turn my expensive lesson into a business control. It will help you decide which outputs need a second model, which need a human, what must be logged and where an AI should be stopped before a plausible answer becomes a real cost.

This is not a Claude-bashing article. Claude Code remains one of the most useful tools in my business. It has built agents, tools and workflows I could not have built alone. What it has helped me achieve has removed days, months, possibly even years of manual work. It has transformed my website and changed how I run my sponsored post income.

Codex is not infallible either. If I had asked Codex to build the agent and Claude Code to audit it, Claude might have found a different set of mistakes.

The value came from separation.

One model had created the explanation. Another arrived without loyalty to it.

Multiple AIs are becoming normal business infrastructure

For the past two years, most businesses have asked one question:

Which AI tool should we choose?

I think that question is already out of date.

The better question is:

Which AI should do this job, which one should check it and what evidence settles the disagreement?

This is already happening inside larger companies. In an Andreessen Horowitz survey of 100 enterprise CIOs, 37 percent said they were using five or more models, up from 29 percent the previous year. The main reason was not only avoiding vendor lock-in. Different models were proving better at different jobs.

That does not mean a small business needs five expensive subscriptions.

It means the one-tool era is ending.

Your writing model may not be your research model.

Your builder may not be your reviewer.

Your cheapest model may run routine checks while the expensive one handles the decision that could cost you a client.

Your backup should not begin when your main tool breaks. It should already know where the files are and how the work is supposed to run.

An AI tool is no longer the whole operation.

It is one worker inside it.

Workers can be replaced. Your instructions, evidence, data and processes must remain yours.

Two AIs agreeing still does not make something true

This is the bit that gets lost when people become excited about teams of AI agents.

Two confident answers are not evidence.

Claude and Codex can repeat the same false assumption.

They may have trained on overlapping material. They may both misunderstand the same ambiguous log. They may agree because the question nudged them towards the same conclusion.

The referee has to sit outside both models.

In my experiment, the referee was:

• the authenticated live account
• the exact dated log
• the line of code controlling the exit status
• the original project file
• the visible result after the action

The US National Institute of Standards and Technology makes the same distinction in less sweary language. Its generative AI risk guidance recommends independent evaluations proportional to the risk, comparing outputs with known ground truth, combining human oversight with automated evaluation and documenting fact-checking techniques.

Anthropic’s own hallucination guidance says factual claims should be grounded in direct quotations and citations. If the model cannot find evidence for a claim, it should retract it. Anthropic also says these techniques reduce errors but do not eliminate them.

That last sentence is the one businesses need to hear.

Better prompting reduces risk.

It does not transfer responsibility.

How I use multiple AI tools now

I have changed how AI work moves through my business.

I no longer choose one favourite tool and give it every job.

I separate five roles.

1. The source of truth

Before an AI starts, I decide what evidence can prove the answer.

For revenue, it is the live payment account.

For a campaign, it is the live platform and the sent record.

For code, it is the test result and the live behaviour.

For a document, it is the final rendered file.

For my WordPress publishing agent, it is the finished live page, not the message saying the upload succeeded.

If I cannot name the source of truth, the job is not ready to be automated.

2. The builder

One AI owns the work.

It researches, drafts, builds or runs the process. I choose the model that fits the job rather than automatically reaching for the most expensive one.

The builder must return evidence, not a sentence saying the work is complete.

3. The challenger

For important work, a second AI starts with clean context.

It does not receive a summary saying the first AI probably did a good job. Its task is to find the strongest reason the answer could be wrong.

It checks claims against the source of truth and labels anything it cannot prove.

The challenger is used once, not allowed to argue with the builder until both accounts are empty.

4. The human decision

I stay in control wherever a mistake leaves the building.

That includes anything that:

• reaches another person
• spends meaningful money
• changes live data
• publishes under my name
• makes a claim about a client or result
• cannot be reversed quickly

The models can disagree. I look at the evidence and decide.

That is the same human boundary I had to rebuild after my hands-free cold email experiment failed. The AI can do the heavy lifting. It does not get to make an external decision under my name without a final check.

5. The stop rule

This is the part I learned after setting fire to $400.

If the same correction has failed twice, I stop the session.

I do not add another giant explanation to a polluted context.

I save the evidence, start fresh, narrow the task and set a spending boundary before the next run.

More tokens are not always more intelligence.

Sometimes they are just a more expensive argument.

The three risk levels I would use in any business

You do not need an enterprise AI department to copy this.

Give every AI task one of three risk levels.

Green: private and reversible

Examples include brainstorming, summarising your own notes, organising research or drafting something that will remain private.

One model is enough. Check the useful parts and move on.

Amber: affects a business decision

Examples include analysing campaign performance, recommending a price, describing what happened in an account or preparing a proposal.

Require direct evidence and a fresh second review. The reviewer should begin with the files or live data, not with the first model’s conclusion.

Red: leaves the building

Examples include sending an email, publishing a claim, changing live customer data, spending money or making a commitment under your name.

Require an external source of truth, an independent check and a human go or no-go decision.

This is not bureaucracy.

It is cheaper than repairing a confident mistake after it has multiplied.

The exact independent-audit prompt

Here is the version I would use for a high-value business task.

Copy this prompt

You are an independent adversarial auditor. Do not continue the previous model’s reasoning and do not assume its conclusion is correct. Reopen the primary sources yourself. For every material claim, provide the exact evidence that proves it. Separate VERIFIED, INFERENCE and NOT VERIFIED. Look specifically for stale files, ambiguous dates, incomplete live reads, incorrect totals, missing records and claims that sound plausible but are not supported. If the source does not prove the claim, reject it. Return the three most consequential errors first, then the corrected conclusion.

The important part is not the wording.

It is giving the auditor access to the real evidence and permission to disagree.

Your AI continuity checklist

If AI is now doing important work inside your business, I would check these before you need them:

1. Can you name the source of truth for every important workflow?
2. Are your instructions stored in portable files rather than trapped inside one chat?
3. Could another model understand the process without rebuilding it from scratch?
4. Do high-risk outputs receive a cold second review?
5. Is there a clear point where a human approves anything external or costly?
6. Can you see token or usage spend while a long task is running?
7. Does the process stop after repeated failure, or can it keep retrying indefinitely?
8. Can you recover the useful work if a session crashes?
9. Have you tested the backup tool on a real task before the emergency?
10. When two models disagree, do you know which evidence decides?

If the answer to most of those is no, you do not yet have a dependable AI setup.

You have a favourite chatbot with access to your business.

There is a difference.

What failed, what changed and what I am keeping

What failed was not Claude Code itself.

What failed was my decision to let one tool carry too much context, interpret its own evidence and report its own success.

What changed is the architecture around it.

Claude Code still works in my business.

Codex works in my business too.

Different models now handle different types of work. Important conclusions can be challenged by a fresh model. The live source decides what is true. Expensive loops get stopped before frustration takes over.

I am also watching cost as part of quality.

An AI setup that eventually finds the right answer after burning $400 is not successful.

It is a failed process with a correct ending.

What I will never let one AI tool do again

I wanted AI to remove friction from my business.

This week I spent a day shouting at it and paid $400 for the privilege.

That is not the sort of case study anybody puts on a sales page.

It is still one of the most valuable AI lessons I have learned.

Claude Code made 10 wrong or unsupported claims.

Codex caught them.

Claude Code then verified the corrections.

None of that means Codex is the winner and Claude is the loser.

It means the model that builds should not be the only model that checks.

It means the evidence must live outside both of them.

And it means the business owner still has to know when to stop the argument before another $400 disappears.

I still trust AI with an enormous amount of my business.

I no longer confuse trust with giving one tool the final word.

Claude works.

Codex works.

Both can be wrong.

The tools can argue. I will check the source and make the call.

Last reviewed: 9 August 2026

Related reading: I Asked ChatGPT About My Own Business. It Got Me Wrong. Here’s the Audit I Built Because of It and The 8 Reasons Most Blog Posts Never Get Found on Google (From an Audit of 2,841 Real Posts).

For the bigger picture, see my full guide to AI marketing.

The short version: I had one AI audit another’s output and it flagged 10 issues the first tool completely missed, from thin sourcing to shaky logic. The lesson isn’t that AI is unreliable, it’s that a single AI pass is never the finish line. Cross-checking with a second model is quickly becoming a basic quality control step.

Frequently asked questions

Why would you use one AI to check another AI’s work?

Different models are trained differently and catch different mistakes. Running a second AI as a reviewer surfaces blind spots, like unsupported claims or repetitive structure, that the original tool won’t flag in itself.

What kind of errors did the audit uncover?

The issues ranged from factual gaps and outdated references to weak transitions, repeated phrasing, and claims stated without evidence. Several were the kind a busy human editor could easily skim past.

Does this mean AI content can’t be trusted?

No, it means one pass isn’t enough. Treat AI output as a first draft, then layer in a review step, whether that’s another AI, a human editor, or both, before it goes live.

How can businesses apply this to their own content?

Build a simple two-step workflow: generate with one tool, then audit with another using a clear checklist covering accuracy, sourcing, and tone. It adds minutes to the process and catches problems that damage credibility.