Last week I turned five AI agents loose on every internal document in my business. All 195 of them. Go find everything that is broken, I said, like a person who has never once regretted asking a question.
They came back with eight items causing live harm, fourteen that would have caused wasted work, ten unresolved contradictions, and sixty one documents that were completely fine.
I read the top three, felt slightly sick, and went to check them by hand.
All three were wrong.
Not wrong in a fun way either. Wrong in the specific direction that would have sent me off to fix things that were already fine, which is the most expensive kind of wrong there is, because it feels EXACTLY like being productive.
Finding one: I was publishing bad data about real people
This was the one that made me put my coffee down.
The audit said twenty public pages on one of my sites were showing inaccurate information about actual local businesses. Other people's names. Their contact details. Their numbers.
If that had been true it would have been the worst thing I did this year, and possibly a lawyer's phone number.
So I opened the live pages and I queried the database sitting behind them, side by side, one at a time. Contact form found, seven form fields, HTTP 200. Click to call above the fold, yes. Fourteen percent, with a confidence interval printed politely next to it.
Every single value matched. All twenty pages. They had been rebuilt from fresh data days earlier.
The audit had also named two specific companies as being publicly shamed on those pages. Those two companies are not published. They have never been published. They are sitting in the database with the switch firmly OFF, and my robots had them out in the town square with a sign around their neck.
Finding two: my homepage was quoting a broken number
The audit told me my homepage was publishing two statistics that came from a check I already knew was busted. Six percent of contact forms failing. Seventy five percent of sites with no tap to call.
I loaded my own homepage, which I want to note is a thing I could have done before I got upset.
It says three percent. And thirty nine percent.
Those are the CORRECTED numbers, from the fixed check. The seventy five percent figure was real once. It came from a check that was, and I promise this is what it was doing, reading the styling instructions on the page instead of the page. Like grading an essay by looking at the font. It got caught, it got fixed, and the number had been gone from that page for a while.
Finding three, which is small and which I include for completeness
The audit said I had two price increase emails sitting in my drafts folder, waiting to go out.
I had one.
The other one was a talking point for a phone call, written down in a document, that never became an email and was never going to. My robots read a note about a conversation and confidently upgraded it to correspondence.
Okay so why were they wrong
Here is the part that is actually worth your time, and it is not "AI is unreliable," which is the boring answer and also not what happened.
The agents did their job perfectly. They read my documents. My documents described a business that had already been fixed.
And not one single document recorded the fix.
Every one of those three findings was completely accurate on the day the document describing it was written. Then the world moved. The document did not. And there was no newer document contradicting it, because every time I fixed one of those things I fixed the thing, felt finished, closed the laptop, and never once went back to correct the note.
That is not an AI problem. That is a me problem, and I would put money on it being a you problem too.
The bit that genuinely rattled me
I went and looked at the fourteen items in the "would have wasted your time" pile, expecting a mess.
Every single one had already been solved, correctly, in a document written later. The right answer existed. Somebody, and that somebody was me, had already worked it out and written it down properly.
And in every single case, the original wrong document was still sitting there completely untouched.
So I had pairs. A confident, thorough, beautifully titled document saying one thing. And a scrappier, later document saying the opposite and being correct. Both in the same folder. Neither pointing at the other. Both looking equally trustworthy at a glance.
Now guess which one gets found first, six months from now, by a new hire or a contractor or a robot or me on four hours of sleep.
It is the older one. It is ALWAYS the older one, because the older one has the better title and more detail and more confidence in it, since it was written on a day when somebody had time to write a proper document instead of a hurried correction.
The wrong answer wins the search. That is the whole thing, in one sentence, and it compounds quietly every month nobody deals with it.
A completely serious offer buried in a post about robots
You are still here, which statistically makes you unusual, so here is a thing.
I will run my full visibility dashboard on your business, for free, for the first three people who ask. That is the paid one, the whole thing, every page analyzed with the actual fixes and the reasoning behind each one. Not the free scan. The one with a price on the pricing page.
To claim it, email me with the subject line THE ROBOTS LIED. That is it. No form, no call, no "let's hop on a quick fifteen." Just the three words, so I know you got this far down and I am not sending it to somebody who bounced at the coffee.
First three. When they are gone I will come back and edit this paragraph to say so, because leaving a dead offer up is its own version of the exact problem this post is about, and I would never hear the end of it from myself.
The fix, and it is four minutes a document
I am not going to tell you to reorganize your drive. Nobody in the history of business has finished that project.
What I do now is stamp a banner on the top of anything that has been overtaken. Three kinds, and picking the wrong one is its own small crime.
SUPERSEDED, plus the date. This document is wrong enough that following it causes harm. The banner says do not follow this as written, names what specifically is wrong, names the document that replaces it, and says what is still worth keeping out of it. That last part matters. A superseded document almost always contains one genuinely good section, and deleting the whole thing throws it away.
CORRECTED, plus the date. The main claim or the title is wrong, but real work inside it still stands.
STILL CURRENT, one correction, plus the date. The document is sound and exactly one line went stale. Name the line. Do not reach for SUPERSEDED here, because if everything is flagged then nothing is, and people start scrolling past banners the way they scroll past cookie notices.
Then the one rule that makes any of it work: when you correct something old, go stamp it in the same sitting. Not on the list. Not next week. Right then, while you still remember which document was wrong and why it was wrong.
I stamped nine of them the same night I found this. Under an hour for the whole set. It is the least glamorous work imaginable and it is the highest return thing I did all month.
Twenty minutes on your own business tonight
I would be genuinely surprised if this comes back clean, and I say that with affection.
Open the three documents your business actually runs on. Onboarding, the price sheet, and whatever you hand somebody on day one.
Check ONE hard fact in each, against the real thing. Not against your memory, which is lying to you warmly and with total confidence. Load the live page. Open the payment processor. Look at the actual number.
If it is stale, do not just fix it. Stamp it, dated, and say what changed. The stamp is the part that stops the next person finding the wrong version.
Then go hunt for the pair. If one document was wrong, there is usually a newer one somewhere that already got it right. Make them point at each other.
Why I would rather tell you this
Because I sell this kind of work, and the most useful thing I own is not a case study where everything went beautifully. It is the shape of a mistake I actually made.
I ran a thorough, well-designed audit. It gave me a confident answer. That answer was wrong three times out of three on its most serious items, and it was wrong because it read my documents instead of reading my business.
So the rule I came out the other side with, which is now stuck to everything I do: query the thing, do not quote the document. If a number matters, go get it from the source. If two documents disagree, the newer one wins and you say so out loud instead of quietly picking your favorite. And when you turn out to be wrong, write the retraction down where the original lives, not just in your own head where it helps nobody.
It is a slower way to work. It is also the only version that survives somebody checking.
Subject line is THE ROBOTS LIED. First three. After that the 5-Minute Ops Quiz is free forever, takes five minutes, and nobody calls you.