On a fact-recall test OpenAI reports, one of its older models gave a wrong answer 75% of the time and declined to answer 1% of the time. AI tools make things up because they're built to produce the likeliest next words, and they've been graded in a way that rewards a confident guess over admitting a gap. You can still trust them with business work, as long as you know which kinds of answer to check.
That's close to OpenAI's own account, published in September 2025 (Why language models hallucinate, OpenAI, 2025-09-05, checked 2026-10-20). Anthropic's own guide says the techniques that cut wrong answers don't remove them (Reduce hallucinations, Anthropic's Claude docs, checked 2026-10-20). So the useful question for an owner is a narrow one: which outputs get read before they leave the building.
In one line each
- Hallucination
A plausible but false statement that an AI model gives confidently.
True on October 20, 2026. Source: OpenAI, Why language models hallucinate
- Grounding
Tying a model's answer to sources you can check, such as a document you hand it, so it has less room to invent.
True on October 20, 2026. Source: Google Cloud, Grounding overview
It guesses because guessing scored better#
A model learns by predicting the next word across a huge amount of text, and none of that text comes labelled true or false. Patterns that repeat, like spelling, it gets right almost every time. Facts that turn up rarely and follow no pattern, like a person's birthday, it can't work out from patterns, so it fills the gap with something that sounds right. OpenAI's own example: asked for the PhD dissertation title of Adam Tauman Kalai, one of the paper's authors, a widely used chatbot gave three different titles, all wrong, and then three wrong birthdays.
The grading makes it worse. Most of the tests models are scored on count only right answers. OpenAI compares it to a multiple-choice exam: leave a question blank and you get zero, guess and you might get lucky. A model that guesses a birthday it doesn't know has a 1 in 365 chance of scoring. One that says "I don't know" scores nothing. Across thousands of questions, the guesser looks better on the leaderboard.
OpenAI's numbers from that test, SimpleQA, show the trade. Its older o4-mini model got 24% right, 75% wrong, and declined to answer 1% of the time. Its newer gpt-5-thinking-mini got 22% right, declined 52% of the time, and was wrong on 26% (same post; the full argument is in the paper by Kalai, Nachum, Vempala and Zhang, 2025-09-04). By our arithmetic, 26% against 75% is about a third of the wrong answers, for nearly the same share of right ones. That's one test on two of OpenAI's own models, and we can't say how the tool you use would score. The direction is the point: the model that declined more often made far fewer confident mistakes.
A chatbot's wrong answer is still your answer#
A Canadian tribunal has already ruled on this. In November 2022, Jake Moffatt's grandmother died, and Moffatt asked the chatbot on Air Canada's website about bereavement fares. It said they could travel now and apply for the reduced fare within 90 days of the ticket being issued. Air Canada's own bereavement page, linked from that same answer, said the policy didn't apply once travel was finished. Moffatt paid CA$1,630.36 for two flights and was refused the partial refund.
Air Canada argued it couldn't be held liable for what its chatbot said. The tribunal member, Christopher C. Rivers, called that "a remarkable submission." He found the chatbot was part of Air Canada's website, that the company is responsible for everything on it whether it sits on a fixed page or comes out of a chat window, and that a customer has no reason to double-check one part of a website against another. Air Canada was ordered to pay CA$812.02: CA$650.88 in damages, CA$36.14 in interest and CA$125 in tribunal fees (Moffatt v. Air Canada, 2024 BCCRT 149, Civil Resolution Tribunal of British Columbia, 2024-02-14, checked 2026-10-20).
Two limits on that case. It's one small-claims decision in one province. And the decision notes Air Canada gave no information about what kind of chatbot it was, so it doesn't show that a model like ChatGPT made the mistake. What carries over is narrower: the tribunal treated the bot's answer as the company's answer. That's a description of one decision, not legal advice.
Check the facts, skim the wording#
Anything a customer could act on gets checked. Anything about how it's said can be skimmed.
OpenAI's explanation maps onto business work neatly. Your prices, your hours, your refund terms and the date a client's job is due are the rare, specific facts a model can't infer from patterns, because they're yours. Tone, structure and phrasing are patterns, which is what these models do well. That mapping is our reading of the research; the paper doesn't discuss businesses.
| Check before it goes out | Fine to skim | |---|---| | Prices, fees, discounts and totals | Tone and word choice | | Dates, deadlines and opening hours | The order and structure of a draft | | Your policies: refunds, warranties, cancellations | A first draft you'll rewrite anyway | | Quotes, citations and statistics | Subject lines and headings | | Names, addresses and phone numbers | A list of ideas to choose from |
Five quick checks#
Anthropic says tying the answer to your source and letting the model admit a gap both reduce invented answers. Google says the same of grounding, and OpenAI's research points the same way on gaps. None of them says by how much in owner work, and we haven't measured it. Checks 1 to 3 and the rerun tip in check 4 follow the vendors' guidance. Picking two numbers to check yourself, and check 5, are our take.
1. Give it the source and forbid outside facts
Paste the price list, the policy or the client's email, and tell it to use only that. Anthropic's guide lists this as restricting the model to the documents you provide instead of its general knowledge. Google calls the wider idea grounding, and says tying the answer to sources you can check reduces the chance of invented content (Grounding overview, Google Cloud docs, checked 2026-10-20).
2. Ask it to quote the line each fact came from
Anthropic's guide gives two versions. For long documents, over 20,000 tokens, it suggests asking for word-for-word quotes first and doing the task from those. Separately, it suggests checking each claim against a supporting quote and dropping any claim that has none. Our take: both work on a one-page price list too, and they turn checking into matching. Find the quoted line in your own document and that fact is done.
3. Tell it to mark gaps instead of guessing
Give it explicit permission to say it doesn't know. That's the first technique in Anthropic's guide, and it's the behaviour OpenAI says the grading has been discouraging. A visible gap like [ASK: is the deposit refundable?] is a question you can answer before anything goes out. A guessed answer costs you whatever the customer does with it.
4. Check two numbers against the original
Pick the two figures that would hurt most if wrong, usually a price and a date, and find them in the source yourself. That habit is ours. Anthropic's guide closes on the general version: validate critical information, especially for high-stakes decisions. If something still feels off, run the same request twice. Anthropic notes that answers which differ between runs can point to a made-up fact.
5. Never send a customer-facing fact unread
A reply to a Google review, a quote, a booking confirmation: anything with your name on it and a fact in it gets read by a person before it goes. The Air Canada decision is the reason. The customer had no way to tell which part of the website was wrong, and the tribunal didn't expect them to.
A prompt that runs the first three checks#
Paste this into ChatGPT, Claude or Gemini, put your source between the quote marks, then read the gaps it leaves before you read anything else.
Every [ASK: ...] it leaves is a question only you can answer, and every quoted line is a fact you can tick off against your own document.