Hallucinations & Limitations
AI models can generate convincing but incorrect information. Learn to recognize and verify hallucinations to work reliably with AI.
What are AI hallucinations?
AI hallucinations are a fundamental characteristic of how language models work. The model generates convincing but incorrect or non-existent information. It "makes up" answers that seem plausible, based on the data it was trained on, without actually understanding the facts.
Why does this happen? AI models are essentially prediction machines that calculate the most likely next word based on patterns from billions of texts. It's not a search engine and not a database of facts. It generates new text, every time, based on probabilities rather than truth.
About 3% of AI output contains inaccuracies. For comparison, an average knowledge worker makes around 5% errors. The difference is that AI delivers a mistake just as confidently as a correct answer. With intensive use, you'll regularly encounter hallucinations. That's why verification is essential.
Think of AI as a hyper-smart intern. Very capable and fast, but you always need to check the work. AI is trained on general knowledge, not on your specific work processes, brand guidelines, or current company data.
Want to know how different AI models perform regarding hallucinations? Check the Vectara Hallucination Leaderboard for a current overview.
Two examples from the training
One mistake you spot straight away, and one you only find when you try it yourself. Both sound equally certain.
Clear error
5.11 or 5.3: which is bigger?
5.11 is bigger than 5.3. If you look at the numbers, you can see that 5.11 has more decimal places and therefore represents a bigger value than 5.3.
A calculation error, with an explanation that sounds certain.
Convincing, but not true
How do I use the 'Smart Summarize' feature in Microsoft Word?
The 'Smart Summarize' feature in Microsoft Word helps you quickly and efficiently pull the key points out of a document.
- Open your document: make sure the document is open.
- Go to the 'Insert' tab: click the tab in the ribbon.
- Select 'Smart Summarize': find the option and click it.
A neat guide, but the feature and the steps are wrong.
When do you check?
Not everything needs the same scrutiny. Look at two things: how much is at stake, and how long does checking take?
Checking takes time
Little at stake
Use it, read once
Add the source, check everything
Just use it
Use it, check the facts yourself
A lot at stake
Quick to check
How to check
- Always verify numbers against reliable sources such as official statistics, annual reports, or government data.
- Verify names and dates via LinkedIn, company websites, or news sources.
- Test against expertise. If something sounds strange, ask a colleague or use your own domain knowledge.
- Ask for sources. Ask AI explicitly to name sources, and check if they actually exist.
- Compare with internal guidelines. Check if tone of voice, style, and content match your brand guidelines.
Reducing hallucinations
Working with a document and a lot at stake? Use these three rules. For a brainstorm or a first draft you don't need them.
Attach the document as a file
Not as a link. With a link, Copilot often reads only part of it, or nothing, and fills in the rest itself.
Quotes first, then the answer
"First find the relevant passages in the document. Then answer based on those passages."
"I don't know" is allowed
"If it's not in the document, say so. Don't guess."
More examples
Hallucinations aren't theoretical: you'll encounter them regularly. Below are some examples to help you recognize what they look like.
Inconsistent training data
The same question asked to three different AI models can produce three completely different answers. This is because each model is trained on different datasets and has different priorities.
Example from the training: "How many self-employed people are there in the Netherlands?" (from its own knowledge, without internet). The answers: 1.2 million, 1.3 million and 1.6 million. All three sound certain. That's why you provide the source yourself.
Let Copilot think deeper
Modern "thinking models" can analyze their own thought process and correct errors. These models show their reasoning and can self-correct when they discover a mistake. This makes them more reliable, but verification remains essential.
In Copilot you turn this on with Think Deeper in the model selection. The Reasoning tab shows the steps the model takes. That way you see how it arrives at an answer and where a mistake could be.