IT Glossary
What is an AI hallucination?
An AI hallucination is a generative model producing false or invented information, delivered in a perfectly confident and credible tone.
The dangerous property is not that these systems are sometimes wrong. It is that being wrong looks exactly like being right. A generative model produces text by predicting what should plausibly come next, and plausibility is not truth — so when the training data is thin, the question is ambiguous or the answer simply does not exist, the model composes something well-formed and delivers it with the same fluent assurance it uses for facts. Invented case citations, non-existent product specifications, a refund policy your company never wrote, a confidently misquoted deadline. For a business deploying this in front of customers, the exposure is real and has already been tested: a company has been held to the terms its own chatbot invented, on the reasonable principle that a customer cannot be expected to know which parts of your website to disbelieve. The mitigations are known and none of them is a promise of perfection. Ground answers in your own verified documents and require citations, keep the model narrow, refuse rather than guess when confidence is low, log everything, and put a human in front of any answer that creates an obligation.
Let’s talk about your project
Message us on WhatsApp or send an email — you talk directly to a developer.
office@northdan.com · +40 752 070 247
Why it matters for your business
Legal exposure kept under control
Answers grounded in approved sources with visible citations stop the assistant from creating commitments the company never made.
Measurable confidence in answers
Systems that cite sources and decline when unsure can be evaluated against a test set, turning trust into a number rather than a hope.
Decisions based on facts, not fiction
Internal users learn which outputs are verifiable, so AI assistance speeds up work without quietly injecting invented figures into it.
Frequently asked questions
Why do AI models hallucinate at all?
Because their job is producing the most probable continuation of text, not verifying claims. They hold statistical patterns rather than a database of facts, so when a detail is absent, rare or contradictory in training data, the most probable-sounding phrasing wins over the honest admission of ignorance. Training that rewards helpful-sounding answers has historically made this worse, not better.
Can hallucinations be eliminated completely from a company chatbot?
Reduced dramatically, not eliminated. Retrieval from approved documents, mandatory citation, a narrow scope, explicit permission to say it does not know, and a fallback to a human take a well-built assistant to a low single-digit error rate on in-scope questions. Anyone promising zero is selling something. Design the process assuming a small residual rate and decide in advance what that costs you.
Is a company legally responsible for what its chatbot invents?
Treat the assistant as your own statement, because regulators and courts have been doing so. The defence that the model produced it independently has failed in consumer disputes. Practically this means answers that create obligations — prices, warranties, refunds, availability, legal advice — should come from controlled sources or from a person, with logs kept to show what was said and when.
Let’s talk about your project
Message us on WhatsApp or send an email — you talk directly to a developer.
office@northdan.com · +40 752 070 247