Why AI agents hallucinate — and how to stop it on your website
When a customer asks your AI agent "Do you ship to Canada?" and it answers "Yes, free shipping to Canada!" — except you don't — that's a hallucination. And it's not a model bug. It's the default behaviour of an LLM you don't actively constrain.
What a hallucination actually is
An LLM is a next-token predictor trained on the open web. Given a plausible-sounding question, it will confidently produce a plausible-sounding answer, whether or not that answer is grounded in something real.
On raw ChatGPT, this is sometimes funny. On your e-commerce site, it's a refund, an angry email, or a compliance problem.
Three controls that remove hallucination
1. Retrieval before generation
Don't let the model speak from memory. Before each answer, retrieve the most relevant passages from the customer's own content (docs, policies, product pages). Pass those as hard context.
If retrieval finds no passage on the site for the question, the agent must not answer it.
2. Decline when no passage matches
The agent should be willing to say "I don't know — let me connect you with a human." This is the hardest control to get right, because models are trained to be helpful.
NovixAgent AI's system prompt has an explicit "decline before guess" rule. When retrieval finds no passage on your site for the question, the agent is told not to answer from general knowledge: it declines in one sentence and offers to connect the visitor with your team. Separately, a low retrieval score shows the visitor a hand-off button — to a live operator when one is online, otherwise to your lead form if you have one.
3. Source citation
A factual answer should cite which passage it came from. This does two things: gives the customer a way to verify, and forces the agent to answer from grounded content only (no citation, no claim).
Why most "RAG chatbots" still hallucinate
A naive RAG setup retrieves passages and shoves them in the prompt — but the prompt still lets the model add "helpful" commentary. If you don't explicitly forbid it, it will.
The fix is a system prompt that says: "answer only with facts present in the provided context; if the context doesn't answer the question, say so."
The trade-off
A strictly-grounded agent will sometimes say "I don't know" to questions you wish it could answer. This is a feature. The alternative — confidently making up a policy — costs you trust in minutes.
What we do at NovixAgent AI
- Hybrid retrieval (semantic + keyword) to surface the passage the question refers to.
- System prompt with explicit "decline if unsure" rule.
- When no passage on your site matches, the agent declines and offers human escalation instead of answering. Below a score threshold, the widget also shows a hand-off button when an operator is online or you have a lead form.
- Per-answer source citation in the UI.
The result: conversations where the agent answers crisply when it can, and hands off gracefully when it has no answer. That's the bar a production NovixAgent AI has to clear.