LLMs confidently generate false information. They invent facts, cite non-existent sources, and create plausible-sounding nonsense.
In production, hallucinations erode user trust and can cause real harm. A customer service bot that invents policies, a legal assistant that cites fake cases, a medical chatbot that fabricates symptoms - these are not edge cases.
This post documents how we reduced hallucination rates from 15% to under 3%.
| Type | Description | Example |
|---|
| Factual | Incorrect facts | "Paris is the capital of Germany" |
| Attribution | Fake sources | "According to a 2023 Nature study..." (does not exist) |
| Temporal | Wrong timeframes | Events in wrong order or date |
| Numerical | Made-up numbers | Invented statistics |
| Entity | Wrong entities | Attributing quotes to wrong people |
| Logical | Invalid reasoning | Conclusions that do not follow |
| Cause | Explanation |
|---|
| Training data gaps | Model fills gaps with plausible content |
| Probability-based generation | Most likely next token, not most accurate |
| No knowledge boundaries | Model does not know what it does not know |
| Prompt ambiguity | Unclear prompts invite speculation |
| Context limitations | Cannot verify against external sources |
Generate multiple responses and compare them:
| Approach | Implementation |
|---|
| Multiple samples | Generate 3-5 responses at temperature > 0 |
| Compare answers | Check if key facts align |
| Flag inconsistencies | Different answers = possible hallucination |
| Consistency | Hallucination Rate |
|---|
| All 5 agree | 2% |
| 4/5 agree | 8% |
| 3/5 agree | 23% |
| No majority | 67% |
Cross-reference claims against trusted sources:
| Step | Action |
|---|
| 1 | Extract factual claims from response |
| 2 | Search knowledge base for verification |
| 3 | Compare claims to retrieved facts |
| 4 | Flag unverifiable or contradicted claims |
Use model uncertainty signals:
| Signal | How to Access | Interpretation |
|---|
| Logprobs | API parameter | Low prob = uncertain |
| Hedging language | Text analysis | "I think", "possibly" = uncertain |
| Response length | Observation | Very short or very long = potential issue |
Break response into individual claims and verify each:
| Original | Decomposed Claims |
|---|
| "Einstein won the Nobel Prize in 1921 for relativity" | 1. Einstein won Nobel Prize 2. Year was 1921 3. Award was for relativity |
| Verification | 1. True 2. True 3. False (was for photoelectric effect) |
| Method | Accuracy | Latency | Cost |
|---|
| Self-consistency | 78% | High (multiple calls) | High |
| Retrieval verification | 85% | Medium | Medium |
| Confidence estimation | 62% | Low | Low |
| Claim decomposition | 89% | High | High |
| Combined approach | 94% | High | High |
Force the model to only use provided context:
| Prompt Element | Purpose |
|---|
| "Only use information from the context below" | Explicit grounding instruction |
| "If the answer is not in the context, say so" | Encourage admission of uncertainty |
| "Quote the relevant passage" | Force attribution |
| Approach | Hallucination Rate |
|---|
| No grounding instruction | 15% |
| Basic grounding | 8% |
| Grounding + attribution | 4% |
| Grounding + uncertainty acknowledgment | 3% |
Provide relevant context before generation:
| Component | Impact on Hallucinations |
|---|
| High-quality retrieval | -40% hallucinations |
| Multiple sources | -25% additional |
| Source attribution | -15% additional |
Lower temperature reduces creativity and hallucinations:
| Temperature | Creativity | Hallucination Rate |
|---|
| 0.0 | None | 5% |
| 0.3 | Low | 7% |
| 0.7 | Medium | 12% |
| 1.0 | High | 18% |
Train users and models to express uncertainty:
| Instead Of | Say |
|---|
| "The answer is X" | "Based on the available information, X appears to be the case" |
| "X happened in 1995" | "X happened around 1995, though I recommend verifying this date" |
| Making up an answer | "I do not have enough information to answer this accurately" |
Technique 5: Domain Constraints
Limit the model to its area of competence:
| Constraint | Implementation |
|---|
| Topic boundaries | "Only answer questions about our products" |
| Knowledge cutoff | "My information may be outdated after [date]" |
| Capability limits | "I cannot provide medical/legal/financial advice" |
| Detection Confidence | Response |
|---|
| High confidence hallucination | Block response, generate alternative |
| Medium confidence | Add disclaimer, show sources |
| Low confidence | Log for review, serve response |
| Bad | Good |
|---|
| Show hallucinated content | "I am not certain about this. Let me find a verified source." |
| Silent failure | "I could not verify this information. Here is what I found in our documentation:" |
| Overconfident | "Based on [source], the answer appears to be X. Please verify for critical decisions." |
| Scenario | Fallback |
|---|
| Cannot verify answer | Acknowledge uncertainty |
| Contradicts known facts | Show contradiction, ask for clarification |
| Outside knowledge domain | Redirect to appropriate resource |
| Critical use case | Require human verification |
| Stage | Action | Latency |
|---|
| 1 | Retrieve relevant context | 100ms |
| 2 | Generate with grounding prompt | 2-3s |
| 3 | Extract claims | 200ms |
| 4 | Verify against sources | 300ms |
| 5 | Add confidence indicators | 50ms |
| Total | End-to-end | ~3.5s |
| Metric | Target | Alert |
|---|
| Detected hallucination rate | < 3% | > 5% |
| User-reported inaccuracies | < 1% | > 2% |
| Verification failure rate | < 10% | > 15% |
| Uncertainty acknowledgments | > 5% | < 2% (might be overconfident) |
| Activity | Frequency | Purpose |
|---|
| Sample review | Daily | Catch new hallucination patterns |
| Prompt refinement | Weekly | Improve grounding instructions |
| Knowledge base updates | Ongoing | Keep retrieval current |
| Model evaluation | Per update | Ensure no regression |
| Metric | Before | After |
|---|
| Hallucination rate | 15% | 2.8% |
| User trust score | 3.2/5 | 4.4/5 |
| Verified accuracy | 78% | 96% |
| "I do not know" rate | 1% | 8% |
| Support escalations | 18% | 6% |
- 1RAG is your best defense - Grounding generation in retrieved context is the most effective prevention technique.
- 2Self-consistency catches problems - If the model gives different answers to the same question, something is wrong.
- 3Lower temperature for facts - Use temperature 0-0.3 for factual tasks.
- 4Teach uncertainty - Models that admit "I do not know" are more trustworthy than those that always answer.
- 5Verify, do not trust - Build verification into your pipeline, not as an afterthought.
- 6Monitor continuously - Hallucination patterns change. What works today may not work tomorrow.
Hallucinations are not a solved problem, but they are a manageable one. With the right techniques, you can build AI systems that users can actually trust.
