Hallucinations and Model Limitations

A hallucination is when a model gives incorrect or fabricated information in a confident tone.

Why hallucinations happen

• Model predicts likely text, not guaranteed truth.
• Prompt lacks context or clear constraints.
• Requested knowledge is missing or outdated.

Other limitations

• Sensitive to prompt wording.
• Can fail in multi-step reasoning if not guided.
• May inherit bias from training data.
• Can produce non-deterministic outputs.

Mitigation strategies

1. Use RAG and reliable source data.
2. Ask for citations and confidence notes.
3. Add validation and post-processing checks.
4. Keep human review for high-impact decisions.

Example

For medical or legal use, never rely only on model output. Always verify with trusted sources and domain experts.