Evaluating the Efficiency of Large Language Models in Detecting and Mitigating Hallucinations Using a Curated Domain-Oriented Dataset and Post-Hoc Retrieval

Chandra Sekhar Sanaboina; Tammisetti Surekha

doi:10.55524/ijircst.2026.14.5.3

Abstract

Hallucination in large language models (LLMs) refers to the generation of plausible and coherent information that is incorrect, unsupported, or inconsistent with the available evidence. Detecting such hallucinations and verifying the information in generated responses are important for improving the reliability of LLM based systems. This study evaluates the ability of LLMs to detect hallucinated responses and investigates whether external evidence can improve the verification process. A curated dataset of 700 question and answer pairs covering 20 academic disciplines was used, including both factual and hallucinated responses. Four models, GPT 3.5 Turbo, GPT 4, GPT 4 Turbo, and Gemma 7B, were first evaluated using their internal knowledge. For responses that required further checking, the responses were divided into individual claims, and relevant evidence was retrieved from Wikipedia using BM25 and semantic re ranking. Each claim was then verified against the retrieved evidence, and FActScore was used to support the final classification as factual or hallucinated. With no evidence the models scored accuracies between 0.653 and 0.943. After adding evidence from Wikipedia accuracy rose from 0.88 to 0.986. Fleiss’ Kappa also increased from 0.544 to 0.829, indicating greater agreement among the models after external evidence was introduced. These results show that relevant external evidence can improve the reliability and consistency of LLM based hallucination detection. The proposed approach uses retrieval to verify claims and reclassify responses when the initial judgment needs evidence. Retrieval does not directly correct the content. This checking step can therefore help mitigating hallucination by spotting information that is not supported.

Keywords

Large Language Models, Hallucination Detection, Claim Verification, Evidence Retrieval, Hallucination Mitigation, FActScore, Semantic re?ranking, Factuality Assessment

References

R. H. Ajmal, M. U. Sarwar, M. K. Hanif, and M. I. Khan, "Evaluating the Effectiveness of Advanced Language Models in Detecting and Mitigating Hallucinations Using Structured Question-Answering, Novel Metrics, and Post-Hoc Retrieval," IEEE Access, vol. 13, pp. 173805–173815, 2025. Available from: https://ieeexplore.ieee.org/document/11177188
Saxena, D. Aggarwal, N. Badathala, and P. Bhattacharyya, "A framework to synthetically generate fine-grained hallucinated data," Language Resources and Evaluation, vol. 59, pp. 423–452, 2025. Available from: https://link.springer.com/article/10.1007/s10579-025-09864-x
J. Li, X. Cheng, W. X. Zhao, J.-Y. Nie, and J.-R. Wen, "HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models," in Proc. 2023 Conf. Empirical Methods in Natural Language Processing, 2023, pp. 6449–6464. Available from: https://aclanthology.org/2023.emnlp-main.397/
P. Manakul, A. Liusie, and M. J. F. Gales, "SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models," arXiv:2303.08896, 2023. Available from: https://arxiv.org/abs/2303.08896
Y. Zhang, "Hallucination Detection in Large Language Models by Classifying Information Sources," in Proc. CONF-SPML 2026 Symposium: 2nd Neural Computing and Applications Workshop, 2025. Available from: https://doi.org/10.54254/2755-2721/2026.TJ30007
S. Min, K. Krishna, X. Lyu, M. Lewis, W. Yih, P. W. Koh, M. Iyyer, L. Zettlemoyer, and H. Hajishirzi, "FActScore: Fine-grained atomic evaluation of factual precision in long form text generation," in Proc. 2023 Conf. Empirical Methods in Natural Language Processing (EMNLP), pp. 12076–12100, 2023. Available from: https://aclanthology.org/2023.emnlp-main.741/
S. Lin, J. Hilton, and O. Evans, "TruthfulQA: Measuring How Models Mimic Human Falsehoods," in Proc. 60th Annu. Meeting Assoc. Comput. Linguistics (Vol. 1: Long Papers), 2022, pp. 3214–3252. Available from: https://aclanthology.org/2022.acl-long.229/
Z. Cao, Y. Yang, X. Li, and H. Zhao, "AutoHall: Automated Factuality Hallucination Dataset Generation for Large Language Models," IEEE Trans. Audio, Speech, and Language Processing, vol. 34, 2026. Available from: https://ieeexplore.ieee.org/document/11267179/
Abdeen, S. M. T. Siddiqui, M. T. Ahmed, A. Singhal, and L. Khan, "Hallucination Detection in Large Language Models Using Diversion Decoding," Univ. of Texas at Dallas / NIST, 2024. Available from: https://doi.org/10.1007/978-3-031-96590-6_7
Barathidass and K. Namburu, "Rapid End-to-End Test Generation and Hallucination Mitigation Using Generative Artificial Intelligence," IEEE Access, vol. 14, 2026. Available from: https://doi.org/10.1109/ACCESS.2026.3657407
B. Zhu, X. Zhang, M. Gu, and Y. Deng, "Knowledge Enhanced Fact Checking and Verification," IEEE/ACM Trans. Audio, Speech, and Language Processing, vol. 29, 2021. Available from: https://doi.org/10.1109/TASLP.2021.3120636
Z. Lin, S. Guan, W. Zhang, H. Zhang, Y. Li, and H. Zhang, "Towards trustworthy LLMs: a review on debiasing and dehallucinating in large language models," Artificial Intelligence Review, vol. 57, no. 243, 2024. Available from: https://link.springer.com/article/10.1007/s10462-024-10896-y
S. S. Rahman et al., "Hallucination to truth: a review of fact-checking and factuality evaluation in large language models," Artificial Intelligence Review, vol. 59, no. 70, 2026. Available from: https://link.springer.com/article/10.1007/s10462-025-11454-w
N. Sharma and S. Singh, "A Hybrid Extractive and Encoder-Decoder-Based Approach for Mitigating Hallucination in Automatic Text Summarization," J. Transformative Technologies and Sustainable Development, vol. 9, no. 15, 2025. Available from: https://link.springer.com/article/10.1007/s41314-025-00083-4
S. Liu, Y. Gao, S. Li, P. Wang, and T. Wang, "A hallucination detection and mitigation framework for faithful text summarization using LLMs," Scientific Reports, vol. 16, no. 1374, 2026. Available from: https://www.nature.com/articles/s41598-025-31075-1
M. Omar et al., "Multi-model assurance analysis showing large language models are highly vulnerable to adversarial hallucination attacks during clinical decision support," Communications Medicine, vol. 5, no. 330, 2025. Available from: https://doi.org/10.1038/s43856-025-01021-3
Z. Ji et al., "Survey of Hallucination in Natural Language Generation," ACM Computing Surveys, vol. 55, no. 12, pp. 1–38, 2023. Available from: https://doi.org/10.1145/3571730
H.-Y. Lin, Y.-C. Chang, and J.-T. Chien, "Variational Dehallucination via Retrieval-Augmented Prompt Learning," IEEE/ACM Trans. Audio, Speech, and Language Processing, vol. 34, pp. 259–271, 2026. Available from: https://doi.org/10.1109/TASLPRO.2025.3646477
L. Kong, X. Zhong, J. Chen, H. Fu, and Y. Wang, "Multi-perspective consistency checking for large language model hallucination detection: a black-box zero-resource approach," Frontiers of Information Technology & Electronic Engineering, vol. 26, no. 11, pp. 2298–2309, 2025. Available from: https://www.fitee.zjujournals.com/en/article/doi/10.1631/FITEE.2500180/
H. Huang, X. Bu, H. Zhou, Y. Qu, J. Liu, M. Yang, B. Xu, and T. Zhao, "An Empirical Study of LLM-as-a-Judge for LLM Evaluation: Fine-tuned Judge Model is not a General Substitute for GPT-4," in Findings of ACL 2025, Vienna, Austria, 2025, pp. 5880–5895. Available from: https://aclanthology.org/2025.findings-acl.306/

Cites this article as

C. S. Sanaboina, T. Surekha, "Evaluating the Efficiency of Large Language Models in Detecting and Mitigating Hallucinations Using a Curated Domain-Oriented Dataset and Post-Hoc Retrieval", International Journal of Innovative Research in Computer Science and Technology (IJIRCST), vol. 14, no. 5, pp. 19-33, 2026. doi: https://doi.org/10.55524/ijircst.2026.14.5.3

Corresponding Author

Chandra Sekhar Sanaboina

Assistant Professor, Department of Computer Science and Engineering, UCEK, JNTUK, Kakinada, Andhra Pradesh, India

Download Full Paper

Download PDF

No. of Downloads: 2 | No. of Views: 20

Evaluating the Efficiency of Large Language Models in Detecting and Mitigating Hallucinations Using a Curated Domain-Oriented Dataset and Post-Hoc Retrieval

Citations

Total View 20

Total Download 2