CLSeq and Nscp: novel methods for reducing hallucinations in text summarization for pre-trained models and LLMs

Abstract With the advancement of text summarization technology, the issue of hallucinations in summaries has garnered increasing attention. Pretrained models often incorporate additional factual information to minimize the occurrence of hallucinations. In research on text summarization using LLMs, accurate samples are typically provided via a chain-of-thought approach, enabling the model to learn the implicit relationship between the source text and the target summary. To address the hallucination problem in text summarization, this paper proposes CLSeq and Nscp, which are specifically designed for pre-trained models and LLMs, respectively. CLSeq integrates the strengths of human-generated summaries and model-generated summaries to produce high-quality positive samples as target, while refining the loss function to handle negative samples more effectively. Nscp provides negative examples and explanatory information through the chain-of-thought mechanism. These strategies aim to enhance the model’s understanding of the characteristics and causes of hallucinations, thereby reducing the likelihood of factual inconsistencies in the summaries. Experimental results demonstrate that both methods effectively mitigate the hallucination problem in text summarization and exhibit a certain degree of robustness.