arXiv Open Access 2024

Generation Constraint Scaling Can Mitigate Hallucination

Georgios Kollias Payel Das Subhajit Chaudhury

Lihat Sumber

Abstrak

Addressing the issue of hallucinations in large language models (LLMs) is a critical challenge. As the cognitive mechanisms of hallucination have been related to memory, here we explore hallucination for LLM that is enabled with explicit memory mechanisms. We empirically demonstrate that by simply scaling the readout vector that constrains generation in a memory-augmented LLM decoder, hallucination mitigation can be achieved in a training-free manner. Our method is geometry-inspired and outperforms a state-of-the-art LLM editing method on the task of generation of Wikipedia-like biography entries both in terms of generation quality and runtime complexity.

Topik & Kata Kunci

cs.CL cs.AI cs.LG

Penulis (3)

Georgios Kollias

Payel Das

Subhajit Chaudhury

Format Sitasi

APA MLA BibTeX

Kollias, G., Das, P., Chaudhury, S. (2024). Generation Constraint Scaling Can Mitigate Hallucination. https://arxiv.org/abs/2407.16908

Akses Cepat

Lihat di Sumber

Informasi Jurnal

Tahun Terbit: 2024
Bahasa: en
Sumber Database: arXiv
Akses: Open Access ✓