DOAJ Open Access 2025

Evaluating base and retrieval augmented LLMs with document or online support for evidence based neurology

Lars Masanneck Sven G. Meuth Marc Pawlitzki

Abstrak

Abstract Effectively managing evidence-based information is increasingly challenging. This study tested large language models (LLMs), including document- and online-enabled retrieval-augmented generation (RAG) systems, using 13 recent neurology guidelines across 130 questions. Results showed substantial variability. RAG improved accuracy compared to base models but still produced potentially harmful answers. RAG-based systems performed worse on case-based than knowledge-based questions. Further refinement and improved regulation is needed for safe clinical integration of RAG-enhanced LLMs.

Topik & Kata Kunci

Computer applications to medicine. Medical informatics

Penulis (3)

Lars Masanneck

Sven G. Meuth

Marc Pawlitzki

Format Sitasi

APA MLA BibTeX

Masanneck, L., Meuth, S.G., Pawlitzki, M. (2025). Evaluating base and retrieval augmented LLMs with document or online support for evidence based neurology. https://doi.org/10.1038/s41746-025-01536-y

Akses Cepat

PDF tidak tersedia langsung

Cek di sumber asli →

Lihat di Sumber doi.org/10.1038/s41746-025-01536-y

Informasi Jurnal

Tahun Terbit: 2025
Sumber Database: DOAJ
DOI: 10.1038/s41746-025-01536-y
Akses: Open Access ✓