Comparative Evaluation of AI-based Systems for Tinnitus

INTRODUCTION: Today, with the development of technology, the variety of information sources has increased. It is now possible to access information obtained from encyclopedias in seconds with a few clicks of a button. Rapid developments in artificial intelligence (AI) and the widespread use of large language models (LLMs) such as ChatGPT, Gemini, and Perplexity have revolutionized access to medical information. However, the accuracy and readability of the answers provided by these models are critical, especially in the healthcare domain. This study evaluates the performance of ChatGPT, Gemini, and Perplexity in addressing frequently asked questions about tinnitus, a common symptom in otolaryngology practice. METHODS: Twenty frequently asked questions about tinnitus were posed to the models and their responses were evaluated by two otolaryngologists using global quality (GQS) and Likert scales for accuracy and reliability and the Gunning-Fog Index (GFI) for readability. RESULTS: The findings reveal no significant difference in the reliability and quality of information between the models (median: 5.00, p=0.159), but it was observed that Gemini came out ahead in readability (p=0.001) and ChatGPT in accuracy (p=0.007). However, Perplexity lagged in both metrics. These results highlight the varying strengths and weaknesses of LLMs, emphasizing the importance of model selection based on user needs. For example, ChatGPT is ideal for complex medical information, while Gemini is more accessible to wider audiences. DISCUSSION AND CONCLUSION: This study demonstrates the potential of AI-enabled systems in healthcare; however, we suggest that future improvements should increase both accuracy and accessibility.