AI Response Quality in Public Services: Temperature Settings and Contextual Factors

This study investigated how generative Artificial Intelligence (AI) systems—now increasingly integrated into public services—respond to different technical configurations, and how these configurations affect the <i>perceived quality</i> of the outputs. Drawing on an experimental evaluation of <i>Govern-AI</i>, a chatbot designed for professionals in the social, educational, and labor sectors, we analyzed the impact of the <i>temperature</i> parameter—which controls the degree of creativity and variability in the responses—on two key dimensions: <i>accuracy</i> and <i>comprehensibility</i>. This analysis was based on 8880 individual evaluations collected from five professional profiles. The findings revealed the following: (1) the high-temperature responses were generally more comprehensible and appreciated, yet less accurate in strategically sensitive contexts; (2) professional groups differed significantly in their assessments, where trade union representatives and regional policy staff expressed more critical views than the others; (3) the <i>type of question</i>—whether operational or informational—significantly influenced the perceived output quality. This study demonstrated that the AI performance was far from neutral: it depended on technical settings, usage contexts, and the profiles of the end users. Investigating these “behind-the-scenes” dynamics is essential for fostering the <i>informed governance</i> of AI in public services, and for avoiding the risk of technology functioning as an opaque <i>black box</i> within decision-making processes.