Inter-rater agreement of respiratory distress observation scale measurement between physicians and nurses in the emergency department

Abstract Background Dyspnea is an individual’s sensation of discomfort during breathing. For patients with dyspnea who are unable to communicate, the Respiratory Distress Observation Scale (RDOS) was used to rate the severity based on eight parameters observed. In the emergency department, emergency nurses triage the patients with dyspnea and monitor their symptom severities, while emergency physicians evaluate the patients to determine treatment decisions. We aim to study the inter-rater agreement of RDOS measurement between emergency physicians and nurses. Method Between March 2024 and January 2025, an observational cross-sectional study was conducted in resuscitation rooms of two university academic hospitals. The participants were emergency physicians and nurses who were the first responders to adult patients presenting to the resuscitation room with dyspnea. The RDOS assessment was done individually by the data record forms within 20 min after arriving in the resuscitation rooms. The primary outcome was inter-rater agreement on RDOS measurement between emergency physicians and nurses. Result By 176 patients with dyspnea (N = 176), 44 emergency physicians and 55 nurses were included with no difference in either age or clinical experiences. The overall physicians and nurses reported a fair agreement of RDOS severity (58%) with a Kappa statistic of 0.54 (95% CI: 0.47–0.61, SE 0.075, p < 0.001), with 57% agreement in patients with intact communication and 61% agreement in those with impaired communication. The ratings were internally consistent and homogeneous among each profession. (the overall IIC 0.737 and 0.767, respectively). The inter-rater reliability was poor to moderate across both professions when scoring seven of eight RDOS parameters. Conclusions Emergency physicians and nurses have fair inter-rater agreement on RDOS measurement. Seven of the eight RDOS parameters revealed poor to moderate inter-rater reliability in both professions. Therefore, implementing RDOS in the ED requires customized training and calibrating the RDOS assessment.