Development and Validation of a Geriatrics Knowledge Test to Evaluate Geriatrics Fellowship Programs

Background/Objectives Although the Accreditation Council for Graduate Medical Education requires that geriatrics fellowship programs be evaluated, evaluation is challenging because of lack of appropriate instruments. The purpose of this study was to develop and validate a geriatrics knowledge test appropriate for evaluation of geriatrics fellowship programs. Design Initial and replication cross‐sectional validation studies. Setting Academic medical center. Participants Initial study: 11 postgraduate year ( PGY )‐2 and five PGY ‐3 internal medicine residents, eight incoming and eight graduating geriatrics fellows, and 11 geriatrics faculty (N = 43). Replication study: nine graduating fellows and three mid‐year fellow cohorts (n = 11, 8, and 9) (N = 37). Measurements A geriatrics knowledge test was developed consisting of 31 multiple‐choice questions ( MCQ s) selected from a 54‐item pool of MCQ s that the authors created. Selection criteria included content appropriateness, item correlation with total score, item discriminatory power, and item difficulty. Results The instrument demonstrated high reliability (Cronbach alpha = 0.83) and known group validity. The mean percentage correct scores on the knowledge test were progressively higher with more geriatrics training ( P < .001). The replication study continued to show patterns of progressive increases in score with additional training. There were no floor or ceiling effects. Conclusion A 31‐item geriatrics knowledge test demonstrates sound reliability and validity. The findings support that it is appropriate as a tool for evaluation of geriatrics fellowship programs.