Does Interrater Variation Influence the Classification According to the Diagnostic Criteria for Multiple Sclerosis?

BACKGROUND: Studies on interrater agreement in MRI analysis for patients experiencing a first clinical episode suggestive of multiple sclerosis (MS) are limited and outdated. This study aimed to evaluate interrater agreement for lesion identification in the brain, optic nerve, and spinal cord, and to assess variability in applying the 2017 and 2024 McDonald criteria. METHODS: Seventy-eight patients underwent a standardized multisequence 3 Tesla MRI of the brain, optic nerves, and spinal cord, analyzed by three readers with varying expertise. Lesions were categorized based on location according to McDonald criteria. Interrater variability was measured using Cohen's κ for pairwise reader comparisons and Fleiss κ for overall agreement. RESULTS: Interrater agreement ranged from slight to fair for total lesion count and moderate to substantial for lesion presence. The highest agreement occurred for spinal cord lesions (κ = 0.84), periventricular T2 lesions (κ = 0.70), and gadolinium-enhancing brain lesions (κ = 0.75). Classification agreement based on diagnostic criteria was substantial between the 2017 and 2024 McDonald criteria revisions (κ = 0.65). CONCLUSIONS: Despite advanced standardized imaging protocols at 3 T, interrater agreement regarding lesion counts does not improve substantially. However, agreement in classifying lesions according to diagnostic criteria is consistent between the 2017 and 2024 criteria.