Background: Detecting new and enlarged lesions in multiple sclerosis (MS) patients is needed to determine their disease activity. LeMan-PV is a software embedded in the scanner reconstruction system of one vendor, which automatically assesses new and enlarged white matter lesions (NELs) in the follow-up of MS patients; however, multicenter validation studies are lacking. Purpose: To assess the accuracy of LeMan-PV for the longitudinal detection NEL white-matter MS lesions in a multicenter clinical setting. Study Type: Retrospective, longitudinal. Subjects: A total of 206 patients with a definitive MS diagnosis and at least two follow-up MRI studies from five centers participating in the Swiss Multiple Sclerosis Cohort study. Mean age at first follow-up = 45.2 years (range: 36.9–52.8 years); 70 males. Field Strength/Sequence: Fluid attenuated inversion recovery (FLAIR) and T1-weighted magnetization prepared rapid gradient echo (T1-MPRAGE) sequences at 1.5 T and 3 T. Assessment: The study included 313 MRI pairs of datasets. Data were analyzed with LeMan-PV and compared with a manual “reference standard” provided by a neuroradiologist. A second rater (neurologist) performed the same analysis in a subset of MRI pairs to evaluate the rating-accuracy. The Sensitivity (Se), Specificity (Sp), Accuracy (Acc), F1-score, lesion-wise False-Positive-Rate (aFPR), and other measures were used to assess LeMan-PV performance for the detection of NEL at 1.5 T and 3 T. The performance was also evaluated in the subgroup of 123 MRI pairs at 3 T. Statistical Tests: Intraclass correlation coefficient (ICC) and Cohen's kappa (CK) were used to evaluate the agreement between readers. Results: The interreader agreement was high for detecting new lesions (ICC = 0.97, Pvalue < 10−20, CK = 0.82, P value = 0) and good (ICC = 0.75, P value < 10−12, CK = 0.68, P value = 0) for detecting enlarged lesions. Across all centers, scanner field strengths (1.5 T, 3 T), and for NEL, LeMan-PV achieved: Acc = 61%, Se = 65%, Sp = 60%, F1-score = 0.44, aFPR = 1.31. When both follow-ups were acquired at 3 T, LeMan-PV accuracy was higher (Acc = 66%, Se = 66%, Sp = 66%, F1-score = 0.28, aFPR = 3.03). Data Conclusion: In this multicenter study using clinical data settings acquired at 1.5 T and 3 T, and variations in MRI protocols, LeMan-PV showed similar sensitivity in detecting NEL with respect to other recent 3 T multicentric studies based on neural networks. While LeMan-PV performance is not optimal, its main advantage is that it provides automated clinical decision support integrated into the radiological-routine flow. Evidence Level: 4. Technical Efficacy: Stage 2.
A Multicenter Longitudinal MRI Study Assessing LeMan-PV Software Accuracy in the Detection of White Matter Lesions in Multiple Sclerosis Patients
Pravata, Emanuele;
2023-01-01
Abstract
Background: Detecting new and enlarged lesions in multiple sclerosis (MS) patients is needed to determine their disease activity. LeMan-PV is a software embedded in the scanner reconstruction system of one vendor, which automatically assesses new and enlarged white matter lesions (NELs) in the follow-up of MS patients; however, multicenter validation studies are lacking. Purpose: To assess the accuracy of LeMan-PV for the longitudinal detection NEL white-matter MS lesions in a multicenter clinical setting. Study Type: Retrospective, longitudinal. Subjects: A total of 206 patients with a definitive MS diagnosis and at least two follow-up MRI studies from five centers participating in the Swiss Multiple Sclerosis Cohort study. Mean age at first follow-up = 45.2 years (range: 36.9–52.8 years); 70 males. Field Strength/Sequence: Fluid attenuated inversion recovery (FLAIR) and T1-weighted magnetization prepared rapid gradient echo (T1-MPRAGE) sequences at 1.5 T and 3 T. Assessment: The study included 313 MRI pairs of datasets. Data were analyzed with LeMan-PV and compared with a manual “reference standard” provided by a neuroradiologist. A second rater (neurologist) performed the same analysis in a subset of MRI pairs to evaluate the rating-accuracy. The Sensitivity (Se), Specificity (Sp), Accuracy (Acc), F1-score, lesion-wise False-Positive-Rate (aFPR), and other measures were used to assess LeMan-PV performance for the detection of NEL at 1.5 T and 3 T. The performance was also evaluated in the subgroup of 123 MRI pairs at 3 T. Statistical Tests: Intraclass correlation coefficient (ICC) and Cohen's kappa (CK) were used to evaluate the agreement between readers. Results: The interreader agreement was high for detecting new lesions (ICC = 0.97, Pvalue < 10−20, CK = 0.82, P value = 0) and good (ICC = 0.75, P value < 10−12, CK = 0.68, P value = 0) for detecting enlarged lesions. Across all centers, scanner field strengths (1.5 T, 3 T), and for NEL, LeMan-PV achieved: Acc = 61%, Se = 65%, Sp = 60%, F1-score = 0.44, aFPR = 1.31. When both follow-ups were acquired at 3 T, LeMan-PV accuracy was higher (Acc = 66%, Se = 66%, Sp = 66%, F1-score = 0.28, aFPR = 3.03). Data Conclusion: In this multicenter study using clinical data settings acquired at 1.5 T and 3 T, and variations in MRI protocols, LeMan-PV showed similar sensitivity in detecting NEL with respect to other recent 3 T multicentric studies based on neural networks. While LeMan-PV performance is not optimal, its main advantage is that it provides automated clinical decision support integrated into the radiological-routine flow. Evidence Level: 4. Technical Efficacy: Stage 2.I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.
