Contributions
Abstract: EP1526
Type: Poster Sessions
Abstract Category: Pathology and pathogenesis of MS - MRI and PET
Introduction: Manual delineation is the gold standard for measuring MS WM lesion (WML); however, it suffers from inter and intra-rater disagreement and is too resource demanding to be feasible for large studies. Automated methods can be an alternative, but their training and validation requires manually labelled reference datasets, which poses the same challenges above.
Aims: Here we present a platform for making the manual segmentation of WML feasible at a large scale by distributing the segmentation to many raters as small, quick tasks so that many raters perform a single segmentation to generate a consensus segmentation. This should make the consensus segmentation more robust and faster to generate.
Methods: Since the platform is intended for crowd-sourcing we need to ensure full anonymity to safeguard patients' identity. Images are anonymised, skull-stripped, and divided into rectangular sub-slices, hence patch, not covering more than a quarter of the brain visible in the slice. Raters are presented with a patch to rate, as well as the corresponding patches from slices immediately above and below. Raters select parts of the image they think contain WML, and parts they deem WML free. This is done simply by clicking on the relevant sub-regions into which the image has already been divided. These so-called super-pixels are irregularly shaped regions automatically generated from the image (1). At the time of abstract writing, the platform is undergoing evaluation in a pilot study comparing the consensus segmentations from multiple raters to fully manual segmentations. Raters are drawn from different groups: radiologists, medical students and general public. Using Dice overlap, the effect of number and experience of raters is evaluated.
Conclusions: If confirmed, distributed segmentation provides key benefits over manual segmentation which enables usage in large studies: (a) since patches are segmented by many raters, the results are less prone to bias (b) Each rater can contribute many or few tasks, making rater recruitment feasible (c) Raters' expertise can be traded by number of raters and thus a variety of raters can contribute to a study. With these benefits, distributed segmentation can generate large scale segmentation for research as well as training and validation of automatic WML segmentation tools.
1 Radhakrishna A., et al. SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans. Pattern Anal. Mach. Intell 34.11 (2012)
Disclosure: Authors have nothing to disclose that could constitute a conflict of interest for this work.
S. Damangir is financed through ECTRIMS-MAGNIMS fellowship. A. de Sitter is employed on a project sponsored by a research grant from Teva Pharmaceuticals (grant to H. Vrenken and F. Barkhof). I Brouwer is funded from research grants from Novartis, Teva and the Dutch MS Research Foundation. C.R.G. Guttmann has received research funding from Sanofi, the National Multiple Sclerosis Society, and the International Progressive Multiple Sclerosis Alliance. D. Pareto has received speaking honoraria from Novartis. A. Rovira serves on scientific advisory boards for Novartis, Sanofi-Genzyme, Icometrix, SyntheticMR, and OLEA Medical, and has received speaker honoraria from Bayer, Sanofi-Genzyme, Bracco, Merck-Serono, Teva Pharmaceutical Industries Ltd, Novartis, Roche and Biogen Idec. H. Vrenken has received research grants from Novartis, Teva and MerckSerono, and consulting fees from MerckSerono; all funds were paid directly to his institution.
Abstract: EP1526
Type: Poster Sessions
Abstract Category: Pathology and pathogenesis of MS - MRI and PET
Introduction: Manual delineation is the gold standard for measuring MS WM lesion (WML); however, it suffers from inter and intra-rater disagreement and is too resource demanding to be feasible for large studies. Automated methods can be an alternative, but their training and validation requires manually labelled reference datasets, which poses the same challenges above.
Aims: Here we present a platform for making the manual segmentation of WML feasible at a large scale by distributing the segmentation to many raters as small, quick tasks so that many raters perform a single segmentation to generate a consensus segmentation. This should make the consensus segmentation more robust and faster to generate.
Methods: Since the platform is intended for crowd-sourcing we need to ensure full anonymity to safeguard patients' identity. Images are anonymised, skull-stripped, and divided into rectangular sub-slices, hence patch, not covering more than a quarter of the brain visible in the slice. Raters are presented with a patch to rate, as well as the corresponding patches from slices immediately above and below. Raters select parts of the image they think contain WML, and parts they deem WML free. This is done simply by clicking on the relevant sub-regions into which the image has already been divided. These so-called super-pixels are irregularly shaped regions automatically generated from the image (1). At the time of abstract writing, the platform is undergoing evaluation in a pilot study comparing the consensus segmentations from multiple raters to fully manual segmentations. Raters are drawn from different groups: radiologists, medical students and general public. Using Dice overlap, the effect of number and experience of raters is evaluated.
Conclusions: If confirmed, distributed segmentation provides key benefits over manual segmentation which enables usage in large studies: (a) since patches are segmented by many raters, the results are less prone to bias (b) Each rater can contribute many or few tasks, making rater recruitment feasible (c) Raters' expertise can be traded by number of raters and thus a variety of raters can contribute to a study. With these benefits, distributed segmentation can generate large scale segmentation for research as well as training and validation of automatic WML segmentation tools.
1 Radhakrishna A., et al. SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans. Pattern Anal. Mach. Intell 34.11 (2012)
Disclosure: Authors have nothing to disclose that could constitute a conflict of interest for this work.
S. Damangir is financed through ECTRIMS-MAGNIMS fellowship. A. de Sitter is employed on a project sponsored by a research grant from Teva Pharmaceuticals (grant to H. Vrenken and F. Barkhof). I Brouwer is funded from research grants from Novartis, Teva and the Dutch MS Research Foundation. C.R.G. Guttmann has received research funding from Sanofi, the National Multiple Sclerosis Society, and the International Progressive Multiple Sclerosis Alliance. D. Pareto has received speaking honoraria from Novartis. A. Rovira serves on scientific advisory boards for Novartis, Sanofi-Genzyme, Icometrix, SyntheticMR, and OLEA Medical, and has received speaker honoraria from Bayer, Sanofi-Genzyme, Bracco, Merck-Serono, Teva Pharmaceutical Industries Ltd, Novartis, Roche and Biogen Idec. H. Vrenken has received research grants from Novartis, Teva and MerckSerono, and consulting fees from MerckSerono; all funds were paid directly to his institution.