MICCAI MICCAI


Events


Challenges


Every year, hundreds of new algorithms are published in the field of biomedical image analysis. For a long time, validation and evaluation of new methods was based on the authors’ personal data sets, rendering fair and direct comparison of solutions impossible. In the meantime, common research practice has changed and involves organization of international competitions (‘challenges’) that allow for benchmarking algorithms on publicly released data sets. While this was a great step forward, “advancement of science requires continuous examination of the principles and practices by which the research community operates” [1]. After 10 years of grand biomedical challenges at MICCAI, we have carefully analyzed and critically questioned common practice related to the design and organization of biomedical challenges [2,3]. Given this analysis, the mission of the MICCAI challenge working group is to bring MICCAI challenges to the next level of quality.

  

Working group composition

The MICCAI challenge working group was founded in summer 2018 and has the following active members:

lena

Lena Maier-Hein (Chair)

Division of Computer Assisted Medical Interventions (CAMI), German Cancer Research Center (DKFZ), Heidelberg, Germany Read more 


martel

Anne Martel

Department of Medical Biophysics, University of Toronto, Ontario, Canada, Physical Sciences, Sunnybrook Research Institute, Toronto, Ontario, Canada Read more 

 

michal

Michal Kozubek

Centre for Biomedical Image Analysis, Masaryk University, Brno, Czech Republic Read more

 

 

bennet

Bennett Landman

Electrical Engineering, Vanderbilt University, Nashville, Tennessee, USA Read more 

  

koop

Annette Kopp-Schneider

Division of Biostatistics, German Cancer Research Center (DKFZ), Heidelberg, Germany Read more

 

anniika

Annika Reinke
(Student representative)

Division of Computer Assisted Medical Interventions (CAMI), German Cancer Research Center (DKFZ), Heidelberg, Germany Read more 


Activities

Launched in 2018: Structured challenge submission tool for better interpretability and reproducibility of MICCAI challenge results and for improving quality control of challenge proposals during a fully web-based review process.

Coming up: Paper on how to publish a biomedical image analysis challenge

  

Further reading:

  1. Nichols, T. E. et al. (2017). Best practices in data analysis and sharing in neuroimaging using MRI. Nature Neuroscience, 20(3), 299. doi: 10.1038/nn.4500.
  2. Maier-Hein et al. (2018). Why rankings of biomedical image analysis competitions should be interpreted with care. Nature Communications 9: 5217. doi: 10.1038/s41467-018-07619-7
  3. Reinke et al. (2018, September). How to Exploit Weaknesses in Biomedical Challenge Design and Organization. In International Conference on Medical Image Computing and Computer-Assisted Intervention (pp. 388-395). Springer, Cham. doi: 10.1007/978-3-030-00937-3_45. 
  4. Grand-Challenges. (2012) at https://grand-challenge.org/
  5. Mendrik, A.M. and Aylward, S.R. (2018, December). Beyond the Leaderboard: Insight and Deployment Challenges to Address Research Problems. NIPS 2018 Challenges in Machine Learning (CiML) Workshop. https://arxiv.org/abs/1811.03014/

  

Acknowledgements

Many thanks to Annika Reinke, Sinan Onogur and Matthias Eisenmann (Div. Computer Assisted Medical Interventions, DKFZ Heidelberg) for implementing the structured challenge submission system.

 

 

 


Developed by iGraph