Firas Khader1, Gustav Müller-Franzes1, Johannes Stegmaier2, Martin Pixberg3, Jonas Müller-Hübenthal3, Christiane Kuhl 1, Sven Nebelung4, and Daniel Truhn1
1Department of Diagnostic and Interventional Radiology, Aachen University Hospital, Aachen, Germany, 2Institute of Imaging and Computer Vision, RWTH Aachen University, Aachen, Germany, 3Praxis im Köln Triangle, Cologne, Germany, 4Department of Diagnostic and Interventional Radiology, Düsseldorf University Hospital, Dusseldorf, Germany
1Department of Diagnostic and Interventional Radiology, Aachen University Hospital, Aachen, Germany, 2Institute of Imaging and Computer Vision, RWTH Aachen University, Aachen, Germany, 3Praxis im Köln Triangle, Cologne, Germany, 4Department of Diagnostic and Interventional Radiology, Düsseldorf University Hospital, Dusseldorf, Germany
Comparing the performance of neural networks to detect ACL and meniscus tears on a knee MRI dataset comprised of 3887 manually annotated exams show that the neural networks do not benefit from expert annotations by board-certified radiologists.
Figure 1. Comparison of the Receiver Operating Characteristic (ROC) curves for varying numbers of training samples, i.e. n=500 (blue), n=1000 (turquoise), n=1500 (green), and n=2493 training samples (purple) in algorithm-based detection of ACL (a) and meniscus tears (b). For the detection of the ACL tears, the area under curve (AUC) increased from 0.64 (n=500) to 0.80 (n=2493). For the detection of meniscus tears, the AUC increased from 0.68 (n=500) to 0.75 (n=2493).
Figure 2. Receiver Operating Characteristic (ROC) curves and corresponding area under curve (AUC) for the test set depicting the difference in performance when training the neural network with expert (purple) vs non-expert (yellow) annotations. Neither in the case of ACL tears (a) nor in the case of meniscus tears (b) does the network benefit from the additional expert annotations by a board-certified radiologist.
