3976
Necessity for a common dataset for a fair comparison between deep neural networks for QSM
Chungseok Oh1, Woojin Jung1, Hwihun Jeong1, and Jongho Lee1
1Seoul National University, Seoul, Korea, Republic of
A comparison between deep neural networks for QSM can be unfair when training datasets have different characteristics (e.g., different resolution) or network hyperparameters are not optimized. A common dataset can be a solution for a fair comparison between networks.
Figure 1. Performance comparison of the networks, QSMnet1x1x1 and QSMnet1x1x3, trained by the datasets of different resolutions (1x1x1 mm3 vs. 1x1x3 mm3). The network trained with the same resolution data (e.g., QSMnet1x1x1 with the test data of 1x1x1 mm3) outperformed the other network.
Figure 3. Performance comparison of the networks of two different hyperparameter sets. The hyperparameter of the original QSMnet was empirically determined for the 1x1x1 mm3 training dataset. When the same network was trained with a new training dataset of 1×1×3 mm3 resolution, it resulted in NRMSE of 57.8 ± 7.1% (QSMnet1x1x3-ref). Since the training dataset characteristic has changed, we can further improve the performance by hyperparameter tuning, resulting in NRMSE of 56.6 ± 7.0% (QSMnet1x1x3-hyper). The calculated p-value was 4.7e-4.