View Presentation Video

Naoto Fujita¹ and Yasuhiko Terada^1
1Graduate School of Science and Technology, University of Tsukuba, Tsukuba, Japan

Keywords: Image Reconstruction, Machine Learning/Artificial Intelligence

Deep learning (DL) reconstruction networks are predominantly architectures that unroll traditional iterative algorithms and tend to perform better than non-unrolled models. Both types of models use convolutional neural networks (CNNs) as building blocks, but CNNs have the disadvantage of focusing on local relationships in the image. To overcome this, hybrid models have been proposed that combine CNNs with Transformers that focus on long-range dependencies. However, these hybrid transformers have been limited to non-unrolled reconstruction networks. Here, we propose an unrolled reconstruction network using a hybrid Transformer, Deep Cascade of Swin Transformer (DC-Swin), and verify that DC-Swin has high performance.

View Presentation Video

Chi Zhang^1,2, Davide Piccini^3,4, Omer Burak Demirel^1,2, Gabriele Bonanno⁴, Steen Moeller², Burhaneddin Yaman^1,2, Matthias Stuber^3,5, and Mehmet Akçakaya^1,2
1Department of Electrical and Computer Engineering, University of Minnesota, Minneapolis, MN, United States, ²Center for Magnetic Resonance Research, University of Minnesota, Minneapolis, MN, United States, ³Department of Diagnostic and Interventional Radiology, Lausanne University Hospital and University of Lausanne, Lausanne, Switzerland, ⁴Advanced Clinical Imaging Technology, Siemens Healthineers International, Lausanne, Switzerland, ⁵Center for Biomedical Imaging, Lausanne, Switzerland

Keywords: Image Reconstruction, Machine Learning/Artificial Intelligence

Although recent studies enabled physics-guided deep learning (PG-DL) reconstruction of 3D non-Cartesian MRI, it suffers from blurring, partially due to limited training data. In this study we propose 2.5D PG-DL using three 2D CNNs on orthogonal views for 3D reconstruction to efficiently exploit the limited training data. Results on 3D kooshball coronary MRI show the proposed strategy noticeably improves image sharpness.

View Presentation Video

Arushi Gupta¹, Batu M Ozturkler², Arda Sahiner², Tolga Ergen², Arjun D Desai^2,3, Shreyas Vasanawala³, John M Pauly², Morteza Mardani², and Mert Pilanci^2
1California Institute of Technology, Pasadena, CA, United States, ²Electrical Engineering, Stanford University, Stanford, CA, United States, ³Radiology, Stanford University, Stanford, CA, United States

Keywords: Image Reconstruction, Image Reconstruction

Deep learning (DL) has recently shown state-of-the-art performance for accelerated MRI reconstruction. However, supervised learning requires fully-sampled training data, and training these networks with end-to-end backpropagation requires significant memory for high-dimensional imaging. These challenges limit the use of DL in high-dimensional settings where access to fully-sampled data is unavailable. Here, we propose self-supervised greedy learning for memory-efficient MRI reconstruction without fully-sampled data. The method divides the end-to-end network into smaller network modules and independently calculates a self-supervised loss for each subnetwork. The proposed method generalizes as well as end-to-end learning without fully-sampled data with at least 7x less memory usage.

View Presentation Video

Hideaki Kutsuna¹, Shun Uematsu², and Kensuke Shinoda^2
1MRI Systems Development Department, Canon Medical Systems Corporation, Kanagawa, Japan, ²MRI Systems Development Department, Canon Medical Systems Corporation, Tochigi, Japan

Keywords: Image Reconstruction, Machine Learning/Artificial Intelligence, Super resolution

The authors propose a new reconstruction method to obtain higher resolution images from an MR acquisition. The method incorporates MR physics and two neural networks, which are functionally separate, for denoising and upsampling. The proposed method was evaluated by applying it to both retrospectively and prospectively undersampled data. The result showed that the proposed technique is capable of reconstructing higher resolution images over a conventional method, by multiplying the matrix size while keeping more detail structure in the originally sampled data.

View Presentation Video

Ilias Giannakopoulos¹, Patricia Johnson¹, Riccardo Lattanzi¹, and Matthew Muckley^2
1The Bernard and Irene Schwartz Center for Biomedical Imaging (CBI), Department of Radiology, New York University Grossman School of Medicine, New York, NY, United States, ²Facebook AI Research, Meta, New York, NY, United States

Keywords: Image Reconstruction, Parallel Imaging, Compressed Sensing, Deep Learning

Deep learning (DL) methods have enabled state-of-the-art reconstructions of magnetic resonance images of highly undersampled acquisitions. The end-to-end variational network (E2E VarNet) is a DL method that can output high quality reconstructions through an unrolled gradient descent algorithm. Nevertheless, the network discards a lot of high-level feature representations of the image to perform data consistency in the image space. Here, we adapted the E2E VarNet architecture to perform the data consistency in a feature space. We trained the proposed network using the fastMRI brain dataset and observed 0.0013 SSIM improvement for eight-fold accelerations.

 

View Presentation Video

Christopher Man^1,2, Vick Lau^1,2, Jiahao Hu^1,2, Junhao Zhang^1,2, Linfang Xiao^1,2, and Ed X. Wu^1,2
1Laboratory of Biomedical Imaging and Signal Processing, The University of Hong Kong, Hong Kong SAR, China, ²Department of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong SAR, China

Keywords: Image Reconstruction, Image Reconstruction

3D MRI data contains more redundant information than 2D MRI data, which is favourable for reconstruction. However, deep learning reconstruction of 3D MRI data remains to be explored due to the computational burden that scales exponentially with spatial dimensions. This study presents a deep learning method to reconstruct single-channel 3D MRI data with uniform undersampling along two phase-encoding directions, in which conventional multi-channel parallel imaging methods are generally not applicable. The results demonstrate the robust reconstruction for single-channel 3D MRI data at high acceleration and in the presence of anomaly.

View Presentation Video

Parna Eshraghi Boroojeni¹, Wejie Gan², Jiaming Liu², Yuyang Hu², Yasheng Chen³, Paul Commean⁴, Cihat Eldeniz⁵, Tongyao Wang⁶, Gary Skolnick⁷, Corinne Merrill⁷, Kamlesh Patel⁷, Hongyu An⁶, and Ulugbek Kamilov^8
1Washington University in Saint Louis, Saint Louis, MO, United States, ²Engineering - Computer Science and Engineering, Washington University in Saint Louis, Saint Louis, MO, United States, ³Neurology, Washington University in Saint Louis, Saint Louis, MO, United States, ⁴Radiology - Main - Research - Radiological Sciences, Washington University in Saint Louis, Saint Louis, MO, United States, ⁵Radiology - Research Imaging Facilities - MR Facility, Washington University in Saint Louis, Saint Louis, MO, United States, ⁶Radiology - Main - Research - Radiological Sciences - Biomedica, Washington University in Saint Louis, Saint Louis, MO, United States, ⁷Surgery - Plastics, Washington University in Saint Louis, Saint Louis, MO, United States, ⁸Engineering - Electrical & Systems Engineering, Washington University in Saint Louis, Saint Louis, MO, United States

Keywords: Image Reconstruction, Image Reconstruction

We develop a self-supervised and physics-guided deep unfolding (DU) network for MR image reconstruction by weighting the k-space sampling Density in network training Loss (wDeLo). We have demonstrated that high-quality MR images at a spatial resolution of 0.6x0.6x0.8 mm3 could be achieved using an acquisition time of 1 minute (x6.25 acceleration) or 45 seconds (8x acceleration). Compared to SSDU and its uniform weighted counterpart (un-wDeLo), the wDeLo method significantly improves PSNR and SSIM. It had fewer artifacts, lower noise, and preserved image sharpness.

View Presentation Video

Matthew Charles Bendel¹, Rizwan Ahmad², and Philip Schniter^1
1Dept. ECE, The Ohio State University, Columbus, OH, United States, ²Dept. BME, The Ohio State University, Columbus, OH, United States

Keywords: Image Reconstruction, Machine Learning/Artificial Intelligence

For magnetic resonance (MR) image reconstruction, Fourier-domain measurements are collected at rates far below Nyquist to reduce clinical exam time. Because many plausible reconstructions exist that are consistent with a given measurement, we use machine learning to sample from the posterior distribution rather than generate a single image reconstruction. Many such works leverage score-based generative models (SGMs), which seek to iteratively denoise a random input but require many minutes to generate each sample. We propose a conditional generative adversarial network (GAN) that generates hundreds of posterior samples per minute and outperforms the current state-of-the-art SGM for multi-coil MR posterior sampling.

View Presentation Video

Aniket Pramanik¹, Sampada Bhave², Saurav Sajib², Samir Sharma², and Mathews Jacob^1
1University of Iowa, Iowa City, IA, United States, ²Canon Medical Research USA, Mayfield Villlage, OH, United States

Keywords: Image Reconstruction, Machine Learning/Artificial Intelligence

Model-based Deep Unrolled Networks offer high quality reconstructions but the performance degrades if any mismatch occurs in acquisition settings. Different networks for different acquisition settings require extensive training data, in addition to making the clinical deployment difficult. We propose a single unrolled deep-learning algorithm called as Ada-MoDL, whose parameters are conditioned on the acquisition information (metadata) of a dataset, using a Multi-Layer Perceptron (MLP) that maps the metadata to network parameters. Ada-MoDL outperforms models trained for a specific acquisition setting or a single model trained with all the available contrasts, when the training data in each acquisition setting is limited.

View Presentation Video

Jeffrey Wen¹, Rizwan Ahmad², and Philip Schniter^1
1Electrical Engineering, The Ohio State University, Columbus, OH, United States, ²Biomedical Engineering, The Ohio State University, Columbus, OH, United States

Keywords: Image Reconstruction, Machine Learning/Artificial Intelligence

For accelerated MR image reconstruction, machine learning (ML)-based methods outperform traditional sparsity-based methods by exploiting large datasets to learn effective priors. However, most ML methods output only a single image reconstruction when in fact there may be many plausible reconstructions given the measurement and prior. To extract this diagnostically relevant information, we propose to explore the space of plausible images, i.e., to sample the posterior, using ML. Among ML methods, conditional normalizing flows (CNFs) stand out for rapid sample generation and simple likelihood-based training. In this work, we present the first CNF for posterior sample generation in accelerated multicoil MRI.

View Presentation Video

Hongjian Kang¹, Liping Zhang¹, and Weitian Chen^1
1Department of Imaging and Interventional Radiology, The Chinese University of Hong Kong, Hong Kong, Hong Kong

Keywords: Image Reconstruction, Machine Learning/Artificial Intelligence

Accelerated reconstruction for magnetic resonance imaging (MRI) is a challenging ill-posed problem because of the excessive under-sampling operation in k-space. Existing CNN-based and Transformer-based solutions face difficulties obtaining powerful representation due to relatively unitary local or global feature modeling capability. In this study, we develop a dual-branch network that can simultaneously exploit the complementarity of the two-style features by leveraging the merits of CNN and Transformer, to generate high-quality reconstruction from zero-filled images in the spatial domain. Qualitative and quantitative results from the fastMRI dataset demonstrate that the proposed method can achieve improved performance compared with other benchmark methods.

View Presentation Video

Jun-Hyeok Lee¹, Gawon Lee¹, and Se-Hong Oh^1
1Department of Biomedical Engineering, Hankuk University of Foreign Studies, Yongin, Korea, Republic of

Keywords: Image Reconstruction, Parallel Imaging, Neural ODE

In parallel MRI, DNN-based models have recently outperformed conventional reconstruction techniques and can reconstruct high-quality MRI images, especially at high acceleration factors. We propose a model-based neural ODE network to reconstruct artifact-free MR images from under-sampled k-space data. We replaced the existing U-Net with a modified U-Net framework using neural ODEs with E2E-VarNet as the backbone. Our network solves unrolled iterations of reconstruction optimization with neural ODEs, and each neural ODE uses a gradient update step as a dynamics step. Our approach showed the improved reconstruction performance comparable to the SOTA method with few parameters.

View Presentation Video

Muntasir Shamim¹, Chenwei Tang², Leonardo Rivera², and Kevin M Johnson^2
1Electrical and Computer Engineering, University of Wisconsin Madison, Madison, WI, United States, ²Medical Physics, University of Wisconsin Madison, Madison, WI, United States

Keywords: Image Reconstruction, Brain

Deep equilibrium (DEQ) image reconstruction models have demonstrated improved performance while remaining memory efficient for single coil MRI reconstructions; however, DEQ models are known to be susceptible to model drift due to variabilities  in the forward model. In the case of multicoil MRI, the forward model is changed  due to the sampling patterns and coil sensitivity encoding. In this work, we implemented a multicoil DEQ model and analyzed model drift with respect to the coil sensitivities.  

View Presentation Video

Sizhuo Liu¹, Philip Schniter¹, and Rizwan Ahmad^1
1The Ohio State University, Columbus, OH, United States

Keywords: Image Reconstruction, Machine Learning/Artificial Intelligence

Recovery with a self-calibrated denoiser (ReSiDe) is an unsupervised learning method based on the plug-and-play (PnP) framework. In ReSiDe, denoiser training and a call to the denoising subroutine are performed in each iteration of PnP. However, ReSiDe is computationally slow, and its performance is sensitive to the noise level selected to train the denoiser. Here, we extend ReSiDe from single-image to multi-image recovery (ReSiDe-M), improving both performance and computation speed. We also propose an auto-tuning method to select the noise level for denoiser training. Using data from fastMRI and MRXCAT perfusion phantom, we compare ReSiDe-M with other unsupervised methods.

View Presentation Video

Peizhou Huang¹, Hongyu Li², Ruiying Liu², Xiaoliang Zhang¹, Xiaojuan Li³, Dong Liang⁴, and Leslie Ying^1,2
1Biomedical Engineering, State University of New York at Buffalo, Buffalo, NY, United States, ²Electrical Engineering, State University of New York at Buffalo, Buffalo, NY, United States, ³Program of Advanced Musculoskeletal Imaging (PAMI), Cleveland Clinic, Cleveland, OH, United States, ⁴Paul C. Lauterbur Research Center for Biomedical Imaging, SIAT CAS, Shenzhen, China

Keywords: Image Reconstruction, Machine Learning/Artificial Intelligence, Unpaired unsupervised training, Transformer

In this abstract, we propose a novel deep-learning reconstruction method that enables training with only unpaired undersampled k-space data without the ground truth. The network utilizes a statistical model for the undersampling artifacts to enable unsupervised learning, and the generative adversarial network to enable unpaired training. In addition, the physics model is incorporated into the transformer network by unrolling the underlying optimization problem. Experiment results based on the fastMRI knee dataset exhibit marked improvements over the existing state-of-the-art reconstructions.

View Presentation Video

Mamata Shrestha¹, Nian Wang^2,3, and Ukash Nakarmi^1
1Computer Science and Computer Engineering, University of Arkansas, Fayetteville, AR, United States, ²Radiology and Imaging Sciences, Indiana University School of Medicine, Indianapolis, IN, United States, ³Stark Neurosciences Research Institute, Indianapolis, IN, United States

Keywords: Image Reconstruction, Data Processing, Image super-resolution, model bias, deep learning

Deep Learning based image super-resolution methods are biased towards training data modeling. Generalizability of DL based super-resolution frameworks can be improved by introducing variability and diversity in the training data. 

View Presentation Video

Yanting Yang¹, Andrew F. Laine¹, and Jia Guo^2,3
1Department of Biomedical Engineering, Columbia University, New York, NY, United States, ²Department of Psychiatry, Columbia University, New York, NY, United States, ³Mortimer B. Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, United States

Keywords: Image Reconstruction, Machine Learning/Artificial Intelligence

Conventional medical image reconstruction methods are less parametric and lack generality due to random error and noise. A novel artificial Fourier transform (AFT) framework is developed which determines the mapping between k-space and i-space like DFT while can be fine-tuned with further training. The flexibility of AFT allows it to be simply incorporated into any existing deep learning network as learnable or static blocks. Reconstruction and denoising tasks are combined into a unified network that simultaneously enhances the image quality. AFT-Net achieves competitive results compared with other methods and proofs to be more robust to additional noise and contrast differences.

View Presentation Video

Ibsa Kumara Jalata¹ and Ukash Nakarmi^1
1Computer Science and Computer Engineering, University of Arkansas, Fayetteville, AR, United States

Keywords: Image Reconstruction, Data Processing

Unsupervised deep learning framework that integrates system priors using unrolled optimization and general image priors can reconstruct high quality Magnetic Resonance images comparable to supervised methods from highly undersampled k-space data.  We develop an unsupervised deep learning framework that integrates system priors in MR acquisition and image priors to reconstruct high quality MR images from highly undersampled k-space data without using ground truth images. 

Xue Liu¹, Cheng Li¹, Yu Zhang¹, Haoran Li¹, Yeqi Wang¹, Hairong Zheng¹, and Shanshan Wang^1,2,3
1Paul C. Lauterbur Research Center for Biomedical Imaging, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China, ²Peng Cheng Laboratory, Shenzhen, China, ³Guangdong Provincial Key Laboratory of Artificial Intelligence in Medical Image Analysis and Application, Guangdong Provincial People’s Hospital, Guangdong Academy of Medical Sciences, Guangzhou, China

Keywords: Image Reconstruction, Machine Learning/Artificial Intelligence

Deep learning-based, especially fully supervised learning-based, methods have shown unprecedented performance in MR image reconstruction. Fully sampled data are utilized as references to supervise the learning process. However, it is challenging to acquire fully sampled data in many real-world application scenarios. Unsupervised approaches are required. Here, we propose an iterative data refinement method for enhanced self-supervised MR image reconstruction. Different from Yaman's self-supervised learning method (SSDU), training data in our method are refined iteratively during model optimization to progressively eliminate the data bias between the undersampled reference data and fully sampled data. Better reconstruction results are obtained.

View Presentation Video

Pengfei Guo¹, Yiqun Mei¹, Jinyuan Zhou², Shanshan Jiang², and Vishal M. Patel^1
1Department of Computer Science, Johns Hopkins University, Baltimore, MD, United States, ²Department of Radiology, Johns Hopkins University, Baltimore, MD, United States

Keywords: Image Reconstruction, Image Reconstruction

Accelerating MRI reconstruction process is a challenging ill-posed inverse problem due to the excessive under-sampling operation in k-space. While state-of-the-art algorithms have shown a great progress based on convolutional neural networks (CNN), transformers for MRI reconstruction has not been fully explored in the literature. We propose a recurrent transformer model, namely, ReconFormer, for MRI reconstruction which can iteratively reconstruct high fertility magnetic resonance images from highly under-sampled k-space data. We validate the effectiveness of ReconFormer on multiple datasets with different magnetic resonance sequences and show that it achieves significant improvements over the state-of-the-art methods with better parameter efficiency.