Reference-Driven Undersampled MR Image Reconstruction Using Wavelet Sparsity-Constrained Deep Image Prior

Deep learning has shown potential in significantly improving performance for undersampled magnetic resonance (MR) image reconstruction. However, one challenge for the application of deep learning to clinical scenarios is the requirement of large, high-quality patient-based datasets for network training. In this paper, we propose a novel deep learning-based method for undersampled MR image reconstruction that does not require pre-training procedure and pre-training datasets. The proposed reference-driven method using wavelet sparsity-constrained deep image prior (RWS-DIP) is based on the DIP framework and thereby reduces the dependence on datasets. Moreover, RWS-DIP explores and introduces structure and sparsity priors into network learning to improve the efficiency of learning. By employing a high-resolution reference image as the network input, RWS-DIP incorporates structural information into network. RWS-DIP also uses the wavelet sparsity to further enrich the implicit regularization of traditional DIP by formulating the training of network parameters as a constrained optimization problem, which is solved using the alternating direction method of multipliers (ADMM) algorithm. Experiments on in vivo MR scans have demonstrated that the RWS-DIP method can reconstruct MR images more accurately and preserve features and textures from undersampled k-space measurements.


Introduction
Magnetic resonance imaging (MRI) is a noninvasive imaging technology that can provide structural, functional, and anatomical information for clinical diagnosis. However, its slow imaging speed may result in motion artifacts and image quality degradation, as well as lead to patient discomfort. To accelerate MRI scans, researchers are seeking methods to increase imaging speed by reducing the amount of acquired k-space data without degrading the image reconstruction quality.
Accelerated MR image reconstruction from undersampled k-space measurements is, in essence, a highly underdetermined inverse problem. Reconstruction methods based on signal processing have evolved rapidly over the past decades and can now explore and utilize the prior information about the desired MR image to achieve the reconstruction by using regularization methods under the premise of ensuring the uniqueness and stability of the solution. Sparsity is a commonly used prior information with the emerging popularity of Compressed Sensing (CS) theory [1][2][3], including fixed sparse transform (e.g., wavelet or/and gradient) [4][5][6] and more flexible adaptive sparse representation (e.g., data-driven tight frame [7] and dictionary learning [8][9][10]). High-resolution reference images obtained in advance in practical application scenarios can also provide prior information. They can provide structural similarity for the target MR images and obtain more sparse difference images [11][12][13]. In addition, the structured priors, such as image support information [14][15][16] and structural sparsity (e.g., group sparsity, block sparsity, and tree sparsity) [15,17,18], can be introduced into a reconstruction model based on the union-of-subspace sampling theory [19], which has been verified to be efficient in improving reconstruction accuracy.
In recent years, deep learning has received a great deal of attention in the field of medical imaging, especially for segmentation, denoising, classification, and acceleration of MRI tasks [20]. MRI approaches based on deep learning can be either data-driven or model-driven [21,22]. Datadriven approaches are aimed at learning the mapping from undersampled k-space/images to fully sampled k-space/images [23][24][25][26][27][28]. Model-driven approaches start from MR image reconstruction models and import the procedure of iterative reconstruction algorithms into networks [29][30][31][32].
To ensure the quality of reconstruction performance, both approaches require pre-training processes with the aid of large, high-quality patient-based datasets. However, this is a challenge in clinical applications because it is difficult to obtain sufficient amounts of patient-based MR datasets due to patient privacy concerns.
In this paper, we propose a novel deep learning-based Reference-driven method using Wavelet Sparsityconstrained DIP (RWS-DIP) for CS-based undersampled MR image reconstruction, which can achieve improved performance without any pre-training procedures. Our proposed RWS-DIP method incorporates structure and sparsity priors into a DIP framework and utilizes the priors to further improve the efficiency of learning. It not only builds a bridge between the constrained reconstruction method and deep learning, but also largely reduces the dependence on patientbased datasets and contributes to the expansion of clinical applications. Experimental results have shown that the proposed RWS-DIP method can obtain more accurate reconstruction than traditional DIP, particularly in preserving image textures and features. The main contributions of this paper can be summarized as follows: (1) The proposed RWS-DIP method utilizes both structure and sparsity priors of MR images. The former is introduced by using a high-resolution reference image obtained in advance as the input of CNN, whose structure is similar to target MR images and thereby incorporates structural information into network. The latter is used by regularizing the l 1 norm of coefficients in a wavelet domain to further enrich the implicit regularization of traditional DIP, which is enforced by the fixed network structure. These priors improve the efficiency and effectiveness of deep learning and contribute to the improvement in reconstruction performance (2) The proposed RWS-DIP is a novel deep learningbased MR image reconstruction method inspired by traditional DIP and does not require any pretraining. This advantage renders the training datasets unnecessary, which has significance in clinical applications The remainder of this paper is organized as follows. Section 2 presents details on the proposed RWS-DIP method, as well as a review of traditional DIP. Section 3 includes experimental results from in vivo MR scans and also includes details about data acquisition, undersampled schemes, and the experimental setup. Section 4 provides a summary of the paper's main points and its results.

Traditional DIP for Undersampled MR Image
Reconstruction. Applying traditional DIP to undersampled MR image reconstruction, the object function is where y ∈ ℂ M×1 is the undersampled k-space measurements of the desired MR image I t ∈ ℂ N×N , F u denotes an undersampled Fourier transform operator, and k•k 2 is the l 2 norm. f ðθ | zÞ is an untrained deep CNN parameterized by θ, with the random noise z as input.
The desired MR image can then be reconstructed bŷ The training of the network parameters θ is performed by solving the optimization problem in Equation (1) iteratively, which is guided by the attempt to best match the network output to the measurements in k-space. In DIP, no pretraining procedure is needed and the network training, or optimizing of network parameters, begins with an untrained CNN initialized randomly. Figure 1 depicts an overview of our proposed RWS-DIP method, in which the procedure of the target MR image reconstruction can be achieved in three steps: network training, MR image reconstruction, and data correction. In the first step, we do not need high-quality MR datasets and pre-training. The network parameters of the untrained CNN are optimized by solving the proposed constrained object function iteratively, which not only restricts the data consistency and explores wavelet sparsity but also introduces structural prior by using a similar reference image as the input of CNN. Next, the trained network outputs the reconstructed MR image. In the third step, the data correction process uses the prior measurements in k -space to further improve the reconstruction accuracy. A further explanation will be provided in the following sections.

Network Training with a Reference and Wavelet
Sparsity-Constrained DIP. Leveraging the concept of the traditional DIP framework, our proposed RWS-DIP method uses a high-resolution reference MR image and the wavelet sparsity to provide prior information for the target MR image 2 Computational and Mathematical Methods in Medicine reconstruction. Therefore, the objective function for network parameter optimization is as follows: where I r ∈ ℂ N×N denote a high-resolution reference MR image acquired in advance with similar anatomical structure to the target image I t ∈ ℂ N×N , Ψ is the wavelet transform operator, and k•k 1 is the l 1 norm. The regularization parameter λ > 0.
Our proposed objective function in Equation (3) consists of the data fidelity term and the l 1 regularization term. It is aimed at finding the optimal network parameters that ensure the sparsity of the target MR image in wavelet domain on the premise of maintaining data consistency.
The data fidelity term restricts the data consistency between the network output and k-space measurements. We use the known reference MR image I r as the network input, instead of random noise in traditional DIP. This strategy is capable of exploring and introducing the structural prior of the target MR image into the network for learning because of the high structural similarity between the reference and target images. The l 1 regularization constrains the sparsity of the target MR image in a wavelet domain, which merges more prior information for efficient training of network parameters. Let The constrained optimization problem in Equation (4) can be transformed into a penalty using the augmented Lagrangian:  Figure 1: Overview of the proposed RWS-DIP method: (a) overall process for ADMM-based reconstruction; (b) network architecture [33] used in the proposed method.

Computational and Mathematical Methods in Medicine
In the expression above, μ stands for the Lagrange multiplier vector and ρ is a penalty parameter.
To solve the problem in Equation (5), we use the alternating direction method of multipliers (ADMM) algorithm [42] to update the three unknowns θ, α, and μ iteratively: (1) For the subproblem in Equation (6), this optimization is close in spirit to that performed in traditional DIP. However, we further modify the optimization by a proximity regularization that forces Ψf ðθ | I r Þ to be close to ðα k−1 − μ k−1 Þ, which helps to provide additional stabilization and robustness (2) For the subproblem in Equation (7), the solution can be written as where S λ/ρ is the soft thresholding operator defined as [42]

Data Correction. Performing data correction operator
Corð·Þ to CNN outputÎ rec in the last step below, we obtain corrected k-space data y cor as follows: where F denotes Fourier transform and y is the priori acquired measurements of the target MR image, which are sampled at the spatial locations corresponding to the undersampled mask U in k-space. Let U denote the complementary set of U. This data correction strategy, defined in Equation (12), reserves all the priori acquired measurements to enforce the k-space data consistency, so that the reconstruction error will focus only on the missing k-space data. The final reconstructed target MR image can then be achieved by performing an inverse Fourier transform on y cor The algorithm flowchart of our proposed RWS-DIP method is presented in Algorithm 1.

Network
Architecture. The CNN architecture employed in the proposed RWS-DIP method is summarized in Figure 1(b), which is the same as that used in [33]. It is an encoder-decoder ("hourglass") architecture with skip  The maximal depth of the network is L. n d ½i, n u ½i, and n s ½i denote the number of filters at the ith depth for downsampling, upsampling, and skip connections, respectively. k d ½i, k u ½i, and k s ½i correspond to the respective kernel sizes.

Experimental Setup.
Experiments were conducted to evaluate the performance of our proposed RWS-DIP method. The comparisons with the proposed RWS-DIP method included zero-filling and traditional DIP [33]. To ensure a fair comparison, the zero-filling reconstructions and corresponding k -space measurements were used as inputs for all the methods, and the same network architectures was employed for our RWS-DIP method and traditional DIP.
We quantified the reconstruction quality using the metrics of relative error (RelErr), peak signal-to-noise ratio (PSNR), and structural similarity index (SSIM) [43]: In the descriptions in Equations (14)- (16), the reconstructed MR imagex and the ground truth x are the same size of N × N, and MAX x denotes the largest value in x. Moreover, for the SSIM shown in Equation (16), μ x , μx, σ x , and σx represent the means and standard deviations of x andx, respectively, and σ xx denotes the crosscovariance between x andx, and constants c 1 = 0:01 and c 2 = 0:03.

Data Acquisition.
To demonstrate the performance of our RWS-DIP method, simulations were conducted on three groups of invivo MR images. To simulate the data acquisition, we undersampled the 2D discrete Fourier transform of the MR images from invivo MR scans, which were acquired from a 3T Siemens MRI scanner. The imaging parameters of the first group of scanned data (Brain A) were GR sequence, flip angle = 70°, TR = 250 ms, TE = 2:5 ms, field of view ðFOVÞ = 220 mm × 220 mm, and slice thickness = 5:0 mm. The reference and target images in Brain A were of size 512 × 512, as shown in Figures 2(a) and 2(b). The imaging parameters of the second and third groups of scanned data (Brain B and Brain C) were as follows: SE sequence, flip angle = 120°, TR = 4000 ms, TE = 91 ms, FOV = 176 mm × 176 mm, and slice thickness = 5:0 mm. The MR images in Brain B and Brain C were of size 256 × 256 and are shown in Figures 2(c)-2(f), respectively.

Training
Setting. We used the same CNN architecture as the traditional DIP in [33], which is shown in detail in Figure 1(b). The parameters used in the experiments, including network hyperparameters, iteration number (MaxIt and SubIt), wavelet (wavelet function and decomposition level), ADMM penalty parameter ρ, and regularization parameter λ, are shown in Table 1.
The models were implemented on the Ubuntu 16.04 LTS (64 bit) operating system, running on an Intel Core i9-7920X 2.9 GHz CPU and Nvidia GeForce GTX 1080Ti GPU with 11 GB RAM in the PyTorch open framework with CUDA and CUDNN support.

Undersampled Schemes.
To compare the influence of different undersampling masks to the performance of the proposed RWS-DIP method, our experiments employed three types of undersampling masks: Cartesian, variable density, and radial. Figure 3 depicts these three undersampling masks.

Reconstruction Performance Comparison
(1) Reconstruction under Different Sampling Rates. We demonstrated the effectiveness of our RWS-DIP method at different sampling rates under Cartesian mask. Table 2 shows the quantitative performance of the proposed RWS-DIP method, traditional DIP and zero-filling reconstructions in RelErr, and PSNR and SSIM indexes at 10%, 20%, 30%, and 40% sampling rates. Taking into account the randomness     6 show the reconstructed MR images using the proposed RWS-DIP method and the compared methods under Cartesian undersampled mask with 20% and 30% sampling rates. It is obvious that our RWS-DIP method has the best performance in preserving more image textures and features, especially from the zoom-in images. The corresponding error images further show that the reconstruction of our RWS-DIP method has the smallest differences and is closest to the target MR image.
(2) Reconstruction with Different Undersampled Masks. The reconstruction results were compared under radial and vari-able density undersampled masks. The quantitative results tabulated in Table 3 clearly indicate that the proposed RWS-DIP method obtains more accurate reconstruction than with the radial and variable density undersampled masks. Comparisons of the reconstructed MR images are shown in Figures 7 and 8. The corresponding error images and zoom-in images demonstrate that our RWS-DIP method outperforms the compared methods with less structural loss and can preserve more details than the radial and variable density undersampled masks.  9 Computational and Mathematical Methods in Medicine undersampled mask. Figure 9 depicts the relative errors of reconstruction at every ADMM iteration. It can be observed that, as the number of iterations increases, the relative errors gradually converge to a low value at different sampling rates. Although there are slight fluctuations in the iteration procedure, the overall trend maintains convergence.

Parameter Evaluation.
We evaluated the sensitivity of the proposed RWS-DIP method to parameter settings. The main parameters evaluated were the ADMM penalty parameter ρ and the regularization parameter λ. We performed experiments on the Brain C dataset under Cartesian undersampled mask and varied one parameter at a time while keeping the rest as fixed values, as shown in Table 1. Figures 10 and 11 show the plots of PSNR values as a function of the ADMM penalty parameter ρ and the regularization parameter λ. As can be seen from the curves, the optimal numerical settings for ρ and λ (ρ = 0:05 and λ = 0:0001) in the proposed RWS-DIP method under different sampling rates are identical, which means that the RWS-DIP method has robustness in the setting of parameters. In fact, although the reconstructions have lower PSNR values than other numerical settings for parameters ρ and λ, the difference is not significant, and the reconstruction performance is acceptable.

Conclusions
In this paper, we propose a novel reference-driven undersampled MR image reconstruction method using wavelet sparsity-constrained deep image prior. Our RWS-DIP method, which is based on the DIP framework, requires neither a pre-training procedure nor patient-based datasets, which is of great significance for clinical applications. The RWS-DIP method uses both structure and sparsity priors to improve the efficiency of the learning. The structural prior is introduced by employing a reference image as the network input, and the sparsity prior is explored by regularizing the l 1 norm of wavelet coefficients. Experimental results on invivo MR scans show that the RWS-DIP method can achieve improved reconstruction performance and outperforms traditional DIP in preserving texture details and removing artifacts.
Two extensions can be made in order to improve the proposed scheme: (1) mining and incorporating more effective prior information may lead to a further boost in performance, particularly in regard to strengthening the use of structural prior information, and (2) further research is needed for the regularization effect introduced into DIP, which will guide the design of complementary regularizations, so as to achieve a stronger effect and better performance.

Data Availability
The data used to support the findings of this study are available from the corresponding author on reasonable request.

Conflicts of Interest
The authors declare no conflict of interest.