^{1}

^{1}

^{2}

^{3}

^{1}

^{1}

^{2}

^{3}

Image registration is a fundamental task in medical imaging analysis, which is commonly used during image-guided interventions and data fusion. In this paper, we present a deep learning architecture to symmetrically learn and predict the deformation field between a pair of images in an unsupervised fashion. To achieve this, we design a deep regression network to predict a deformation field that can be used to align the template-subject image pair. Specifically, instead of estimating the single deformation pathway to align the images, herein, we predict two halfway deformations, which can move the original template and subject into a pseudomean space simultaneously. Therefore, we train a symmetric registration network (S-Net) in this paper. By using a symmetric strategy, the registration can be more accurate and robust particularly on the images with large anatomical variations. Moreover, the smoothness of the deformation is also significantly improved. Experimental results have demonstrated that the trained model can directly predict the symmetric deformations on new image pairs from different databases, consistently producing accurate and robust registration results.

Computer models have become a usable method for solving biomedical engineering and are applied to the analysis and measurement of data in the biomedical field (e.g., material mechanical behavior measurement [

In recent years, deep learning has been widely applied in medical image analysis [

However, for the aforementioned registration algorithms, it is difficult to accurately register the images with large anatomical variation, and the smoothness is even difficult to preserve and constrain for large deformation. Thus, it is essential to develop an algorithm, which can effectively register the images with large anatomical variations and, meanwhile, keep the transformation field smooth, so that the topology can be well preserved. In addition, symmetric diffeomorphic registration has achieved better performance overall, which estimates symmetrical deformation pathway from two objects (template and subject) to the intermediate point instead of a single pathway from template to subject [

In this paper, we further investigate the deep-learning-based registration by considering the symmetric property. We propose a symmetric registration network (S-Net) by simultaneously aligning the subject and template images to an intermediate space, i.e., the pseudomean space. Specifically, instead of establishing the voxel-to-voxel correspondences in one pathway, i.e., from template space to subject space, we move the template and subject images symmetrically, until they meet in the pseudomean space. In this space, the image similarity is maximized. The main contribution of this work can be summarized as follows:

We propose a symmetric registration network that can register images in the dual direction simultaneously. In this framework, the pseudomean space can be automatically learned by using the symmetric constraint without any supervised guidance.

The symmetric property allows for estimating two short deformation pathways instead of directly estimating a long deformation pathway. It is more effective to register images with large anatomical variations. The final registration result can be more accurate and smoother.

Under the symmetric framework, we can directly obtain the forward (register subject to template) and backward (register subject to template) transformation fields by using the trained S-Net. Therefore, the inverse consistency can be achieved without introducing any additional model or strategy.

The S-Net is trained in an unsupervised manner based on the proposed symmetric way. As shown in Figure

Overview of our method. Learning the parameters of S-Net by unsupervised training. The input consists of subject image

Mathematically, the optimization of symmetric registration can be formulated by minimizing the image dissimilarity in the pseudomean space:

In the testing stage, giving an unseen image pair and their difference maps, we can get their symmetric deformations

The symmetric image registration scheme. (a) Illustration of the hypothesis of the symmetric image registration; (b) the whole deformation field from template to subject can be calculated by

For symmetric registration, the pseudomean is an intermediate space on the image manifold, and the distance between the pseudomean and the template should equal that between the pseudomean and the subject. Therefore, for each location/voxel, the deformation magnitudes of

The S-Net was designed based on the network architecture designed in VoxelMorph [

The similarity loss of the registration task is used to evaluate the registration accuracy, and here, we define the similarity loss by SSD. Conventionally, the subject image should be warped to the template space by the output deformation field, and the loss is calculated in the template space. For the symmetric registration network, we define the similarity loss in the pseudomean space to penalize the symmetric property. Mathematically, it can be formulated as

By minimizing the symmetric similarity loss

where

The regularization loss is used to constrain the smoothness of the estimated deformation field

(1) Laplace smoothness

where

(2) Zero constraint: modifying the displacement value for avoiding unreasonable large deformations:

(3) Antifolds constraint: adding an antifolds constraint [

The final loss function for training the S-Net is

where

The S-Net is implemented in Keras and trained on an NVIDIA Tesla V100 GPUs with 32 GB of video memory. The network is trained by using the Adam strategy [

We used 30 subjects from LONI LPBA40 dataset as the training data, and

We have compared our results with three state-of-the-art registration methods, namely, D. Demons [

The results of DSC scores and runtimes are shown in Table

Dice score (%) for subject-to-subject alignment using Demons, SyN, VoxelMorph, and the proposed S-Net.

Dataset | D. Demons | SyN (CC) | VoxelMorph (CC) | VoxelMorph (MSE) | Proposed method |
---|---|---|---|---|---|

LPBA40 | 68.7 | 71.2 | 71.6 | ||

IBSR18 | 54.6 | 54.2 | 55.2 | ||

CUMC12 | 53.1 | 51.8 | 53.1 | ||

MGH10 | 60.4 | 59.6 | 60.2 | ||

Time (s) | 114 | 1330 | 3.6 |

Folds (

Dataset | D. Demons | SyN (CC) | VoxelMorph (CC) | VoxelMorph (MSE) | Proposed method |
---|---|---|---|---|---|

LPBA40 | 13.71 | 28.52 | 44.04 | ||

IBSR18 | 15.59 | 44.26 | 67.57 | ||

CUMC12 | 21.02 | 39.37 | 48.92 | ||

MGH10 | 18.92 | 42.17 | 56.72 |

The respective results and intermediate results are also shown in Figures

The results of the S-Net. From left to right, the column shows subject, final warped template image, middle warped subject image, middle warped template image, final warped subject image, and template image. (a) Subject image. (b) Final warped template image. (c) Middle warped subject image. (d) Middle warped template image. (e) Final warped subject image. (f) Template image.

It is worth noting that S-Net achieves image registration tasks in an unsupervised end-to-end fashion by using an image similarity metric for optimization so that the training of this S-Net does not require the known deformation field, which is difficult to obtain for medical image registration. Furthermore, we have also evaluated our framework for the number of folds with the traditional registration method and single-direction deep-learning-based registration method. The deformation maps estimated by the proposed S-Net tend to be smoother, since the symmetric displacement map only needs half pathway, instead of a long pathway, which is easier to penalize the smoothness. Experimental results showed that our method successfully reduces the folds of estimated maps while providing more accurate registration results.

S-Net learns for image registration tasks in an unsupervised end-to-end fashion using an image similarity metric for optimization so that the training for this S-Net does not require the known deformation field, which is difficult to obtain for medical image registration. Furthermore, we have also evaluated our framework for the number of folds with the traditional registration methods and single-direction deep-learning-based registration methods. The deformation maps estimated by the proposed S-NET tend to be smoother, since the symmetric displacement map only needs a half pathway, instead of a long pathway, which is easier to penalize the smoothness. Experimental results showed that our method successfully reduces the folds of estimated maps while providing more accurate registration results.

The total loss function in S-NET consists of two types of six losses. However, the multiple losses weight (hyperparameters) of our S-NET training is hard to balance. Therefore, we did some experiments to determine the weight of multiple losses in Figure

Effect of varying the regularization parameters

We presented a new symmetric training strategy for an unsupervised deep-learning-based registration framework, which can better estimate the large local deformation during registration. In particular, we utilize a pseudomean as an intermediate target registration space, and a long deformation pathway can be divided into two short deformation pathways. Experimental results have shown promising registration performance for both accuracy and field smoothness.

The databases of LPBA40, IBSR18, CUMC12, and MGH10 can be downloaded from the registration grant challenge at

The authors declare that they have no conflicts of interest to report regarding the present study.

This work was supported by the National Natural Science Foundation of China (Grants nos. 81871508 and 61773246), the Major Program of Shandong Province Natural Science Foundation (Grant nos. ZR2019ZD04 and ZR2018ZB0419), and the Taishan Scholar Program of Shandong Province of China (Grant no. TSHW201502038).