In rodent brains images may become distorted due to instrument imperfections or, in the case of histology, tissue processing. For example, magnetic field inhomogeneity causes geometric distortions in echo planar imaging (EPI) MRI; mechanical forces acting on a harvested brain during slicing may cause tissue tearing. And chemical preparation for histological analysis may cause deformation of tissue, evident in histological micrographs. Correcting (or “rectifying,” “warping”) the distorted images is required to adequately represent rodent brains (in the case of EPI distortion in MRI), or to compare different acquisition modalities (e.g., comparison of in vivo images and histological micrographs). Normally, affine (linear + translation) transformation cannot reconcile severe distortions making nonlinear transformation necessary.
Among the nonlinear transformation techniques, point-based interpolating transformation techniques are widely employed because they are easy to implement and flexible for different applications [
Accurate interpolating transformation requires an exact match of homologous landmarks. Manual identification of landmark points is time consuming and prone to intra- and interobserver variations. A number of investigators have attempted to automate the landmark definition process by exploiting the geometry of anatomical or biological structures. Typical geometrical features include line intersections [
In the current investigation, we developed a technique to automate and optimize the landmark generation using the local curvature on anatomical contours and validated the technique. The technique presented here is for two dimensional (2-D) brain image nonlinear registration. Although the rodent brain is a three dimensional (3-D) object, some brain imaging studies are essentially carried out on 2-D planes such as in histological microscopy, and in 2-D MRI studies. More importantly, nonlinear distortion mostly occurs on 2-D as well in these imaging studies. For example, tissue tearing, shearing, shrinkage, and enlargement, during sectioning and section handling, and eddy current in 2-D EPI MRI acquisition, mostly cause 2-D in-plane nonlinear distortions. To guarantee the correspondence of the two brain images to be registered, some prior process 3-D registration may be necessary. Detailed description of the process follows.
The strategy of this method is first to generate contours on corresponding anatomical features on the source and target images and then generate landmarks on homologous anatomical contours. The landmarks are then relaxed from the original locations and allowed to slide along the contours to achieve optimal matching. The relocation of the landmarks is governed by a cost function constituted by the local curvatures of landmarks and their displacements. The procedure is described using the pseudocode in Table
Pseudocode of the landmark generation and optimization technique.
(1) |
(2) |
(3) |
For |
Add the |
Optimize landmarks by minimizing the cost function in ( |
End |
(4) |
The homologous contours on which the landmarks are generated can be manually drawn on the images or identified as the borders of objects using border detection methods, or first identified automatically using border detection methods and then manually modified to correct errors resulting from noise and artifacts. Depending on the image properties, appropriate border detection methods such as dynamic programming or active contour models [
The next step is landmark generation and optimization on each curve. An example of this step is illustrated in Figure
An example of the landmark generation and optimization. The left and right columns show the source and target curves, respectively. (a) The first two landmarks (1 and 2, circled X) and their homologues (1′ and 2′, circled X) are fixed at the two ends of the corresponding source and target curves. The 3th landmark (
The registration of MRI and histological slices using optimized landmarks on curves (a) and (b), manually selected discrete landmarks (c) and (d). The colors of homologous landmarks are matched in (c) and (d). The MRI slice transformed using the thin-plate splines with optimized landmarks in (a) and (b) is shown in (e), and overlaid on the histological section (f). The average NMI and TRE of transformed MRI and histological slices in five image pairs (I–V) were shown in (g) and (h), respectively. The NMI and TRE were calculated on the MRI slices transformed using manually selected “discrete landmarks (LMs)” (black columns), landmarks generated on curves but not optimized (“Nonoptimized LMs on curves”) (white columns), curve landmarks optimized using (
The correction of the geometrical distortion on a B0 image (a) in EPI DTI employing the T2-wt image (b) as the reference by optimized landmarks on curves (a) and (b), and manually selected discrete landmarks (c) and (d). The colors of homologous landmarks are matched in (c) and (d). The B0 image transformed using the thin-plate splines with optimized landmarks in (a) and (b) is shown in (e), and overlaid on the T2-wt image plotted in pseudocolor (f). The average NMI and TRE of corrected B0 and T2-wt images in five image pairs (I–V) were shown in (g) and (h), respectively. NMI and TRE were calculated on the B0 images transformed using manually selected “discrete landmarks (LMs)” (black columns), landmarks generated on curves but not optimized (“Nonoptimized LMs on curves”) (white columns), curve landmarks optimized using (
The landmark generation and optimization technique was evaluated using two types of mouse brain imaging studies. In the first study, mouse brain MRI was nonlinearly registered with histological sections. The second study examined the ability of the algorithm to correct geometric distortion of EPI—a fast MRI acquisition technique. The convergence criterion of the cost function minimization was set as either landmark displacement of less than 10−4 pixels in subsequent iterations or a maximum of 500 iterations, and the weighting factor
Mouse brains were harvested after 3D T2*-weighted MRI, fixed and embedded in paraffin for histological sectioning. The blockface of the embedded brain was photographed during sectioning (blockface imaging). Individual blockface images were stacked to reconstruct the 3D brain volume. Brain slices were stained with Prussian blue and hematoxylin. The MRI volume was linearly registered to the blockface volume, and then computationally resliced in the coronal plane to match the corresponding histological sections [
Five mice were scanned with T2-weighted (T2-wt) spin-echo imaging and diffusion tensor imaging (DTI) with EPI acquisition. The MRI slices were prescribed at the same anatomical locations in these scans. In the DTI EPI scan, a baseline image without diffusion weighting (B0) was also acquired. The B0 images were used as the source images, and the T2-wt images were used as the target images for registration. Since the B0 image and all diffusion-weighted images undergo same geometric distortion, the transformation procured from the T2-wt/B0 registration is applied to correct the diffusion-weighted images. The B0 images were registered to the T2-wt images by the same three technicians in the MRI/histology registration using manually selected landmarks, landmarks manually adjusted on anatomical contours, and automatically generated and optimized on contours, respectively.
Figures
Registration results with landmarks generated using different methods in five pairs of MRI and histological slices (from Pair I to V) were compared. The mean and standard error of the NMI and TRE of each pair of images by the three technicians is presented in Figures
As demonstrated in Figure
A typical pair of B0 and T2-wt images is shown in Figures
The same statistical analysis was performed in the EPI distortion correction as for the MRI/histology registration, allowing a direct comparison of the results (Figures
In this study, we developed a landmark generation and optimization technique for point-based nonlinear image registration methods. This technique extracts landmarks from the anatomical contours and optimizes the landmark positions by minimizing the cost function constituted by the displacements and the local curvatures of the landmarks. The technique was evaluated in two distinct applications: the registration of MRI and histological slices and distortion correction of EPI MRI images. Statistical analyses have shown that the automation of landmark selection resulted in significant accuracy improvement in image registration compared to manually selected landmarks. Although in most experiments the improvement in NMI and TRE resulting from the landmark optimization was not statistically significant compared to the results using nonoptimized landmarks, the trends towards improvement in registration accuracy was demonstrated in several experiments. Manually adjusting the landmarks on curves could improve registration accuracy on several experiments and show a trend towards improvement in some other experiments. We found no difference in registration accuracy between landmark automatic optimization and manual adjustment. However, automated landmark selection provides increased efficiency by minimizing the required user intervention.
We used two methods including NMI to validate our technique. NMI has been intensively used as an image similarity measure. It is not a monotonic function of the image similarity and, thus, may be trapped at local minima when used as a driving force for image registration. In this study, the image pairs to compare have all been already registered; thus, NMI was only calculated on a small interval on which it is reasonable to think that NMI is monotonic. TRE was also calculated for the registration evaluation in this study. TRE is a measure of the registration accuracy of a set of points on the images. The points for TRE calculation are usually identified on anatomical features; thereby, the evaluation may be more meaningful than NMI with regard to the anatomical accuracy of registration.
The anatomical contours were manually drawn in this study. Border detection methods can be used to automatically identify the contours. It is reasonable to think that automatically generating contours can minimize inter- and intrauser variance. But on the other hand, some automatic border detection techniques are more susceptible to noise and artifacts compared to manual delineation. We are currently investigating the registration accuracy using different border detection methods.
Not only can this method be used in imaging studies similar to those presented here, but also can be used for multiple brain registration, or similarly, for registration to an atlas with minimum modification. Before using this method, a 3-D affine transformation is likely necessary to first align the brain volumes together, or to the atlas, and then each brain volume needs to be resliced to match individual brain slices, or to the atlas slices.
Overall, this method results in improved registration accuracy and efficiency. However, this technique still requires user intervention and thus suffers inter- and intra-investigator inconsistencies. We are currently improving this technique by including the image intensity and more geometrical information in addition to displacement and curvature to fully automate the landmark generation process. In this study, the number of landmarks either manually selected or automatically generated on curves was determined by the technicians according to their experience. It is desirable to precalculate the necessary landmark number for different landmark generation methods to improve registration accuracy. Investigations using previously published methods to accomplish this automation are underway [
This work has been supported by NIH Grants K25MH089851, P20 RR021937, P01 NS043985, P30 MH062261, and P01 DA028555. The authors acknowledge Erin McIntyre, Melissa Mellon, and Lindsay Rice for technical assistance.