Current clinical breast imaging modalities include ultrasound, magnetic resonance (MR) imaging, and the ubiquitous X-ray mammography. Microwave imaging, which takes advantage of differing electromagnetic properties to obtain image contrast, shows potential as a complementary imaging technique. As an emerging modality, interpretation of 3D microwave images poses a significant challenge. MR images are often used to assist in this task, and X-ray mammograms are readily available. However, X-ray mammograms provide 2D images of a breast under compression, resulting in significant geometric distortion. This paper presents a method to estimate the 3D shape of the breast and locations of regions of interest from standard clinical mammograms. The technique was developed using MR images as the reference 3D shape with the future intention of using microwave images. Twelve breast shapes were estimated and compared to ground truth MR images, resulting in a skin surface estimation accurate to within an average Euclidean distance of 10 mm. The 3D locations of regions of interest were estimated to be within the same clinical area of the breast as corresponding regions seen on MR imaging. These results encourage investigation into the use of mammography as a source of information to assist with microwave image interpretation as well as validation of microwave imaging techniques.
X-ray mammography is the current gold standard breast imaging technique [
Different modalities rely on different tissue properties in order to generate an image; for example, X-ray image contrast results from tissue density, whereas ultrasound imaging relies on acoustic impedance. Examining multiple modalities can provide diagnostic information that might be missed if only a single imaging technique was used [
Various studies have measured differences in the electromagnetic (EM) properties of fatty, fibroglandular, and cancerous breast tissues, suggesting a possible source of image contrast [
Tissue sensing adaptive radar (TSAR) is a 3D radar-based microwave imaging technique that is currently in the preliminary stages of clinical testing [
Mammograms are a routine procedure in breast cancer cases [
To use mammograms to interpret TSAR data, the 2D information must be translated into 3D space. However, mammograms are only obtained at two orientations, approximately 45° apart. Furthermore, the breast is compressed up to 50% of its original diameter, causing significant distortion of the 2D images [
This paper presents a method to estimate the 3D skin surface and 3D location of regions of interest from standard two-view mammograms. The accuracy of the estimation is quantified by directly comparing to MR images and computing the difference between estimated and true skin surface in four anatomical directions.
Previous work in reconstructing the surface of the skin from mammograms has been presented in only three related publications [
Figure
Overview of the estimation algorithm. Dotted outlines indicate steps performed only on images with identifiable features.
First, the distortion due to compression is compensated through registration with a 3D reference shape. Next, the two mammographic views and their corresponding skin contours are aligned spatially to represent physical imaging planes. The contours are then used to estimate the skin surface by fitting ellipses to coronal slices of the breast at equally spaced intervals. In cases where features such as lesions could be identified on all data sets, the 3D locations of the features are estimated through 3D backprojection. Finally, the skin surface and features are rendered in 3D and compared to MR images.
In acquiring mammograms, the breast is compressed up to 50% of its original diameter, causing significant anatomical distortion of the tissues and leading to CC and MLO projection images consisting of different tissue configurations [
The registration technique used a combination of landmark- and intensity-based registration to reduce distortion of both the external shape and the internal tissues of the mammograms [
Original mammogram (left) was registered to an MR projection image to remove the distortion resulting from mammographic compression. The resulting image (right) is reduced to approximately 70% of the original surface area.
The three circular markers of Figure
The preprocessing procedure of the previous section served to map the 2D mammographic projections into an undistorted 2D projection space. However, the relative alignment of the CC and MLO mammograms is unknown, as a given feature visible in the projected image can be located at any point along the X-ray beam vector (illustrated in Figure
Illustration of MLO mammographic acquisition angles.
Left
Right
Unlike previous work, which assumed a 90° difference of projection angles between the CC and MLO views, this work uses the MLO acquisition angle provided in the metadata of digital mammograms. The MLO data (image, landmarks, and contour) were rotated according to the reciprocal of this angle in order to spatially align the two views. Figure
Following rotation, translation of the two images was required to further align the views in an approximation of acquisition geometry. The nipple landmarks were chosen as the
Further spatial alignment required further assumptions. The contour of the mammogram, describing the largest edge or shadow of the breast, is located at the centre of the volume along the X-ray beam vector. The MLO data provide nipple location in the The CC data provide nipple location in the The centroid of a coronal slice taken at the chest wall corresponds to the intersection of the midpoints between the chest wall landmarks of both views.
Using these assumptions, the two image contours were aligned by rotating the CC view about the
Sparse wireframe formed by spatially aligning CC and MLO breast contours.
Examination of a given
Ellipse in canonical position (
Any coronal plane intersects the wireframe model of Figure
To obtain the best fitting ellipse for the four known data points of a coronal slice, the centre point
With all the parameters of (
Sample skin surface estimation.
The skin surface estimate described in the previous section describes the general shape of the breast and provides a frame of reference to compare the two modalities. However, identification of a particular region of interest (ROI) is of even greater utility, providing the 3D location of particular features of the breast. As mammograms provide 2D views of ROIs, these regions can be identified and their 3D location computed and displayed relative to the skin surface estimation.
For the purpose of identifying the general ROI in 3D space, ROIs as seen on the undistorted mammograms were modelled as simple spheres. Corresponding features on the CC and MLO views were identified in consultation with a radiologist and marked as 2D circles through an interactive display. These points were then located in 3D space using only the mammographic data by calculating the intersection of the two lines orthogonal to the imaging planes as illustrated by Figure
Reconstruction of internal feature points.
This is computed as follows:
The skin surface estimation technique was applied to six sets of patient data, resulting in a total of twelve breast models. Mammograms were collected digitally according to the Canadian standards, yielding CC and MLO images at varying resolutions. Both contrast-enhanced and fat-suppressed T1-weighted MR images were collected, also in compliance with the Canadian breast imaging protocols. For the purposes of image registration and comparison with mammography, the fat-suppressed structural MR images were used.
With the assistance of a radiologist, regions of interest on the data sets were identified. Only one was a candidate for internal feature location estimation due to a need for discrete lesion visibility on all three images (CC, MLO, and MR).
For each data set, an MR image was loaded and displayed to provide ground truth geometry. This image was aligned as accurately as possible relative to the feature estimation model, but it should be noted that alignment is subject to errors due to the assumptions made in aligning the wireframe model.
Figures
Comparison of estimated skin surface with MR (sample data set).
Sagittal view
Axial view
Comparison of estimated skin surface with MR (data set with lesion).
Sagittal view
Axial view
Examination of the skin estimations of both Figures
The sagittal view of the estimation shown in Figure
A coronal cross-section of the sample data set at two different locations is shown in Figure
Comparison of estimated skin surface with MR. Arrows indicate error between estimated and true skin surface at four anatomical directions (arrow size exaggerated for visibility).
Near chest wall
Near breast centre
The skin estimation errors were computed such that negative errors represent underestimation of the breast surface. To observe the relationship between coronal slice location and estimation error, the errors were plotted against coronal slice location (
Skin estimation error versus coronal slice (
Sample data set
Data set with lesion
To examine trends across all data sets, the absolute values of the skin estimation errors were plotted against
Skin estimation error versus
Cranial and caudal
Medial and lateral
In both cranial-caudal and medial-lateral directions, the best-fit line for the combined data suggests a slight trend of increasing accuracy towards the chest wall. However, this may be partially a result of the nonelliptical shape of the breast at the chest wall as seen in Figure
Comparing Figures
Overall, the skin surface estimation was accurate to an average contour deviation of 13 mm in the cranial-caudal direction and 6 mm in the medial-lateral direction. This is considered acceptable given the number of assumptions required and the limited information available for estimating the 3D skin surface. As this is the first work to quantify the accuracy of a 3D skin surface estimation created from mammograms, it is difficult to determine whether this result is comparable to that of previous publications.
Figure
Comparison between feature reconstructed from mammograms and corresponding feature seen on MR.
Coronal view
Sagittal view
The radiology report for this patient states “mass lesion 4 o’clock right breast.” From visual inspection of Figure
The major difference between the methods described in this work and the methods of Kita et al. is the mapping of features into the prone acquisition geometry of MR imaging. As this similar to the geometry used for TSAR imaging, 3D ROI prediction from mammography will enable interpretation of TSAR images without reliance on MR data.
As a preliminary example, Figure
Comparison between experimental TSAR image and MLO mammogram with corresponding lesion encircled.
TSAR image (coronal)
MLO mammogram
The 3D model of Figure
Due to the immersion medium used in TSAR acquisition, the breast floats and changes shape relative to MR imaging, where the breast is hanging freely. This can be observed in Figures
The data for the TSAR image shown in Figure
A final application for the 3D estimation methods described in this paper is as a verification of features seen on mammography. Radiologists undergo extensive training in order to interpret medical images; without such experience, identification of the same feature on both mammographic views is difficult. However, once the ROI is reconstructed in 3D, it becomes obvious whether the two visible features correspond. Figure
Identification of non-corresponding lesions results in mapping outside the breast volume.
CC
MLO
3D reconstruction
The comparison between MR ground truth and mammographic estimation is subject to errors related to the assumptions in aligning the two mammographic planes (see Section
In addition, each step of the method described in this work requires significant estimation and a large number of assumptions. As such, the technique has inherent inaccuracies and details that cannot be recovered with only two mammographic views, and the estimation should not be taken as an exact model of the 3D breast. However, for the intended purposes of assisting with microwave image interpretation, the accuracies obtained should suffice.
The skin surface and internal feature estimation techniques described in this paper have been found to be sufficiently accurate to assist with microwave image interpretation and provide a means of comparing TSAR results to the gold standard of mammography. While similar estimation techniques have been presented in the past, quantification of accuracy as compared to MR imaging is novel in the literature.
Future work will be to modify the mammogram undistortion technique to account for the geometric differences introduced by the TSAR immersion liquid. Following this, the 3D mammographic information will be incorporated into current TSAR image processing protocols.
The authors would like to thank Dr. Bobbie Docktor for her assistance in identifying radiographic features.