An algorithm was developed to segment solid pulmonary nodules attached to the chest wall in computed tomography scans. The pleural surface was estimated and used to segment the nodule from the chest wall. To estimate the surface, a robust approach was used to identify points that lie on the pleural surface but not on the nodule. A 3D surface was estimated from the identified surface points. The segmentation performance of the algorithm was evaluated on a database of 150 solid juxtapleural pulmonary nodules. Segmented images were rated on a scale of 1 to 4 based on visual inspection, with 3 and 4 considered acceptable. This algorithm offers a large improvement in the success rate of juxtapleural nodule segmentation, successfully segmenting 98.0% of nodules compared to 81.3% for a previously published plane-fitting algorithm, which will provide for the development of more robust automated nodule measurement methods.
One of the most reliable indicators of the malignancy of a pulmonary nodule is its growth rate [
The problem of isolated nodule segmentation has been well studied; isolated nodules can often be segmented via intensity and shape-based methods, as in one method proposed by Zhao et al. [
Example of a juxtapleural nodule, with (a) one central slice from CT scan and (b) 3D visualization of region of interest.
We propose a method to segment a nodule from the thoracic wall by fitting a polynomial function to the pleural surface. Surface-fitting methods have been used to solve problems in several different areas, including range [
The algorithm relies on several assumptions, the principal one being that the chest wall is a large surface with a curvature lower than the nodule surface. These assumptions are used in the development of a robust algorithm for juxtapleural nodule segmentation.
We model the juxtapleural nodule into several different regions, illustrated in Figure
Juxtapleural nodule model, with the modeled pleural surface (MPS) separating the nodule (N) from the thoracic wall (TW). The lung parenchyma in the region indicated by LP.
The pleural surface is modeled as a 3D cubic polynomial function, and the goal of the surface-fitting algorithm is to determine the polynomial function that best fits the pleural surface. To accomplish this, the points belonging to the pleural surface are identified and used to estimate the parameters of the polynomial function. The algorithm is divided into the six stages shown in the flowchart in Figure
Flowchart overview of algorithm.
The surface points in a region of interest are identified by first segmenting the nodule from the lung parenchyma and other soft tissue attached structures. After the preliminary segmentation of the nodule, the next task is to compute a coordinate transformation to assist in later stages of the algorithm that require computing the residuals from the estimated polynomial function. To ensure that the estimated polynomial function is computed from just the pleural surface points (excluding points belonging to the surface of the nodule), the next three steps encompass an iterative process that selects a subset of the surface points in the region, estimates a polynomial function from the subset of points, and determines if the change in residuals corresponds to the estimate of the polynomial surface that includes all the pleural surface points but none of the nodule surface points. Finally, in the last stage, the estimated surface function is used to segment the nodule from the pleural surface.
The surface-fitting algorithm is an extension of previous work on pulmonary nodule segmentation by Reeves et al. [
The nodule segmentation system by Reeves et al. [
In the image processing step, a small region of interest (ROI) sized more than twice the nodule diameter is extracted from entire CT scan centered at the seed point. From this ROI, an estimate of the nodule size and location is computed using an iterative template matching technique. This information is used to further reduce the size of the ROI. The small ROI is resampled into isotropic space (0.25 mm3 voxel size).
Next, the soft tissue in the resampled ROI is separated from the lung parenchyma by applying a threshold of −400 HU to the ROI. This results in a binary image with the lung parenchyma assigned a value of 0, and the soft tissue assigned a value of 1. To separate the nodule from other small, attached soft tissue structures, an iterative morphological filter is applied to the binary image which removes attached structures such as blood vessels but will not remove larger attached structures such as the pleural surface.
After these steps, we have the following information: (1) the volumetric region of interest resampled into isotropic space, (2) an approximate nodule radius, (3) an approximate nodule center, and (4) isotropic binary thresholded volume with the vessels removed. These are all used by the algorithm in subsequent steps.
The nodule is separated from the pleural surface by creating a boundary based on local shape information. The boundary is modeled as an explicit polynomial surface of the form
This explicit function requires finding the parametrization of the surface using techniques similar to those described by Quek et al. [
For an explicit surface function, we can improve the error computation by choosing a coordinate system that is parallel to the pleural surface. As shown in Figure
Estimating distances from a set of surface points to an estimated surface in (a) the original coordinate system, the error estimated by the observable variable (dashed lines) differs greatly from the actual error (solid lines), but if (b) the pleural surface is parallel to the explanatory variable plane, the error estimated by the observable variable provides a good approximation to the actual error.
Overview of coordinate system selection.
In addition to the basis vectors of the coordinate system, computing a coordinate transformation requires a point for the origin of the coordinate system. In this method, a reference point
Procedure for finding nodule surface point. Beginning from the center of mass (COM), the algorithm searches for a surface point (
At this point, we have a coordinate system and points along the boundary of the thoracic wall/nodule region and the lung parenchyma. To prevent bias in the surface estimate, the surface points belonging to the nodule must be excluded. We use the following model as a basis for deciding which points belong to the nodule.
We consider the region of interest containing the nodule to consist of two sections: the pleural surface and the nodule surface. These sections can be separated by partitioning the region of interest into two subregions. One such partition is illustrated in Figure
(a) Model of region of interest, with nodule subregion indicated by the dotted line and (b) region excluded for pleural point subset selection, indicated by gray-shaded circle (all surface points outside the circle,
We define the following parameters that can be found without knowledge of the exact partition. A point that is known to be on the nodule surface is the reference point
Outliers or surface irregularities can be identified based on their inconsistency with the surface fit. A distance-based measure, such as mean squared error, can be used to determine the consistency of points with a surface model. To ensure that the mean squared error can be used to detect outliers, a surface model must be selected that can represent clean sections of the pleural surface but not ones containing the nodule. We make the following assumptions about the nodule surface: the nodule surface contains multiple inflection points, all the nodule surface points are on the lung side of the pleural surface, and the points that are farthest from the origin are adjacent to the pleural surface.
Inflection points are defined as points where the convexity of the surface changes. The presence of multiple inflection points differentiates a surface containing the nodule from the normal pleural surface. Several inflection points, labeled
Model of a juxtapleural nodule with inflection points labeled as
In contrast to the nodule, the pleural surface has low curvature with few inflection points. Figure
Pleural surface: (a) slice of a whole lung scan, (b) outline from a slice of a whole lung scan.
Once we have a method to select which surface points to include in our surface estimation algorithm, our next step is to find the optimal polynomial function that fits those points. An overview of the algorithm is shown in Figure
Overview of pleural point subset selection and surface parameter estimation.
The algorithm uses a forward search, starting with a small subset of points that is known to contain no or few outliers. The radius of the initial clean subset is initialized to a value of 10% higher than the known clean subset
As the algorithm iterates between
Example of forward search algorithm showing the central slice of a nodule, with the top row of images displaying several different radii of the region excluded from the subset of pleural surface points, and the bottom row of images displaying the resulting segmentations. Radius
To detect the change in behavior of the diagnostic sequence, we used a likelihood-based approach for constant level signals with noise described by Gustafsson [
Finally, the most likely sequence is found by minimizing the following expression:
To find the change time for the sequence of diagnostic values, we formulate the problem using the framework just described. We treat the sequence of diagnostic values as a discrete time function and make the assumption that the function increases linearly, with a change in the slope of the function at the nodule radius,
To identify the time where the change in slope occurs, we can define a difference sequence
Once the surface parameters have been estimated, there is enough information to segment the nodule from the thoracic wall. We start with a binary image containing the thoracic wall and the nodule and eliminate voxels that are below the estimated pleural surface, where “below” is defined as the opposite direction of the average surface normal. Figure
Thoracic wall removal. Light-shaded visualizations of (a) original region of interest, (b) estimated surface function, and (c) nodule after thoracic wall removal.
This study used a dataset of 150 solid attached nodules with one primary attachment from 114 patients selected from the Weill Cornell Medical Center database. The solid consistency of the nodules was confirmed by a radiologist; juxtapleural nodules were noted by a radiologist and confirmed by visual inspection. Of the 150 nodules, six nodules were on whole-lung scans while the remainder were on targeted scans. All nodules were imaged on thin-slice scans, with 129 nodules on 1.00 mm scans and 21 nodules on 1.25 mm scans. All of the scans were acquired using the scanners and parameters shown in Table
Image acquisition parameters.
Parameter | Value |
---|---|
Scanner | GE HiSpeed CT/i, LightSpeed Ultra, LightSpeed QX/i |
Current | 40–340 mA |
Voltage | 120, 140 kVp |
The “true” segmentation of a juxtapleural nodule is difficult to accurately determine, even for radiologists. Studies have shown that there is high interobserver variability in nodule measurements [
Examples of nodule segmentation ratings, (a) the segmentation includes extensive nonnodule regions, (b) the segmentation does not include the entire nodule, (c) the segmentation stops slightly short of the wall, and (d) the segmentation includes all of the nodule but none of the wall. Segmentations with a rating of 3 or 4 were considered acceptable. The top image shows the grayscale region of interest, while the bottom image shows the segmentation, with the nodule indicated in white and high-intensity nonnodule structures indicated by gray. Images were all windowed to enhance visibility and are not to scale.
The new surface-fitting method was compared to the latest published method from Reeves et al. [
Nodule segmentation results comparing the surface-fitting method described in this paper with a previously published method using a plane-cutting approach.
Method | Nodules segmented | Percent segmented |
---|---|---|
Surface fitting | 147 | 98.0% |
Reeves et al. [ | 122 | 81.3% |
Total number of nodules | 150 |
The average rating of the surface-fitting algorithm was 3.28 over all the nodules, while the average rating for the algorithm by Reeves et al. was 2.95, with the distributions shown in Figure
Distribution of ratings for segmentations by the surface-fitting and plane-cutting methods.
The runtimes of both the methods were measured on a dual-processor Intel Xeon 3.0 GHz computer. Both methods were implemented in unoptimized research software. For most nodules, the runtimes of both methods were only a few seconds. The surface-fitting algorithm was slower than the plane-cutting method, with a range of runtimes of approximately 200 ms to 14 seconds. The plane-cutting method had runtimes which ranged from approximately 100 ms to 10 seconds. The runtimes of both methods were higher for larger nodules.
A new algorithm for juxtapleural nodule segmentation was developed which combined robust surface estimation methods with knowledge of the characteristics of juxtapleural nodules to improve upon previous segmentation algorithms without requiring any additional user intervention. Although the majority of the nodules in this study were on targeted CT scans, the algorithm should work effectively on whole-lung CT scans as well.
Unlike previous studies which reported their segmentation results on both isolated and attached nodules, the dataset used in this study consisted of only the more challenging juxtapleural nodules. On this set of juxtapleural nodules, this new algorithm performed better than a previously published method [
Example of a nodule where the segmentation performed by the surface-fitting algorithm is better than the segmentation performed by the plane-cutting method, with (a) the region of interest containing the nodule, (b) the nodule segmentation via the surface-fitting algorithm, and (c) the segmentation by the plane-cutting method. In (b) and (c), areas of white indicate nodule.
Although there was an improvement in the success rate of nodule segmentation with the surface-fitting algorithm, there were still several cases where both algorithms failed. Many of these failures were due to either respiratory motion, or an apparent shift of the nodule attachment point along the pleural surface. In one case, the nodule was located near the diaphragm. Several CT scan slices of the region of interest for this nodule are shown in Figure
Several slices of the region of interest containing a nodule that the (a) surface-fitting and (b) plane-cutting methods failed to properly segment. The nodule was located near the diaphragm; notice that there was a large change in the position of the wall in frames 9 to 11. The voxels included in the segmentation are indicated in white, while high-intensity nonnodule voxels are shown in gray.
Example of a nodule for which both algorithms segmented a portion of the wall due to respiratory motion in frames 13–18. The voxels included in the segmentation are labeled in white for the (a) surface-fitting method and (b) plane-cutting method.
Example of oversegmentation by both algorithms. (a) An attached structure is included in the segmentation by the surface-fitting method as well as the (b) plane-cutting method, though not to as great an extent.
The runtime of the surface-fitting method was slightly longer than the runtime required for the plane-cutting method. Much of the runtime occurred in the iterative process of selecting a set of pleural surface points and using these points to estimate the parameters of a polynomial function. The runtime could be dramatically reduced by additional program optimizations, possibly by a factor of 10 or more for the larger nodules.
In this study, the segmentation results were subjectively evaluated by a three raters. Having a radiologist manually contour, every nodule on each slice is time-consuming, and though the Lung Image Database Consortium (LIDC) [
We have presented a robust, surface estimation approach to accurately segment solid juxtapleural nodules. In contrast to previous approaches using morphological filtering, plane-cutting, or convex hull operations, this approach fits a polynomial function to a robust set of pleural surface points. We evaluated the performance of this algorithm on a database of 150 solid juxtapleural nodules and compared its performance to a previously published method using an iterative plane-cutting algorithm. Our method performs much better than the plane-cutting approach, correctly segmenting 98% of the nodules compared to 81% with the previous method. The surface estimation approach especially excels with nodules attached to pleural surfaces with high curvature. However, the algorithm is still affected by image problems such as respiratory motion, but this will become less of a problem with improvements in CT scanner technology. This approach improves the success rate of juxtapleural nodule segmentation and will allow for more accurate volumetric measurement of juxtapleural pulmonary nodules.
D. Yankelevitz is a named inventor on a number of patents and patent applications relating to the evaluation of diseases of the chest including measurement of nodules. Some of these, which are owned by Cornell Research Foundation (CRF), are nonexclusively licensed to General Electric. As an inventor of these patents, D. Yankelevitz is entitled to a share of any compensation which CRF may receive from its commercialization of these patents. C. I. Henschke is a named inventor on a number of patents and patent applications relating to the evaluation of disease of the chest including the measurement of nodules. Some of these patents, which are owned by the Cornell Research Foundation (CRF), are nonexclusively licensed to GE Healthcare. As an inventor, C. I. Henschke is entitled to a share of any compensation that CRF may receive from the commercialization of these patents but has renounced any compensation since April 2009. A. P. Reeves is a coinventor on patents and pending patents owned by Cornell Research Foundation, which are nonexclusively licensed to GE and related to technology involving computer-aided diagnostic methods, including measurement of pulmonary nodules in CT images.