We propose a medical image segmentation approach based on the Active Shape Model theory. We apply this method for cervical vertebra detection. The main advantage of this approach is the application of a statistical model created after a training stage. Thus, the knowledge and interaction of the domain expert intervene in this approach. Our application allows the use of two different models, that is, a global one (with several vertebrae) and a local one (with a single vertebra). Two modes of segmentation are also proposed: manual and semiautomatic. For the manual mode, only two points are selected by the user on a given image. The first point needs to be close to the lower anterior corner of the last vertebra and the second near the upper anterior corner of the first vertebra. These two points are required to initialize the segmentation process. We propose to use the Harris corner detector combined with three successive filters to carry out the semiautomatic process. The results obtained on a large set of X-ray images are very promising.
In some circumstances, it is not easy for humans to distinguish objects in X-ray images from their background. Developing algorithms and methods for obtaining a proper object extraction is one of the most important research topics in the image processing field. Computer-based image segmentation facilitates the domain expert work and can automate tasks dealing with interpretation of medical images.
In this paper, we focus on vertebra segmentation applied to X-ray images. This operation is generally the first step to be performed before any disease diagnosis or vertebral mobility analysis. Therefore, this segmentation process is an essential and critical task. Indeed, the segmentation should be effective enough in order to analyze the mobility of the spinal column and accurately estimate the movement of each vertebra.
The goal of the segmentation process is to exploit only the useful information for image interpretation. A wide variety of techniques and approaches have been proposed in the literature. We can cite active contours (or snake) which present a powerful method for edge extraction of objects having arbitrary shapes [
These two methods have recently been used as new paradigms for a large number of segmentation methods due to their flexibility to deform the shape that must be detected. Nevertheless, such methods have an inherent limitation that makes them nonsuitable for many medical segmentation tasks where an
In related works on medical images analysis, Luo [
Other segmentation methods are the template-matching approaches. These methods are used to identify simple geometric shapes like ellipses or parabolas in an image. They match a predefined template to the location of some extracted features such as image gradient, boundary points, or grey level value. These techniques are specific to the structure of segmentation. They can be easily implemented and can give effective results when an appropriate model is chosen [
Active Shape Model (ASM) [
This method is commonly used for MRI image segmentation in the brain area or for cardiac images. However, the quality of the segmentation is highly dependent on the initialization phase. A good initialization is required to accelerate and help the morphing phase to obtain effective results. The ASM relies on the fact that the search is based on an
In this context, Rueda et al. [
Another variation of the ASM method is Active Appearance Model (AAM) which is largely described in the scientific literature [
Several publications [
In this paper, we propose to use an Active Shape Model segmentation approach in order to extract vertebra contours. In addition, we focus on improving the initialization phase of this method. Therefore, we propose a semiautomatic method allowing to ideally place the mean shape on the vertebrae to be segmented. We achieve this task by using the Harris corner detector followed by a series of filters aiming to detect the two anterior corners of each vertebra on the X-ray image.
The structure of the paper is presented as follows: in Section
In this paper we propose a segmentation approach based on Active Shape Model in order to identify vertebra edges. This method allows to model vertebrae whose appearance and location in the spinal column differ depending on the patient.
The statistical nature of the method involves the use of sample shapes that can be adopted by the object model. The sample must be as representative as possible to improve the quality of the model. In fact, the ASM algorithm defines a set of forms that well characterize the shape to be identified. This set of shapes that contains the different variations of the mean shape depends on the sample. Therefore, if the created model is not realistic enough, it could accept some shapes that are not really corresponding to the desired shape or conversely reject the shapes that are good. This aspect is the first difficulty of the ASM-segmentation-based method. It is important to know or to estimate as precisely as possible the actual distribution of the shapes to model.
Once the model is determined, it can be used to detect other similar shapes in new images. To this end, the mean shape model is extracted and placed in an area of interest. The shape is then iteratively warped until it fits at best the real edge of the object.
The ASM method [
The steps of the ASM framework.
it consists of the placement of landmarks on the images in order to describe the vertebrae. The specialist knowledge can be included in this step.
all the marked shapes have to be aligned before the creation of the model. It could be useful for the specialist to build a model corresponding to a particular pathology. For instance, if he wishes to detect vertebra arthritis, the vertebrae of the sample is presented as a shade whiter than normal and shows an abnormal bone growths. Once the model is created, these same patterns can be found in an X-Ray with this disease.
placing the mean shape model on the area of interest. This step can be manual or semiautomatic.
each point of the mean shape evolves in order to fit the vertebra edge.
The goal of the learning phase is to build an image sample which will be the basis for the model creation. An annotated training set is used to build this model [
It is a common practice to choose as landmarks the corners of the vertebra and a reasonable number of equidistant points between the corners. Figure
Vertebra marking.
The shape of an object is represented by a set of
This marking phase is time-consuming as the specialist has to put the landmarks manually on the images. He can then determine the location of strategic points that will be used in the model. Furthermore, automated tools such as polygonal approximation can be considered to achieve this goal. However, a purely automatic marking requires noiseless images or a preidentified contour.
In addition, one can imagine semiautomatic systems where the user could correct the annotation.
The annotated shapes are generally positioned at various locations and orientations on vertebra edges. For this reason, it is necessary to align all these shapes in order to make a correct statistical treatment [
There are several alignment techniques, but the generalized Procrustes analysis is the most commonly used [ align each shape of the sample on the first one; repeat until convergence: compute the mean shape, adjust the mean shape: to a size, an orientation and an origin by default, to the first shape, align each shape on the mean shape.
The purpose of the iterative process is to reduce the dependency of the model to the first shape. Concerning the adjustment of the mean shape at the second step, we have chosen to align it to the first shape. An example of vertebra alignment is given at Figure
An example of alignment.
The mean shape is characterized by the arithmetic mean of coordinates describing each element of the sample after the alignment. We have
The mean shape constitutes the basis of the vertebra edge detection process. A set of possible models are derived from this mean shape by moving the points through specific directions corresponding to the eigenvectors of the sample variance-covariance matrix, (
The model (see (
This model is used to decide if an object from an image can be considered as acceptable. As the coordinates of the landmarks of an object are known and as the eigenvectors are unit vectors (
The values of the factors
The search initialization consists of placing the mean shape previously computed on the image as close as possible to the real object. This operation can be done manually or in a semiautomatic way. In a manual initialization, the user is prompted to select the left side of each vertebra by clicking on the left superior and inferior corners. The mean shape is positioned according to this information.
The semiautomatic initialization does not require more than two clicks to limit the search window, including the left edge of the
Reduction of the window search.
In this paper, we propose a set of steps in order to place the mean shape on the vertebrae, in a semiautomatic way. Figure
Procedure to detect corners of each vertebra.
The Canny filter [
However, the Harris detector produces a high number of corners as shown in Figure
Illustration of the effect of filtering out corners neighboring a contour (b) and the filtering of false corners (c).
Original edge
Corner detection (427 corners)
Filtering corners outside vertebra contours (196 corners)
Applying angle filter (59 corners)
Filtering the corners outside the vertebra contour is based on the search for neighboring points. During this process, some points belonging to the Canny edge can be filtered. This occurs when a Harris corner is isolated (e.g., in the case of the extremity of a contour) or when the edge is too small.
When Harris corner is used, if finding good neighbors from a distance equal to an estimated height of a vertebra is not possible, then, the point is eliminated. The effect of this filtering is shown in Figure
The step of angle filtering of false corners aims to eliminate the Harris corners belonging to an angle that are not similar to vertebra angles. The main idea is to compute for each assumed corner the angle formed by straight lines linking its neighbors (Figure
The angle filter process.
Case of a real corner
Case of a point belonging to the edge
The distance between the corner and its neighboring points plays an important role. Indeed, if the neighbors are too close to the corner, the angle may be too straight and lead to a reject of the real corner. On the other hand, neighbors too far from corner may lead to the acceptance of too many false corners.
Figure
The Harris corner detector provides a large set of points of interest. The previous filters reduce the number of elements on this set. The goal of this section is to determine exactly the
To this end, we propose an algorithm based on the idea that looking for this sequence of
In this kind of problem, the first idea—the simplest one—is to consider all the possible sequences between the upper anterior corner of the first vertebra and the lower anterior corner of the last one. To do so, we describe a procedure dedicated to the build of those sequences based on an initialization conducted by an operator. Let
Parameters used for the vertebra corners detection.
where
Furthermore, practice gives us an empiric relation between
We can therefore deduce
Once all the parameters are determined,
Every acceptable point is then added to the sequence
Finally, the function stops when the number of points in the sequence is equal to
Once all the recursive calls are terminated, the function provided as a result a set
The global algorithm is given at Algorithm
(i) (ii) (iii) (i) (ii) (i)
(i) (i) (ii) (iii) (iv) (v) Add Add Remove Remove
In order to clarify the algorithm, we propose at Figure
Illustration of the construction of the sequence
Upper corner detection with
Lower corner detection with
Final sequence
The previous steps allowed to determine the anterior corners position of every vertebra in the image. This way, it provides relevant information about the vertebra position, orientation, and height. Therefore, it becomes possible to precisely place the mean shape at every detected vertebra position in order to initialize the segmentation procedure.
The ASM search treats every landmark defining the starting shape. For each of these points the neighborhood texture is analyzed in a specific direction. This analysis is made by considering landmarks along the normal of the contour at the considered point (see Figure
Normal of the contours at each point of the profile.
In relation (
All these considerations are detailed at Algorithm
(i) (ii) (i) Compute Compute Compute
Various parameters can significantly influence the results. In the following section, we propose investigating the influence of each of the following parameters: the number of landmarks per vertebra, the profile structure, the number of images used to build the sample, the mean shape model.
The number of landmarks per vertebra (see Figure
Influence of the number of landmarks.
4 landmarks
8 landmarks
12 landmarks
20 landmarks
The second parameter influencing the segmentation results is the structure of the profiles used for the search process phase. The profile depends on two other parameters: the number of points by profile and the distance between these points. We can notice that, in order to ensure the independence of this distance with respect to the image size, its length is proportional to the vertebra area. After various tests, we conclude that a profile of seven points spaced by a distance equal to 5% of the vertebra size is a good compromise.
The size of the sample remains the most important aspect of the ASM method. It is the basis for building the statistical model of shape and determines the outcome of the segmentation, the final result in an instance of this model. The robustness of the method is reached only if the sample is as representative as possible of the data segmentation.
The specialist knowledge and the practitioner expertise can play a crucial role in the choice of the images for the sample.
In order to test this parameter, we used the single model of vertebra. The ASM search initialization is performed through the user intervention to mark the left side of the vertebrae C3 to C7 by clicking on the anterior superior and inferior corners.
Figure
Influence of the sample size on the mean error of segmentation.
In order to represent the accuracy of the segmentation, we use a particular measure, that is
Point-to-line distance between 2 contours.
We propose two models: the column model and the vertebra model. The first aims to describe all the vertebrae into a single form and thus contains the coordinates of their landmarks. Figure
Effects of variations along the principal directions of a column mode.
−100%
0%
100%
The main advantage of the column model is that it changes the whole column during the search process. A vertebra cannot be rotated independently of others. This can be useful to determine the curvature of spinal column. By consequence, this advantage becomes an obstacle for the detection of a vertebra different from others; hence isolating anomalies are more difficult.
The vertebra model consists of modeling every vertebra by only one model. Therefore, it allows to resolve the shortcomings of the global column model. It is also more suitable for local search in the image. Nevertheless, it has the disadvantage of ignoring information that exists between the vertebra shapes, since each of them can evolve independently.
Figure
Effects of variations along the principal directions of a vertebra model.
−100%
0%
100%
Table
Success rate of segmentation with a sample of 75 images.
Sucess Rate (%) | ||
Type of Vertebra | Column Model | Vertebra Model |
C3 | 54.9 | 92.2 |
C4 | 80.4 | 98 |
C5 | 82.4 | 96.1 |
C6 | 76.5 | 96.1 |
C7 | 56.9 | 98 |
Figure
Results of segmentation using the vertebra model.
Example 1: Initialization
Example 1: Segmentation
Example 2: Initialization
Example 2: Segmentation
Comparison with other approaches is quite difficult. The main reason is that the methods proposed in the literature are applied in different contexts. For instance, the imagery modality is not always the same or the type of vertebra is different. In [
In this paper we presented a vertebra segmentation method using an Active Shape Model recognition approach. The Active Shape Model segmentation method is composed of two phases: a modeling phase, aiming to create a mean shape model, and a searching phase. An important challenge on applying this approach is the impact of the initialization, that is
Another inconvenient principal of the ASM-based segmentation approach is the training stage, for which we constructed the mean shape model by using, respectively, 50, 75, and 100 sample images. The choice of a sample of 75 images produces comparable results with the sample of 100 images.
In addition, we investigated the influence of various other significant parameters on the segmentation results. Thus, we studied the influence of the number of landmarks per vertebra, the profile structure, and the mean shape model. We concluded from this study that the best compromise is to choose 20 landmarks per vertebra and a profile structure of seven points. We also noticed that the results given by the vertebra model were more efficient than those given by the column model.
The various tests that we carried out on a large dataset prove the effectiveness of the suggested approach. We observe that the proposed method allows fast and efficient vertebra contours extraction. Our method can also be adapted to other components of the spinal column: like dorsal or lumbar. In our future works we want to investigate a method aiming to automate the segmentation. We consider also the use of the segmentation results to analyze the mobility of the cervical spinal column.