To solve the problem of scoliosis recognition without a labeled dataset, an unsupervised method is proposed by combining the cascade gentle AdaBoost (CGAdaBoost) classifier and distance regularized level set evolution (DRLSE). The main idea of the proposed method is to establish the relationship between individual vertebrae and the whole spine with vertebral centroids. Scoliosis recognition can be transferred into automatic vertebral detection and segmentation processes, which can avoid the manual data-labeling processing. In the CGAdaBoost classifier, diversified vertebrae images and multifeature descriptors are considered to generate more discriminative features, thus improving the vertebral detection accuracy. After that, the detected bounding box represents an appropriate initial contour of DRLSE to make the vertebral segmentation more accurate. It is helpful for the elimination of initialization sensitivity and quick convergence of vertebra boundaries. Meanwhile, vertebral centroids are extracted to connect the whole spine, thereby describing the spinal curvature. Different parts of the spine are determined as abnormal or normal in accordance with medical prior knowledge. The experimental results demonstrate that the proposed method cannot only effectively identify scoliosis with unlabeled spine CT images but also have superiority against other state-of-the-art methods.
Scoliosis is a common spinal abnormality, and it seriously endangers the people’s health [
Until recently, many methods have been developed for diagnosing the scoliotic deformity [
To intelligently diagnose scoliosis using machine learning, Glocker et al. [
To address this problem, this paper proposes a scoliosis recognition method in the unsupervised setting with unlabeled CT images. The cascade gentle AdaBoost (CGAdaBoost) classifier with multifeature descriptors and distance regularized level set evolution (DRLSE) model are combined into the centroids method. The main contributions of this paper are presented as follows. First, the relationship between individual vertebrae and the whole spine is established using vertebral centroids, which is beneficial to reduce the data-labeling burden on medical staff. Second, three different descriptors are fully combined to achieve more effective features for the CGAdaBoost classifier. Moreover, detected bounding boxes are used as an initial contour of DRLSE to segment vertebral bodies without manual interaction. Our work can provide a feasible and effective scoliosis recognition method for medical intelligence diagnosis.
The remainder of this paper is organized as follows. In Section
In general, the detection and segmentation of the object are important processes of the recognition task. To achieve a reliable detection result, learning-based technique has been extensively adopted to detect the vertebrae [
Gentle AdaBoost [
The weight of each sample is constantly adjusted to form the strongest classifier, thus improving the performance of the classifier and avoiding the overfitting. Hence, the gentle AdaBoost classifier with high efficiency is suitable for the vertebral detection.
Level set based on edge information [
The motivation of this paper is to accurately recognize scoliosis from unlabeled CT images. For convenience, the proposed method is called as CGAdaBoost-DRLSE. Figure
The overview of the CGAdaBoost-DRLSE method.
We make the first attempt to use the cascade gentle AdaBoost detector with multifeature fusion to detect vertebrae. The classifier with the cascade structure is essentially a degenerated decision tree, which arranges a series of strong AdaBoost classifiers from simple to complex [
Schematic of the cascade gentle AdaBoost detector.
In the training process, only positive samples of the previous classifier will be transmitted into the next classifier to continue learning. Then, some subwindows belonging to positive samples in each classifier are output as the detected vertebrae. On the contrary, subwindows belonging to negative samples will be rejected directly. Obviously, the cascade classifier can overcome the problem of the imbalanced sample and significantly improve the efficiency of the detector. The training process of the CGAdaBoost classifier is briefly described, as shown in Algorithm
++ ++ Calculate the detection rate Reduce the threshold of the Calculate detection rate
Currently, Haar-like [
Although each single descriptor is highly efficient, the extracted features are difficult to accurately distinguish vertebral and nonvertebral regions from spine CT images with low contrast. Therefore, to make full use of the advantage of each feature descriptor, we present a multifeature fusion way. HOG, LBP, and Haar-like features will be combined together to construct a feature vector, thus generating the optimal feature set. Before fusing, these features will be normalized for the facility of computation. The final feature set
Haar-like adopts the black-and-white feature template to perform sliding detection on the image. The integral graph is employed to realize the fast summation of subregions. And the sum of the pixel in the white region subtracts the sum of the pixel in the black region as feature value
For the LBP descriptor, the image is divided into several subregions. Then, the LBP feature of each pixel in the subregion is extracted. The statistical histogram of each subregion constitutes the texture feature vector of the whole image. A mathematical description of the LBP can be given as follows:
HOG descriptor also divides the image into several small connected regions. The gradient direction (or edge direction) histogram of the pixels in each region is calculated, thereby combining these histograms as a feature vector. The gradient vector of the HOG descriptor can be obtained as
Vertebrae segmentation is vitally important to extract centroids for scoliosis recognition. DRLSE [
To further reduce manual intervention, detected bounding boxes are viewed as the initial contour
In (
A preferable potential function can keep the sign distance function smoothing in the distance regularization term, which can be expressed as
As a consequent, the vertebral body region is obtained by evolving iteratively. It is noticed that the combination of detection and segmentation solves not only the sensitivity of initialization, but also quickly and accurately converges on vertebral boundaries.
To establish the relationship between the individual vertebrae and the whole spine, we extract vertebral centroids as a shared feature from the segmented result. This work highlights the importance of the vertebral centroid. Let
After extracting vertebral centroids, the least squares method [
The schematic diagram of the curvature angle.
According to the prior medical treatment, viewed from the coronal, the spinal curve looks like a straight line. Generally, if the curvature angle is greater than ten degrees, the spine will be diagnosed as scoliosis:
In addition, viewed from the sagittal, the curve of the spine is in “S” shape. There are three normal curvatures of spine, including cervical lordosis (35° to 45°), thoracic kyphosis (20° to 45°), and lumbar lordosis (40° to 60°) [
The measurement from the coronal view is focused on the diagnosis of scoliosis, while the measurement from the sagittal view refers to the diagnosis of lumbar lordosis, thoracic kyphosis, and cervical lordosis.
To verify the effectiveness and feasibility of the proposed method, a variety of experiments are conducted on about 500 spine CT images to automatically recognize scoliotic deformity. These images are from 20 subjects (11 males and 9 females; age range 18–56 years) of available spine CT volumes on the publicity platform
In the vertebral detection experiment, we only select the vertebral bodies as positive samples without considering the spinal cord, ribs, and sacrum. The positive and negative samples are created by the Training Image Labeler Toolkit of Matlab 2014a, resulting in 520 positive samples and 1058 negative samples. Figure
Parts of training samples: (a) positive samples; (b) negative samples.
The positive samples contain various parts (cervical, thoracic, and lumbar) of the whole spine. Additionally, vertebrae images with different views (sagittal and coronal), arbitrary contrasts, and lesions are also considered as positive samples. We select nonvertebral regions from CT images as negative samples. As far as possible to increase the distinguishability of interclass samples, diverse features will be provided for classifying.
For feature extraction, Figures
The result of the HOG descriptor: (a) CT vertebral image; (b) the visualization of HOG; (c) local amplification result.
Results of LBP and Haar-like descriptors: (a) the visualization of LBP; (b) Haar-like feature with edge features, center-surround features, and line features.
Figure
In the training process, the gentle AdaBoost classifier built a powerful classifier with high accuracy through several simple weak classifiers. Figure
The training process of the optimal classifier: (a) the weak classifier with false-positive rate 0.1; (b) the weak classifier with false-positive rate 0.08; (c) the strong classifier with false-positive rate 0.03.
The CGAdaBoost classifier captured the shape and pathological features of the vertebrae. At the same time, the vertebrae of sagittal and coronal views also are detected from CT spine images. We obtain the optimal parameter of the classifier after several experiments. To reduce the loss of the vertebrae, the true-positive rate should be set to a larger value. Likewise, the smaller the value of the false-positive rate is, the less the number of the falsely detected vertebrae is. As a result, TruePositiveRate (true-positive rate) is set to 0.9, and FalseAlarmRate (false-positive rate) is set to 0.03. The number of the training stage (NumCascadeStages) is set to 10 according to the total number of samples. In our implementation, the subwindow size is experimentally set to 90 × 80. Only in this way, the initial contour of the DRLSE method is closer to the edge of the vertebrae such that the final segmentation results are more accurate to serve for vertebral centroids extraction.
Various detection results with different feature descriptors on sagittal and coronal planes are illustrated in Figure
Detection results of the CGAdaBoost classifier with FalseAlarmRate 0.03 and TruePositiveRate 0.9: (a) the detection result with single feature on the coronal view (a red circle for the undetected vertebrae and a green circle for the false detected vertebrae); (b) the detection result with multifeature on the coronal view; (c) the detection result with multifeature on the high-contrast coronal view; (d) the detection result with multifeature on the sagittal view.
Another detected result: (a) original CT image; (b) the enhanced image by the CLAHE method; (c) the final detected result.
We applied the DRLSE method to segment the vertebrae from spine CT images without any user intervention. On the basis of vertebral detected results, located bounding boxes are regarded as the initial contour
The evolution result of DRLSE: (a) the initial level set function; (b) the final level set function.
By adjusting appropriate parameters, Figures
The segmentation result on the sagittal view: (a) the contour with 100 iterations; (b) the final contour with 200 iterations; (c) the final segmentation result.
The segmentation result on the coronal view: (a) the contour with 100 iterations; (b) the final contour with 200 iterations; (c) the final binary segmentation result.
The final goal of our method is to represent the detailed shape of the spinal curve using the centroid method. It should be pointed out that we would select the appropriate CT slice of the spine image in order to extract vertebral centroids. Figure
Comparison results of the extracted centroids (red solid point) with the ground-truth centroids (blue star): (a) the result without the vertebral pathology case; (b) the result with the severe vertebral fracture case.
Furthermore, we extract the lumbar vertebrae, cervical vertebrae, and thoracic vertebrae from the whole spine. The corresponding spinal curve is calculated by the least squares method. Figure
Curve fitting results of various parts in the whole spine: (a) sagittal lumbar curvature; (b)–(d) coronal lumbar curvature; (e) coronal thoracic curvature; (f) coronal cervical curvature.
After curve fitting, the angle between two tangents to the curve is determined as the spinal curvature angle. Table
A part of diagnosis results.
Binary mask | The curvature angle | The diagnosis result |
---|---|---|
Figure |
Sagittal lumbar 40.4° | Normal |
Figure |
Coronal lumbar 30.5° | Abnormal |
Figure |
Coronal lumbar 3.2° | Normal |
Figure |
Sagittal lumbar 44.6° | Normal |
Figure |
Coronal thoracic 55.2° | Abnormal |
Figure |
Coronal cervical 2.6° | Normal |
To evaluate the performance of the cascade gentle AdaBoost classifier, the receiver operator characteristic (ROC) is used as an evaluation criterion. ROC intuitively shows the compromise between true-positive rate and false-positive rate for the classification model. Figure
The ROC curve on different methods.
Additionally, to verify the credibility of the spinal curve fitting, using two evaluation criteria is to comprehensively assess the quality of curve fitting. One is the coefficient of determination (
RMSE of different methods on twenty subjects.
In Figure
Furthermore, the detection accuracy rate and centroid location error are employed to further assess the proposed method. From the total 231 vertebrae, CGAdaBoost-DRLSE with single feature detects 227 vertebrae, resulting in the detection accuracy rate of 98%. By contrast, the multifeature fusion successfully detects 229 vertebrae and has the detection accuracy rate of about 99%. The centroid location error is computed using the Euclidean distance between extracted centroids and ground-truth centroids. The single-feature method achieves an average centroid location error of 1.51 mm. The multifeature fusion method has the mean centroid location error of 0.87 mm. Two related methods reported by Korez et al. [
The performance comparison of our methods and other related works.
Methods | ||||
---|---|---|---|---|
Evaluation criteria | Supervised classification forests [ |
Interpolation theory + shape-constrained [ |
Single-feature + CGAdaBoost-DRLSE | Multifeature + CGAdaBoost-DRLSE |
Detection accuracy rate | 86% | 97% | 98% | 99% |
Centroid location error | 4.4 mm | 1.1 mm | 1.51 mm | 0.87 mm |
In Table
This paper proposes an unsupervised scoliosis recognition method with unlabeled CT images to improve the accuracy. The CGAdaBoost-DRLSE method consists of vertebral bodies’ detection, segmentation, and centroids extraction. Firstly, diversified training samples and multifeature descriptors are considered to achieve better detection results in the cascade gentle AdaBoost classifier. Then, located bounding boxes represent the initial contour of DRLSE to eliminate the sensitivity of initialization and quickly converge on vertebral boundaries. Finally, vertebral centroid extraction and curve fitting are performed to compute the spinal curvature angle, thereby recognizing scoliosis with the prior medical treatment. Experimental results have demonstrated that the proposed method can effectively and accurately diagnose scoliosis deformity and reduce the need for manual landmark. Besides, the proposed method also is suitable for clinical work with acceptable results and serves as a quick guideline for nonexperts. In future work, we will extend the proposed method to the three-dimensional case by introducing spatial information of CT spine volume and classify various vertebral fractures by the designed multiclass classifier.
The experimental datasets analysed during this study are available in the publicity platform SpineWeb, (
The authors declare that there are no conflicts of interest about this paper.
This work was supported by the Science and Technology Development Program of Jilin Province, China (nos. 20150307030GX, 2015Y059, and 20160204048GX), International Science and Technology Cooperation Program of China (Grant no. 2015DFA11180), Science Foundation for Young Scholars of Changchun University of Science and Technology (no.XQNJJ-2016-08), and National Key Research and Development Program of China (no.2017YFC0108303).