Significant Geometry Features in Tongue Image Analysis

The shape of a human tongue and its relation to a patients' state, either healthy or diseased (and if diseased which disease), is quantitatively analyzed using geometry features by means of computerized methods in this paper. Thirteen geometry features based on measurements, distances, areas, and their ratios are extracted from tongue images captured by a specially designed device with color correction. Using the features, 5 tongue shapes (rectangle, acute and obtuse triangles, square, and circle) are defined based on traditional Chinese medicine (TCM). Classification of the shapes is subsequently carried out with a decision tree. A large dataset consisting of 672 images comprising of 130 healthy and 542 disease examples (labeled according to Western medical practices) are tested. Experimental results show that the extracted geometry features are effective at tongue shape classification (coarse level). Even if more than one disease class belongs to the same shape, the disease classes can still be discriminated via fine level classification using a combination of the geometry features, with an average accuracy of 76.24% for all shapes.


Introduction
The human tongue contains numerous features. Traditionally, medical practitioners would examine these features based on years of experience [1][2][3][4][5]. However, ambiguity and subjectivity are associated with their diagnostic results. To eliminate these qualitative aspects, tongue images can be objectively analyzed, which offers a new way to diagnose disease, one that minimizes the physical harm inflicted to patients (compared with other medical examinations).
In state-of-the-art computerized tongue image analysis, color and texture features are the most prevalent [6][7][8][9][10][11][12][13][14][15][16][17][18][19]. There exists little or no literature on tongue image analysis using geometry features, whereas in traditional medicines such as traditional Chinese medicine (TCM) the shape of a tongue can be used to determine a patients' illness [3,4,20]. The authors in [21] proposed an approach to automatically recognize tongue shapes based on geometry features. The seven geometric features included various measurements of length, area, and angle extracted from tongue images. Using a support decision tool to weight the relative influences of the geometry features, they classified an image into one of six tongue shapes, namely, hammer, rectangle, acute triangle, obtuse triangle, square, and round (based on TCM). Experimental results conducted on 362 tongue images exhibited an accuracy of 90.3% for shape classification. Nevertheless, there was little quantitative analysis between tongue shape and the relationship to its current health state.
In this paper we thoroughly examine the aforementioned problem via geometry features in tongue image analysis. The imaging device used to capture tongue images is made up of a 3-chip CCD camera with 8 bit resolution and two D65 fluorescent tubes placed symmetrically around the camera in order to produce a uniform illumination. The images captured were color corrected [22] to eliminate any noise caused by variations of illumination and device dependency. Also, the tongue image capture device ensures that the images are properly aligned. This allows consistent feature extraction and classification in the following steps. Figure 1 shows the capture device. Using this device we form a large tongue image database consisting of 672 samples. This database is composed of 130 healthy and 542 disease samples, divided into 7 classes with at least 19 examples. Every image is segmented [19] with the background removed and tongue  foreground remaining. From each tongue image consisting of a tip, body, and root [20], 13 geometry features derived from measurements, distances, areas, and their ratios are extracted. Using these features we define 5 tongue shapes based on TCM, rectangle, acute and obtuse triangles, square, and circle. Coarse level classification applying a decision tree [23] was used to classify a tongue into one of the shapes. Experimental results showed that a majority of the samples in the classes from healthy and disease samples tend to be one form, where healthy versus disease samples and disease versus disease are separable using fine level classification, employing a combination of geometry features. This proves the significance of the geometry features at establishing a relationship between a tongue's state and its shape.
The rest of this paper is organized as follows. Thirteen geometry features extracted from tongue images are presented in Section 2 along with the tongue shape classification performed using a decision tree. Following this, experimental results using coarse and fine level classifications are given in Section 3. Finally, concluding remarks are made in Section 4.

Materials and Methods
The tongue image dataset is first introduced in Section 2.1. Afterwards, a detailed description of the 13 geometry features extracted from a tongue image is given in Section 2.2. How these features are then used to classify a tongue into 5 major shapes is provided in Section 2.3.

Tongue Image Dataset.
The tongue image database is composed of 672 images (one image per person) divided into 130 healthy and 542 disease samples. Seven disease classes and healthy classes were captured at Guangdong Provincial Hospital of Traditional Chinese Medicine, Guangdong, China. Patients with diabetes mellitus were processed at the Hong Kong Foundation for Research and Development in Diabetes, Prince of Wales Hospital, Hong Kong. Healthy samples were verified through a blood test and other experiments. If indicators from the tests fall within a certain range they were deemed healthy. In the disease class, samples were collected from inpatients with illness determined by their admission note and diagnosed using western medical practices. Inpatients suffering from the same disease were grouped together into a single class. In total there were 7 disease groups (with at  least 19 samples). A summary of the disease class breakdown is given in Table 1.

Geometry Features.
In the following subsection we describe the 13 geometry features extracted from tongue images (which have been converted to binary images after segmentation [19]). These features based on measurements, distances, areas, and their ratios are used in subsequent sections to define and classify 5 tongue shapes.

Width.
The width ( ) feature (see Figure 2) is measured as the horizontal distance along the -axis from a tongue's furthest right edge point ( max ) to its furthest left edge point ( min ):

2.2.2.
Length. The length ( ) feature (see Figure 2) is measured as the vertical distance along the -axis from a tongue's furthest bottom edge ( max ) point to its furthest top edge point ( min ): Evidence-Based Complementary and Alternative Medicine

Length-Width
Ratio. The length-width ratio ( ) is the ratio of a tongue's length to its width:

Smaller Half Distance.
Smaller half distance ( ) is the half distance of or depending on which segment is shorter (see Figure 2):

Center
Distance. The center distance (cd) (refer to Figure 3) is the distance from 's -axis center point to the center point of ( cp ): where cp = ( max + min )/2.

Center Distance Ratio.
Center distance ratio (cdr) is ratio of cd to : 2.2.7. Area. The area ( ) of a tongue is defined as the number of tongue foreground pixels.

Circle Area.
Circle area (ca) is the area of a circle within the tongue foreground using smaller half distance , where = (refer to Figure 4): 2.2.9. Circle Area Ratio. Circle area ratio (car) is the ratio of ca to :

Square Area.
Square area (sa) is the area of a square defined within the tongue foreground using smaller half distance (refer to Figure 5): 2.2.11. Square Area Ratio. Square area ratio (sar) is the ratio of sa to : 2.2.12. Triangle Area. Triangle area (ta) is the area of a triangle defined within the tongue foreground (see Figure 6). The right point of the triangle is max , the left point is min , and the bottom is max .

Triangle Area Ratio.
Triangle area ratio (tar) is the ratio of ta to : 4

Evidence-Based Complementary and Alternative Medicine
Triangle area (ta) x min x max y max Figure 6: Illustration of feature 12.

Tongue Shape Classification.
Based on TCM we define 5 tongue shapes, rectangle, acute triangle, obtuse triangle, square, and circle, which can be classified using the 13 features explained above (see Section 2.2). A rectangle tongue's vertical length is long, but its horizontal width along the tip, body, and root remains relatively constant. An acute triangle tongue's vertical length is longer than its largest horizontal width (at the root) but gradually decreases from the body down to the tip. If the tongue shape is an obtuse triangle, its horizontal width is greater than its vertical length, with the width steadily decreasing as it approaches the tip. In a square tongue shape both its horizontal width and vertical length are similar. Finally, if a tongue is circle, both the horizontal width and vertical length will be alike, but its car (8) will be closer to 1. Figure 7 depicts the 5 tongue shapes using typical samples. To classify tongue images into its proper shape, a decision tree structure shown in Figure 8 is used. Given a tongue we first examine its length-width ( ) ratio. If this ratio is low ≤ ≤ high , the tongue shape must be square or circle (left branch), and if the ratio is high < or low > , the shape of the tongue can be rectangle, acute triangle, or obtuse triangle (right branch). The values of low and high are 0.95 and 1.05, respectively. Focusing on the left branch, the average radius ( avg ) of the tongue is first calculated as avg = ( + ) 4 (12) which is the average of /2 and /2. Next, the ratio sc is computed as If the tongue shape is approximately square, the value of sc ≈ 4 (i.e., 4 ⋅ avg 2 / avg 2 ), and if it is approximately circle, sc ≈ (i.e., ⋅ avg 2 / avg 2 ). Hence, the two shapes can be defined as Square = sc ≥ + , where is a constant equal to 0.1. Turning our attention to the right branch, we initially calculate the ratio rao : If this ratio is greater than or equal to rect , the tongue shape is rectangle: where rect is 0.85 and the maximum of rao is 1. If rao < rect , the shape of the tongue is either acute or obtuse triangle.
To determine which triangle, length-width ratio is used once again as follows: where ao is given as 1.05. The parameter values listed above to classify a tongue image were chosen empirically.

Results and Discussions
The following section presents the experimental results. A coarse level classification showing the results of tongue shape classification is given in Section 3.1. Classes classified into the same shape are further differentiated in Section 3.2 through fine level classification using a combination of geometry features.

Coarse Level: Tongue Shape Classification Result.
By applying the tongue shape classification algorithm (described above, see Section 2.3) to every image in the dataset, its shape can be determined. This result is listed in

Rectangle
Acute triangle Obtuse triangle

Square Circle
Tongue image t lw low ≤ lw ≤ t lw t lw < lw or t lw low > lw     Evidence-Based Complementary and Alternative Medicine 7 than one class for each shape (see Table 2). To distinguish between classes with the same shape, a set of geometry features were selected and applied to SVM. Half the images in each class were randomly selected for training, while the other half was used as testing. To measure the performance, average accuracy was employed. The linear kernel function (dot product) was used to map the training data into kernel space, while a quadratic kernel produced similar classification results. -NN was also tested but did not perform as well.
Utilizing a grouping of the features is logical, since not every feature can have a positive contribution to the final result, as is the case here where 13 features used for classification produced poor results. Therefore, an optimization of the features is necessary. To reduce the number of features, sequential forward selection (SFS) [23] was implemented. SFS is a feature selection method that begins with an empty set of features. It adds additional features based on maximizing some criterion function and terminates when all features have been added. In our case is the average accuracy of SVM. Below, each tongue shape is examined in detail.

Rectangle.
Both GV and NS were classified into this shape. Applying SFS with SVM to separate the two classes, the highest average accuracy of 70.07% was achieved using features 9, 4, 7, 8, 2, 13, 12, and 1.

Acute
Triangle. For this shape NR and CG were classified together. (feature 2) attained the best average accuracy of 70.00%.

Obtuse
Triangle. DM and CHD were assigned to obtuse triangle. Through a combination of features consisting of 1, 7, 4, 11, 13, and 2, the highest average accuracy of 76.23% was achieved.

Circle or
Square. The maximum average accuracy of 88.65% was obtained using (feature 1) and (feature 7) to classify the two classes (healthy and EG) appointed to circle or square. Table 3 summarizes the results for each tongue shape. For completeness, the average accuracy of healthy samples versus NR, GV, NS, CG, DM, and CHD is shown in Table  4. From this result, it can be seen that healthy samples are distinguishable compared to others, given their different tongue shapes.

Conclusions
In this paper we thoroughly examined tongue shape and its relation to a patients' state (either healthy or diseased) using geometry features through computerized methods. With tongue images captured by a specially designed device that accounts for image correction and a large dataset labeled according to western medical practices, we have a solid foundation to carry out this objective study. Thirteen geometry features including measurements, distances, areas, and their ratios were extracted from each tongue image. The features helped define 5 tongue shapes rooted upon TCM and classified using a decision tree. In the experimental results, coarse level classification first showed that the tongue classes belong to different shapes. Although more than one class occupies the same shape, in fine level classification they are still distinguishable, when employing SFS with SVM (using a grouping of geometry features). This validates the significance of geometry features at shape classification, as well as healthy versus disease/disease versus disease classifications. With tongue shape and a persons' health state now established using computer-based methods, this potentially provides a new painless and efficient way to diagnose patients. A continuation of this work will investigate the fusion of all possible tongue features including color and texture in order to better determine a patients' state.