Sex Determination of 3D Skull Based on a Novel Unsupervised Learning Method

In law enforcement investigation cases, sex determination from skull morphology is one of the important steps in establishing the identity of an individual from unidentified human skeleton. To our knowledge, existing studies of sex determination of the skull mostly utilize supervised learning methods to analyze and classify data and can have limitations when applied to actual cases with the absence of category labels in the skull samples or a large difference in the number of male and female samples of the skull. This paper proposes a novel approach which is based on an unsupervised classification technique in performing sex determination of the skull of Han Chinese ethnic group. The 78 landmarks on the outer surface of 3D skull models from computed tomography scans are marked, and a skull dataset of a total of 40 interlandmark measurements is constructed. A stable and efficient unsupervised algorithm which we abbreviated as MKDSIF-FCM is proposed to address the classification problem for the skull dataset. The experimental results of the adult skull suggest that the proposed MKDSIF-FCM algorithm warrants fairly high sex determination accuracy for females and males, which is 98.0% and 93.02%, respectively, and is superior to all the classification methods we attempted. As a result of its fairly high accuracy, extremely good stability, and the advantage of unsupervised learning, the proposed method is potentially applicable for forensic investigations and archaeological studies.


Introduction
Sex analysis and determination are indispensable and foremost steps in confirming the personal identification of an individual in forensic investigations. The best result is achieved when confirming an individual sex by accessing the entire skeleton, but most of the time the skeleton is incomplete. Thus, various local skeletons such as the patella [1], hip joint [2], pelvis [3], calcaneus [4], carpal [5], and skull and its parts have been utilized for sex determination in different populations worldwide. Among all parts of the skeleton, the skull is a small and distinctive collection of bones. The skull is composed of hard tissue and can be well preserved in most cases. Hence, the skull and its parts are most widely and commonly used in providing information about human origin, ancestry, stature, and sex in forensic anthropological analysis [6].
Sex determination of the skull involves two major techniques: the first one is the measurement of skull traits, which reflects difference of skull morphology between males and females. The second one is the analysis and classification of skull measurements. Both will affect the classification accuracy in sex identification for the skull. The approach earlier used to measure skull traits is subjective visual method. Visual assessment depends heavily on the experience and knowledge of the forensic scientist or biological anthropologist. Thus, it is likely to be inaccurate when performed by an inexperienced observer due to its great subjectivity. To reduce subjectivity, efforts to physically quantify skull traits by using an ordinal scale or software are undertaken. With the development and success of medical imaging, skull traits measurement by means of images and computed tomography (CT) is established. For example, some studies used radiograph to provide morphological details of the skull, and some researchers utilized three-dimensional (3D) imaging of the skull from clinical scans of known individuals to discover metric variables. No matter what method is used to measure the morphological features of the skull, it is very important to employ a high-performance classification method. In existing studies, typically statistical and supervised classification methods are linear discriminant analysis (LDA), logistic regression, and support vector machines (SVM).
At present, many approaches in sex determination of the skull, which consist of skull measurement and data classification techniques, have been published and have achieved a high or higher accuracy of discrimination between the sexes. Walker obtained five cranial traits (glabella, mental, orbital, nuchal, and mastoid) by visual assessment and achieved the best classification results of 88% of the modern skulls with a negligible sex bias of 0.1% via the logistic regression model [7]. Robinson and Bidmos selected 230 skull samples from South Africa and extracted 12 measured skull characteristics and got 72.0-95.5 accuracy by establishing five discriminant function equations [8]. Ogawa et al. obtained anthropological measurements of 113 skulls of modern Japanese individuals from forensic anthropological test records. Ten skull measurements were used for statistical analysis, and nine discriminant functions were established. The classification accuracy is between 79% and 93% [9]. Franklin et al. used OsiriX 03 to mark 31 landmarks on 3D skulls of Australian individuals. They calculated a total of 18 linear interlandmark measurements, which were analyzed by discriminant function. The maximum classification accuracy was 90% [10]. Abdel Fatah et al. utilized 222 cranial CT images of White Americans to construct a statistical bone atlas. They obtained >95% accuracy (97.5% with 11 variables and 95.5% with 8 variables) by cross-validated linear discriminant analysis on metric variables [11]. Musilová et al. used coherent point drift-dense correspondence to analyze the entire cranial surface and used an SVM with a radial kernel to perform classification. The method provided a high level of classification accuracy (90.3%) in the sex determination of male and female skulls of Southern French population [12]. Li manually extracted the mid-sagittal frontal arc on dried skulls and adopted the Fourier transform to analyze the sex difference of adult skull in Northeast China. He obtained the results of 84.21% and 83.33% for male and female classification rates, respectively [13]. Li Ming et al. selected 67 skulls from Southwest China and measured 16 anthropometric characters. They established the equations of single-variable and multivariable analysis and obtained the highest accuracy of 89.2% for males and 90.0% for females [14]. Shui et al. chose 133 digital adult skull samples from Han ethnic group of North China and separately computed a total of 14 measurements (12 geometric measurements and 2 angle measurements). Then, they performed the Fisher step method to build the sex discriminant function and obtained the accuracy of 87.5% for male and 86.67% for female separately for the complete skull [15]. Luo et al. constructed a statistical shape model for 208 Chinese skulls by projecting the high-dimensional skull data into a low-dimensional shape space. Fisher discriminant analysis (FDA) was utilized to classify skulls in the shape space; the correct rates were 95.7% and 91.4% for females and males, respectively [16]. Liu et al. divided the skull into seven partitions and quantized immeasurable features by means of marking the feature points. Then, they used the forward stepwise regression method based on maximum likelihood estimation to select the optimal feature subset of each partition. Experiments showed that any three partitions are enough to determine the sex of incomplete skulls with a high accuracy [17].
Although existing methods fully demonstrate their usefulness in sex identification of the skull, a notable problem is that these methods are not applicable in cases in which category labels in the skull samples are absent. Another situation is that when the distribution of the male and female skull samples is not balanced, the effect of using supervised learning for classification may not be better than unsupervised learning. LDA, logic regression, SVM, and other supervised learning methods need to use a training set with category labels to train the classification model. It is therefore the aim of this study to propose a stable and efficient sex determination approach for the skull that is based on unsupervised robust classification technology.
The contribution of our work is as follows: In terms of sex determination of the skull, the current works are mainly focused on the methods of skull measurement, while the methods of data analysis and data classification are less explored, especially the unsupervised learning method. In this study, we attempt to improve the classification accuracy of sex determination of the skull from the perspective of data mining. Inspired by the clustering theories, we extend the fuzzy C-means clustering (FCM) method and put forward an improved algorithm that is used to classify the skull dataset we have measured. We named it as MKDSIF-FCM. The proposed MKDSIF-FCM is based on an unsupervised learning theory where input is presented without desired output. Compared with existing supervised learning methods, the proposed MKDSIF-FCM can divide the skulls into two categories without known category labels and obtain fairly high accuracy for 3D adult skull from the Han Chinese ethnic group.

Materials and Methods
Our process of sex determination of the skull consists of three broad phases outlined in Figure 1. In the first stage, our approach relies on acquiring skull data and building a database of skull models (Section 2.1). In the second stage, feature points from 3D skull models are marked by utilizing a semi-automatic method, and skull characteristics are extracted that are required to identify a skull (Section 2.2). In the last stage, the extracted characteristics are passed to the classifier. The proposed MKDSIF-FCM algorithm is undertaken to distinguish the skull's sex (Section 2.3).

Skull Data Acquisition.
This study is based on the specimens of 186 whole skulls from living adults representative of Han Chinese ethnic group, which were obtained by a clinical multislice CT scanner system at Xianyang Hospital located in Shaanxi province of China. The total database consisted of 100 female skulls with a mean age of 49.8 years (range: 18-75) and 86 male skulls with a mean age of 48.3 years (range: 18-76). Only intact, undamaged skulls were included in this study;   each skull contains all the bones from calvaria to jaw with the full mouth of teeth.

Skull Characteristics Measurement.
In this study, in order to adequately illustrate the anatomy of the skull, we use the skull calibration and measurement system (with independent research and development by our research group) to extract the characteristics of the 3D skull.
According to the research achievements of forensic anthropology experts, 78 landmarks on the outer surface of the skull are marked, 12 of them are located in the midline, and the rest are symmetrically located about the midline sagittal line on both sides ( Figure 2).
Distances and angles between different skull landmarks may be important components of skull sexual dimorphism. The size-related variables which reflect sex differences between male and female are obtained by calculating a total of 40 interlandmark measurements. Then, essential characteristic indexes for each skull were successfully constructed. Table 1 shows the characteristics and their brief descriptions; the data unit is mm. [19] is an unsupervised learning algorithm and a normal tool for data mining. Clustering is a process for grouping a set of data into classes so that the data within a cluster have high similarity but are very dissimilar to data in other clusters.

Method. FCM
To classify our skull measurements via the unsupervised learning method, we propose an improved FCM algorithm that puts forward the concept of distance weighting coefficient with influence factor (IF) and incorporates the advantage of multiple kernel learning. We named it as MKDSIF-FCM.

Distance Weighting Coefficient with IF. In the generic FCM algorithm,
∈ U is a membership function value from k th vector x k to i th cluster center v i . It reflects to what degree the same sample belongs to each cluster center. In (1a)-(1d), there is an example of distance weighting coefficient with IF. (1a) X is a set of two-dimensional samples. (1b) represents the initial cluster center in the FCM algorithm. (1c) represents the initial membership function value in the FCM algorithm. (1d) This paper puts forward a new concept of distance weighting coefficient with IF and provides a new approach of distance definition. Distance weighting coefficient is defined according to different contributions of sample to the same cluster center in data space. Distance weighting coefficient with IF is defined as follows: Let w ik be a fuzzy weighting coefficient from k th vector x k to i th cluster center v i . Moreover, w ik plays an important role in measuring the distance between k th vector x k and i th cluster center v i . For different types of sample set, the influence on distance d ik by w ik is different. In order to be able to ensure the stable clustering performance of our improved algorithm in regard to different datasets, we introduce an IF for w ik , denoted as .

Euclidean Distance Based on Distance Weighting
Coefficient with IF. In generic FCM, Euclidean distance is commonly used as distance . The notion of distance weighting coefficient with IF is introduced by the proposed MKDSIF-FCM algorithm, and the distance from ℎ vector x k to ℎ cluster center v i is defined in the form of square: Computational and Mathematical Methods in Medicine 5 We can prove that (5) obeys with distance definition in Euclidean space. We shall discuss the significance of (1/ ) . In (1a)-(1d), the Euclidean distances of three samples , , and to cluster center v 1 are 1, 2, and 5, respectively. Suppose the value of is 1. According to (2), (3), and (4), we can get 11 =7/15, 12 =6/15, and 13 =2/15. According to (5), we can obtain our defined distances of three samples , , and to cluster center V 1 : 11 ≈1.46, 12 ≈2.24, and 13 ≈13.69.
From calculating the results, we introduce distance weighting coefficient with IF to distance in Euclidean space, which is equivalent to the function of a zoom lens. It enlarges ( ≥0) or shortens ( <0) all distances, but an enlarged or shortened yardstick is different. For long distances, the enlarged or shortened yardstick is slightly bigger, and for short distances the enlarged or shortened yardstick is slightly smaller. It leads to polarization, in which long distances become much longer, and short distances become much shorter. Thus, an appropriate assignment of distance weighting coefficient with IF can improve the performance of FCM.

Multiple Kernel Learning.
In general, the reliability of the traditional clustering algorithms strictly depends on the feature difference of data. If the feature differences are large, it is easy to implement clustering. However, if the feature differences are small and even some features are crossed in the original space, it is difficult for traditional algorithms to cluster correctly. By using the traditional clustering methods and kernel technique, Wu et al. constructed the kernel clustering algorithm [20]. Kernel-based fuzzy clustering can map the data in the original space to a high-dimensional feature space in which it can produce a remarkable improvement over standard FCM. Then, Sonnenburg et al. put forward the concept of multicore learning [21].
The proposed MKDSIF-FCM algorithm incorporates the advantage of multiple kernel learning. Usually, multiple kernel methods consist of polynomial kernel, Gaussian kernel, and hyperbolic tangent kernel. According to different properties of samples, we can choose different parameters of different kernel functions to extend applicability of single kernel function, and we can choose different kernel functions to make the global kernel function and local complementary kernel function, further improving the categorization of different samples. Ultimately, good clustering effect is achieved, and generalization performance of the kernel is improved.
The form of Gaussian kernel function is as follows: where n is the center of kernel function and is the width parameter and controls the radial range of the function. The form of polynomial kernel function is as follows: ( , ) = ( ⋅ + ) , ≥ 0, ∈ The form of hyperbolic tangent kernel function is as follows: Any function which satisfies the mercer condition [22] can be regarded as a kind of kernel function. The combination of k kernel functions according to different weight coefficients is still a kernel function, denoted as the following: * ( , ) = ∑ =1 ( , ) ≥ 0, = 1, 2, ⋅ ⋅ ⋅ , Under the constraint, By constraining to the Euclidean distance, the squared distance is computed in the kernel space using multiple kernel functions such that If we select the Gaussian kernel which is used almost exclusively in the literature, then ( , ) = 1 and In this way, the objective function J S will become the following: where Φ(.) is the nonlinear map kernel function and Φ(x k ) and Φ(v i ) express sample x k and clustering center v i in feature space, respectively. Minimizing (13), we then can obtain the update expressions of membership function u ik and center of cluster v i as follows: ∈ × is an c×n matrix of fuzzy partition for given training data x k ={ 1 , 2 , . . . , } (k=1,2,. . .,n), where ∈ is a membership function value from ℎ vector x k to ℎ cluster center v i and satisfies the following conditions: 6

Computational and Mathematical Methods in Medicine
The MKDSIF-FCM algorithm aims to determine cluster centers v i (i=1, 2, . . ., c) and the fuzzy partition matrix U by minimizing the objective function J S defined as follows: where parameter s(1<s<∞) influences the fuzziness of the clusters. Large s will increase the fuzziness of the function. For most data, 1.5 ≤s ≤3.0 gives good results. The value of s is often set to 2. Moreover, is the Euclidean distance of the kernel space from sample x k to cluster center v i defined as (11).
The MKDSIF-FCM algorithm uses iterative optimization to approximate minima of an objective function J S . In minimizing J S , the basic step of MKDSIF-FCM algorithm is performed in the following procedures.
Step 1. Given a value of parameters c and commonly in the literature, we let s=2.
Step 2. Initialize the matrix U of fuzzy partition by generating c×n random numbers in the interval [0, 1].
Step 5. U and V are updated by minimizing objective function J S . We can derive the calculating formula of and v i as (14) and (15), respectively.
Step 6. Compute the objective function J S by using (19); stop the MKDSIF-FCM process if the following condition holds: where it converges or the difference between two adjacent computed values of objective functions J S is less than the given threshold . Otherwise, go to Step 4.
The input of MKDSIF-FCM algorithm is a set of samples X={ 1 , 2 , . . . , }, and the number of clusters is required to be predefined. Further, two parameters (s and ) need to be given in advance. The output of MKDSIF-FCM algorithm are the cluster centers v i (i=1, 2, . . ., c) and the fuzzy partition matrix U.

Results
We use a 3.40 GHZ Core(TM) I7-3770 CPU 4GB RAM desktop computer and MATLAB 2015a software in conducting all experiments. For all algorithms presented in this paper, the experiments were repeated 50 times, and the average results were obtained for comparison. In MKDSIF-FCM algorithm, there is a parameter group = { , 1, 2, 1, 2, }, where s represents the fuzziness index, p 1 and p 2 represent the probability, 1 and 2 represent the parameters of the Gaussian kernel function, and represents the IF.
For all supervised classification methods presented in this paper, the skull dataset is split into a training set and testing set; 60 samples were randomly picked as the testing set and the numbers of positive and negative examples are kept the same in each sampling.

The Results of Sex Determination for 3D Skulls.
The metrics used for evaluating the performance of the algorithm on the skull dataset are described below: ACC: it is the number of skulls that are correctly classified as male or female skulls. TPR: it is the proportion of the male skulls that are correctly identified. TNR: it is the proportion of the female skulls that are correctly identified.
T: it represents running time.
From Table 2, it can be seen that when selecting a group of suitable parameter values (s=2, =0.5, p 1 =0.9, p 2 =0.1, 1 =30, and 2 =110), the MKDSIF-FCM algorithm can obtain the best classification accuracy of sex determination of the skull. For 186 skulls of the Han Chinese ethnic group, we obtain the accuracy of 95.70% compared to 87.09%, 92.2%, and 93.55% found in the literature [15][16][17], respectively. There is a classification accuracy of 93.02% for males and 98% for females, respectively. Table 2 that the accuracy had a significant and sharp improvement of nearly 34% for the MKDSIF-FCM algorithm over the original FCM algorithm for the skull dataset. The running time for MKDSIF-FCM is greater than that for FCM, because the number of iterations to convergence is greater.

Comparison with Other Unsupervised Methods. It is clear from
It is also clear from Table 3 that the MKDSIF-FCM algorithm achieved better classification performance on the Iris dataset. There is an improvement of nearly 6% for MKDSIF-FCM over the original FCM algorithm with detecting a group    of suitable parameters. The proposed algorithm appears to have the quite similar time complexity and iterations as the original FCM algorithm. As shown in Figure 3, it is easily observed that the accuracy of MKDSIF-FCM algorithm is higher than that of SAWFCM [23], SWFCM [24], MF-FCM [25], FW-FCM [26], FKCM [27], KFCM [28], FKWCM [29], DWFCM [30], multiple kernel FCM [31], and IWFCM [32]. The accuracy of the MKDSIF-FCM algorithm is quite similar to that of POKFCM [33]. Table 4  It is obvious that the proposed MKDSIF-FCM algorithm obtained not only the highest classification accuracy of 95.7% but also the highest TPR and TNR of 93.02% and 98%, respectively. Both FDA (with the best feature) and SVM did a good job with higher accuracy. The classification accuracies of other methods are no more than 90%. The results reveal several similarities between TPR and TNR. And we can observe that the correct classification rate of females is uniformly higher than that of males.

Stability Analysis of the MKDSIF-FCM Algorithm.
The experimental procedure is repeated 50 times for each classification method; the maximum, minimum, and mean of the accuracy are represented via error-bar plots (Figure 4). The proposed MKDSIF-FCM algorithm presents an extremely stable performance on the skull dataset, and the classification accuracy of other methods fluctuates greatly. The difference between maximum and minimum accuracy ranged from 37% using BP neural network to 17% for SVM.

Discussion
FCM [19] is one of the best-known unsupervised algorithms. However, its performance has been limited to Euclidean distance. In recent years, various kinds of improved FCM algorithms have been reported [23][24][25][26][27][28][29][30][31][32][33]. This paper proposes an improved FCM algorithm to determine the sex of adult skulls from the Han Chinese ethnic group. In order to verify the effectiveness and generality of the proposed algorithm, we performed a comparative analysis among the original FCM, some improved FCM algorithms, and the proposed MKDSIF-FCM algorithm.
The MKDSIF-FCM algorithm achieved better classification performance on both publicly available Iris dataset and skull dataset. Especially in the skull database, the accuracy has been greatly improved. On the Iris dataset, our MKDSIF-FCM algorithm has little change in time complexity and iterations compared with FCM. On the skull dataset, the number of iterations of the MKDSIF-FCM algorithm is much larger than that of FCM. This finding implies that the proposed algorithm can tend to become very computationally demanding when the data has high dimensionality and large volume. Experimental results on the Iris datasets show that, for accuracy, our algorithm is almost better than all algorithms in the literature [23][24][25][26][27][28][29][30][31][32][33].
Our innovative algorithm introduces distance-weights with IF into the commonly used Euclidean distance and increases the difference degree of category between samples. The proposed algorithm incorporates the idea of multiple kernel learning that maps the data into a higher-dimensional space in which the nonlinearity fades away and the data become linearly separable. It is the reason that the proposed   MKDSIF-FCM algorithm can improve the performance of clustering. So far, to our knowledge, supervised learning remains the most widely employed method in sex determination of a skull. In particular, logistic regression and discriminant function analysis are the two most representative statistical learning methods. According to the method used in literature [17], we established the best model using logistic regression and stepwise variable selection. When selecting nine variables (I8, I11, I14, I16, I20, I29, I31, I38, I40), the model obtains 84.93% and 92.53% classification rates for males and females, respectively. In the same way, we select the best feature subset from skull measurements to establish the FDA model. With ten variables (I8, I11, I14, I16, I20,  I23, I29, I31, I38, I40), the classification rates for males and females are 90.93% and 94.80%, respectively. In order to choose the most suitable classifier for the skull dataset, we also compared the results using other popular supervised classification methods, including decision tree, SVM, BP neural networks, and H-ELM [18]. In all the methods we attempted, the proposed MKDSIF-FCM algorithm gives the best classification performance for both male and female skulls.
When classifying the skull dataset, we hope that the results can be reproduced. Thus, it is very important that the classification algorithm is stable. In the 50 repeated experiments, our algorithm obtained the same result. It is obvious that the proposed MKDSIF-FCM algorithm presents extremely stable performance on the skull dataset.
In conclusion, by means of its fairly high accuracy, extremely good stability, and the advantage of unsupervised learning, we have the reason to believe that the MKDSIF-FCM algorithm is the most suitable classifier for our skull dataset. Of course, our experimental results also indicate that skull characteristics we extracted were very accurate and effective in sex determination of the skull.

Conclusions
In this paper, we propose a novel approach to sex determination of skulls of the Han Chinese ethnic group. The first step in our method is extraction of morphological features from the 3D skull. In the second step, the MKDSIF-FCM algorithm is employed to conduct sex determination of the skull of the Han Chinese ethnic group. A comparison with other popular classifiers, such as decision tree, BP neural Computational and Mathematical Methods in Medicine 9 network, logistic regression, FDA, SVM, and H-ELM [18], showed that our proposed MKDSIF-FCM algorithm worked better. The experimental results suggest that the use of the proposed MKDSIF-FCM algorithm in the classification of the skull dataset is an accurate, robust, and reproducible technique. For the Han Chinese ethnic group, there is an accuracy improvement of nearly 8.6%, 3.5%, and 2.2% for our sex determination approach over other methods in the literature [15][16][17].
It is worth noting that the proposed method achieves a better and stable performance for skull sex determination while maintaining its advantages of unsupervised learning. We believe that the methods described here are noteworthy, particularly for researchers who are attempting (or are considering attempting) to engage in skull sex determination by means of unsupervised learning methods.

Data Availability
The data used to support the findings of this study are available from the corresponding author upon request.

Conflicts of Interest
The authors declare that they have no conflicts of interest regarding the publication of this paper.