A Multiscale Constraints Method Localization of 3D Facial Feature Points

It is an important task to locate facial feature points due to the widespread application of 3D human face models in medical fields. In this paper, we propose a 3D facial feature point localization method that combines the relative angle histograms with multiscale constraints. Firstly, the relative angle histogram of each vertex in a 3D point distribution model is calculated; then the cluster set of the facial feature points is determined using the cluster algorithm. Finally, the feature points are located precisely according to multiscale integral features. The experimental results show that the feature point localization accuracy of this algorithm is better than that of the localization method using the relative angle histograms.


Introduction
With the development of 3D information acquisition technology, the research of 3D facial feature has gained more and more extensive attention. The automatic localization of the facial feature points is a study hotspot in the field of medical computer vision, which is a precondition for face recognition, face animation, face tracking, and 3D face reconstruction. At present, the 3D facial feature point localization algorithms have not been researched in depth though the 2D facial feature point localization algorithms have matured. Wang et al. [1] applied the Jet Bunch algorithm of 2D facial feature point localization to the process of 3D facial feature point localization; Xu et al. [2] proposed a hierarchical filtering algorithm, which could locate the feature points of nasal tip. While these methods are beneficial to detect other facial key feature points, they are insensitive to noises and have the rotation and translation invariance characters with a poor generality, and the localization results are unsatisfactory except the nasal tip feature points. Feng et al. [3] proposed a feature point localization algorithm based on relative angle histograms. In a 3D point distribution model, the relative angle histograms of all points are calculated firstly, and then the features are located according to the similarity of the histograms. The algorithm is of high efficiency and stability, but it locates the feature points just in a small scope with an inaccurate result. Li and Da [4] suggested a 3D facial feature point localization algorithm that combined a priori knowledge with differential characteristics, but the result is not satisfactory because of the unique and complex diversity of the face model.
In order to solve these problems, we put forward a 3D facial feature point localization method based on the relative angle histograms and multiscale constraints. The cluster point set of the facial features is created firstly, and then the multiscale integral characteristics are used to locate the feature points accurately. As a result, the accuracy of the feature point localization is improved.

Coordinate System
Transformation. CT (computed tomography) images are acquired from living samples in a hospital, whose contour lines are extracted using the method of combining the improved snake algorithm with the ray method [5]. In order to obtain the 3D face sample that is constructed by a single layer triangular mesh, the 3D surface is reconstructed using the Ganapathy algorithm to connect the neighbor contour lines [6]. When the CT data are collected in a hospital, the postures of every body's head are different, and unavoidably the reconstructed single layer 3D models would lie in different coordinate systems. For the convenience of defining the feature points, all the models are unified to a uniform coordinate system.
A face model (FM) is usually constructed by a triangular mesh which is composed of vertexes ( ) and triangular facets ( ), so FM could be expressed as a linear representation, FM = ( , ). V is the th point of the FM, 1 ≤ ≤ , and the coordinate of V is expressed by V = ( , , ) . The center of the FM is computed by = (1/ ) ∑ =1 V , and then the point set is transformed into the coordinate system whose origin is .
The point set is taken as a 3D random variable, and a positive definite covariance matrix could be obtained from the transformed . Three biggest eigenvalues (1) (1 ≤ ≤ 3) is taken as the th principal component of variable , and then we construct the coordinate system shown in Figure 1, whose origin is and coordinate axes are 1 , 2 , 3 .

Feature Point Definition.
The feature points of a 3D face must have connotations and lie in a cognizable key position. In the same sample the feature points are distinct from their neighbor points on geometrical characteristics, while in different samples the features of the same feature points are similar. To meet the experimental requirements, a total of 39 feature points covering the front half face are defined according to the knowledge of anthropology, anatomy, and the feature points defined by MPEG4 expert group [7,8]. The positions of the feature points are shown in Figure 2.

Relative Angle Histograms.
After the face model is unified into a uniform coordinate system, the next step is to calculate the relative angle histograms of points in the face model. According to Feng et al. [3], any point V of the face model expresses the angle between vector → V V (1 ≤ ≤ , ̸ = ) and 1 , and after the angle is transformed its value range is The relative angles of the point in the face model describe a spatial relationship between every point, which possesses rotation, translation, and scaling invariance and better noise   Figure 3 shows the feature points' relative angle distributions of the forehead center and right mouth corner as well as those which are superimposed by the different samples. It is obvious that the different feature points have different distributions, while the same feature points in the different samples have similar distributions. The similar feature points are selected to be clustered into one group using the cluster method [9] and to set up the cluster set of the feature points. Step 1. According to the prior knowledge, the representative points are selected as the initial cluster center CC of each group.
Step 2. Calculate the similarity between CC and points in the model and sort them in decreasing order.
Step 3. Select the maximum similarity points as the cluster set of the th feature point in the model . Figure 4 shows the cluster point set results of the nasal tip and the forehead center in a model whose feature points are to be located. Experiments prove that the fewer cluster points would contain the optimum feature points in the conspicuous feature point place of a face, such as the nasal tip and the mouth corner, while, in the smoother place, such as the forehead center, more cluster points would be needed. The next task is to select precise feature points from the cluster point set.

Volume Integral Features.
The volume integral features are defined as multiscale feature values, which could weaken the noise influence with bigger robustness than the differential values, and measure the concave-convex level of the model surface.
The volume integral invariant of a surface point is defined as a functional integration in the local sphere region of this point [10][11][12], whose math expression is where ( ) is a sphere, which takes as the sphere center and as the radius. As is shown in Figure 5. ( ) is a surface indicator function. When the point is in the region, = 1; while is out of the region, = 0. The geometrical significance of ( ) is a part of the sphere volume which locates in the lateral of the surface.
Equation (2) is expanded to a polynomial and its limitation value is computed under the condition → 0, so the functional relationship between the volume integral invariant and mean curvature is From (3) we know that the value of the volume invariant is related to the radium of integral sphere region, so the longer the radius is, the bigger the value of volume invariant becomes. Whether there was noise in the interior of the integral region, the impact on the result of the volume invariant is small, so the volume invariant is taken as a feature which possesses robustness. In practice, the geometric feature of the surface on the point is defined as char ( ), which is the ratio of the volume invariant to the volume of sphere region. The equation is According to the calculated values of char ( ), we describe the geometric shape of the surface on the point as convex, approximate plane, and concave. When the scale is and char ( ) < 1/2, the shape of the surface on the point is convex; when char ( ) ≈ 1/2, the shape of the surface on the point is an approximate plane; when char ( ) > 1/2, the shape of the surface on the point is concave.

Multiscale Feature Extraction.
Concave-convex is a relative concept. Concave-convex level will transform with the scale change. As is shown in Figure 6, under a large scale the curve in a box is convex, while under a small scale it will be smooth when the local part is amplified. Multiscale geometrical feature extraction [13] means that the feature extraction algorithm extracts the same feature adopting multiple threshold values. As is shown in Figure 7, the surface geometrical features of the point on the surface are extracted under 3 scales: 1 , 2 , and 3 . The large scale can reduce the noise impact more, while the small scale can more precisely evaluate the integral invariant. The large scale should be combined with the small scale in the feature extraction process. Therefore, adopting the multiscale feature extraction method can not only precisely describe the local features but also reduce the noise influence. in the triangular mesh. The distance square root is defined on the integral features. As is shown in (5), every point in is calculated, as well as the of the standard feature points under the three scales. The point , the minimum value of , is chosen as a feature point of the model . Consider

Experimental Comparison and Analysis
We design a feature point of 3D face localization system (FaceLS) according to the proposed feature point localization algorithm. Forty sets of 3D point distribution face model data are selected randomly from a monolayer face sample library, which belongs to the North West University, and are taken as the face samples of the feature points which will be located. The algorithm is tested and compared with the localization method of Feng et al. [3]. All the programs are operated by Windows 7, Core i5 processors, 2.8 GHz, 2 G RAM, and Matlab 2010. The evaluation method of the feature point localization is as follows: given a distance threshold value , ( , , ) is the accurate position of each feature point in the untested models, and the Euclidean distance between two points can be computed as follows: The steps of our method are as follows.
Step 1. According to the prior knowledge, the representative points are selected as the initial cluster center CC of each group.
Step 2. Calculate the similarity between CC and points in model and sort them in decreasing order. Step 3. Select maximum similarity points as cluster set of the th feature point in model .
Step 4. We calculate the integral features of cluster points and point for three different scales of 1, 2, and 3 times of the average side length in the triangular mesh. The distance square root is defined on the integral features. As is shown in (5), every point in is calculated, as well as of the standard feature points under the three scales. The point , the minimum value of , is chosen as a feature point of the model .
Step 5. Given a distance threshold value , judge whether the point is correct with (6). If ≤ , the feature point localization is considered to be correct; otherwise it has a certain error. Figure 8 shows the results of two methods: one is based on our method and the other is the relative angle histogram localization method of Feng et al. [3] for the same feature point pending model (only 28 feature points are showed, and part of the feature points are coincident from the front view). Using the two localization methods, Figure 9 shows the accuracy rate of 39 feature points in a range of the given distance threshold value. The red dotted line indicates the relative angle histogram localization method of Feng et al. [3], and the blue solid line presents our method. The localization result of our method is more precise. Table 1 shows the mean accuracy rates of our method are higher than those of Feng et al. [3]. The feature point cluster set is determined firstly by the cluster algorithm and the relative angle histogram algorithm, and then the feature points are located accurately by the steady multiscale integral features, so the accuracy rate of localization is improved greatly. The localization method based on relative angle histograms has limitations, because it locates feature points to a smaller range through comparing the similarity of relative angle histograms and this method is suitable for conspicuous feature points, while the accuracy will be decreased for the inconspicuous feature points.

Conclusions
In this work we propose a method that combines the relative angle histograms with the multiscale constraints for localization of the feature points in a 3D face. The feature point cluster set is determined through the cluster algorithm and the relative angle histograms, and then feature points are located accurately by the multiscale integral features, which could avoid many wrong matches in the result caused by the precision and similarity of the local geometrical characteristic. Experimental results show that this method performs well and the accuracy rate of localization is improved. But the accuracy rate of localization is nonideal under the condition of feature points on smooth part of a face, so this should be the next research direction.