3 D Face Recognition based on Local Conformal Parameterization and Iso-Geodesic Stripes Analysis

1College of Information Science and Technology, Beijing Normal University, Beijing, China 2Engineering Research Center of Virtual Reality and Applications, Ministry of Education, Beijing Key Laboratory of Digital Preservation and Virtual Reality for Cultural Heritage, Beijing Normal University, Beijing 100875, China 3School of Data Science and So ware Engineering, Qingdao University, Qingdao, China 4College of Automation and Electrical Engineering, Qingdao University, Qingdao, China


Introduction
Face recognition has been investigated for many years.The related applications of face recognition includes biometric analysis, security system, and information management.Traditional face recognition methods construct the recognition framework based on 2D facial images.The facial data in image are convenient to be achieved and the requirements for data acquisition devices are relatively low.However, there are many factors which influence the recognition rate of the methods such as illuminations, cosmetics, blur, facial expressions, different kind of occlusions (glasses, hair, and hand), and head poses.Such influence factors should be processed in a robust framework which increase the computation cost and complexity of algorithm.Some works attempt to research 3D facial data for face recognition.
The new technologies of 3D data scanning are developing fast in recent years.Comparing the traditional 3D scanning methods, the new scanning methods does not require complexity devices with strict conditions.Using a mobile phone with an additional scanning camera or function of structured light acquisition, the geometric information of a 3D face can be obtained.Based on the new scanning technologies, the 3D face recognition methods are proposed which extract the facial features from 3D geometric information.The advantages of the methods are obviously as follows: (1) robust to texture information of face; (2) providing a possible solution to remove the impact of head poses; (3) providing more geometric information to remove the impacts of facial expressions and different kinds of occlusions.
Based on the 3D facial data, we propose a novel face recognition method which map the 3D facial surface into the 2D parameter domain and extract the geometric features from the iso-geodesic stripes.In 2D parameter domain, the geometric features of 3D facial triangular meshes remained relative and the head pose influences to 3D face are degenerated into 2D scene.Combining the alignment of isogeodesic stripes and expressions robustness measurement of geometric features, the face recognition result can be achieved with a simple way from 2D parameter domain.More 2 Mathematical Problems in Engineering concretely, our face recognition framework includes three steps: (1) detecting the facial landmarks in 3D facial data and using the landmarks to extract the iso-geodesic stripes, which can be regarded as a preprocess; (2) mapping the 3D facial surface into the 2D parameter domain by local conformal parameterization and achieve a facial representation; (3) computing the geometric features from the iso-geodesic stripes in 2D parameter domain.The features' measure, we called Chain 2D Weighted Walkthroughs (C2DWW), is used to represent the geometric information of a 3D face and construct the face measurement function.The pipeline of our method is shown in Figure 1.In summary, our contributions are as follows: (1) We propose a pipeline of 3D face recognition which extract the facial features and compare different facial data in an automatically system.
(2) We propose a measurement method called Chain 2D Weighted Walkthroughs (C2DWW) to compare different facial data, which is robust to different facial expressions and head poses.
(3) We propose 2D facial representation using local conformal parameterization.The representation can be regarded as the facial geometric features which remove the influence of different head poses.
The remainder of our paper is organized as follows.In Section 2, we introduce some related works.In Section 3, we introduce the pre-process of our method.In Section 4, we discuss the 2D facial area distortion representation construction.In Section 5, we illustrate construction of C2DWW.The public facial databases, Texas 3D and Bosphorus3D, were used in our experiment and the evaluation of face recognition by different methods are discussed in Section 6.

Previous Work
For face recognition framework, the important problems are removing the influence of different factors such as head poses, facial expressions, cosmetics, illuminations, and occlusions.
Extracting the discrimination and robustness facial features from the input facial data which decide the facial recognition performance.According to different kinds of facial features, the face recognition methods can be divided into different classes: image features based, facial surface analysis based, local shape descriptors based, and partial face based.
The image features based method was constructing the face recognition framework based on facial images.The challenge task was using the single training sample per person in face recognition process.The facial features included discriminative features based on image patches [1], sparse representation features to build variation dictionary [2], expression subspace representation [3], sparse representation based on illumination transfer [4], local generic representation [5], and sparse discriminative multimanifold embedding features [6].Such methods attempted to reconstruct the highly discrimination facial features from single facial image to cover the different influence factors.In order to achieve the high face recognition rate, the feature extraction algorithm was complicated.Without global geometric features, the methods were limited by some facial images with extremely head poses, blur in image, and occlusions.
Based on the 3D facial surface, the global geometric features were used to construct face recognition framework.The classical thought was extracting facial features from the surface directly.Some methods constructed the parameter representation from the facial surface in Euclidean space such as canonical form [7] and iso-geodesic stripes [8].Based on non-Euclidean space, some researchers propose different frameworks to represent the facial features such as Ricci flow mapping [9], elastic measure based on radial curves [10,11], and spherical harmonic features match [12].The methods extracted the facial feature from facial surface to construct the global face match and achieved the accurate recognition result.However, the methods required the accurate facial surface data from the 3D face scan which limited the field of applications.The quality of raw face scan was affected by scanning devices, distance of the object and different of occlusions (hair, glasses, or hand) which increased the difficult of triangular mesh reconstruction.
Local shape descriptors based methods extracted discrete facial features from facial data to construct the face recognition framework.The discrete facial features included point cloud set [13], 3D key points based [14][15][16][17], and local surface analysis [18][19][20].The discrete facial features such as surface points and local geometric descriptors did not require complex preprocess for face cropping and high quality triangular meshes of 3D face.Some images based learning frameworks were convenient to be employed for such discrete facial features [18,21].However, such methods were limited by the local shape features representation.To achieve the global facial data analysis result, the learning frameworks of local shape features required large cost of computation and complex structure to remove the influence of facial expressions, different kinds of occlusions, and head poses.
Partial face based methods extracted facial features from partial face regions to construct face recognition framework.The methods selected local facial surface around eyes [22,23] and nose [24,25] which was not affected by different facial expressions.Such methods provided a simple solution to remove the influence factors in face recognition which did not require the complexity algorithm for facial feature extraction.For accurate face matching application, such method did not provide the global facial data analysis.In our method, we propose an improvement scheme for partial face based method.The iso geodesic strips around the nasal region are extracted and mapped into the 2D parameter domain.The influence of facial expressions and head poses can be removed in 2D parameter domain.

Preprocess for 3D Face
To construct 3D face recognition framework, we should extract the facial features from the 3D facial data at first.In our framework, the preprocess of facial features extraction includes two steps: facial landmarks detection and isogeodesic stripes extraction.Facial landmarks are needed to achieve geometric features and align different faces.In 3D face data, using shape analysis to detect special points is an enabling method discussed in recent research [26].We apply the idea to achieve the positions of the nasal tip and eyebrows tip.For iso-geodesic stripes extraction, we compute the geodesic of the two landmarks to define a confirm direction and a normalized distance.The geodesic path is extracted by [27].Combining the landmarks with the direction and the distance, we can achieve iso-geodesic stripes.

Achieve 2D facial reflection Pre-process of 3D facial data
In [8], iso-geodesic stripes have been defined.In facial surface, the stripes represent different banding areas that satisfy specific geodesic distances from points in areas to the nasal tip.The banding areas are adjacent to each other and represent different regions of the face.In Function (1), the iso-geodesic stripe is represented.S is the face surface, and p is the point in S. Every point in S has a geodesic distance to the nasal tip p  .Using different geodesic distances, we can achieve different banding areas c  .
In Function (1), the centre of iso-geodesic stripe is nasal tip.The strip does not cover the global facial surface and the nasal region is not considered seriously.The geometric features from single stripe region are insatiable in face recognition.
Following the geodesic path between nasal tip and eyebrows tip (nasal bridge curve), we extract different stripe regions by different centres.In Figure 2, we show the facial landmarks, nasal bridge curve, and different iso-geodesic stripe regions in face surface.

Construction of 2D Face Representation
The iso-geodesic stripes divide facial surface into different regions which are following the nasal bridge.Some methods [8,24,28] extract the facial features from the stripes directly to construct 3D face recognition framework.However, the influence of head poses are not removed from the stripes which reduce the accurate of facial features in the stripes.
In our framework, we map the 3D facial surface into a 2D parameter domain using local conformal parameterization method.In 2D parameter domain, the facial data are convenient to be aligned by two facial landmarks.Based on the face reflection in 2D parameter domain, we present the 2D face representation which correct the distortion of triangular meshes from 3D facial surface to 2D facial reflection.

. . Local Conformal Parameterization.
Local conformal parameterization is used to construct a parameter representation of 3D object, which can be applied in texture mapping and 3D object alignment.In our framework, the method  is employed to map the 3D facial data into 2D parameter domain and the reflection preserves the intrinsic characteristics.The parameterization process can be transferred to optimize two kinds of transfer energy: Dirichlet Energy   (2) and Chi Energy   (3) [29].
where   and   are the points in 3D facial surface,   and   are points in 2D mapping result, () means the adjacent points of i, and the angles   ,   , and   are shown in Figure 3.We achieve the local conformal parameterization result by computing the extreme value from the two energy functions in a linear equation group [29].The 2D reflection of face in 2D parameter domain can be achieved from the local conformal parameterization result.The border of the reflection is the outermost geodesic circle of the iso-geodesic stripe and the centre is the point in nasal bridge.Each triangular mesh in 3D facial surface is mapped into the 2D parameter domain.The meshes construct a new circle in the domain.In Figure 4, we achieve the 2D reflection result with iso-geodesic stripes.The facial landmarks are mapped into the 2D parameter domain.Connecting the nasal tip and eyebrows tip to achieve a direction vector and rotate the 2D reflection to make the vector point to the top, the influence of head poses can be removed.
. .D Face Representation.Ideally, the triangular meshes from 3D face to 2D facial reflection remain the complete geometric information using local conformal parameterization.However, the Gauss curvature of each points in 3D facial data are not the same, which express the change of the first fundamental form and the area distortions are produced in triangular meshes.Therefore, the facial features cannot be extracted from the 2D facial reflection directly.To correct the area distortions of triangular meshes in 2D facial reflection, we propose a 2D face representation which remain the area scaling rate.The area scaling rate is the area ratio between triangular mesh in 3D face and 2D facial reflection, which can be regarded as the "density" in different triangular meshes.For instance, S  is the 3D facial surface, 3D triangular mesh.Here   = { 1 , . . .,   },   is a triangle and k is the number of triangles.  is the 2D facial reflection of   ,   = { 1 , . . .,   ,   } and   is the 2D triangle.  is the area distortion representation,   = { 1 , . . .,   },   = area(  )/area(  ).In Figure 5, we show the area distortions of the 2D face representation.The area distortion is more obvious and the colour is deeper.

Chain 2D Weighted Walkthroughs
Based on the 2D face representation, we propose a facial surface measure method, Chain 2D Weighted Walkthroughs (C2DWW).C2DWW is based on the 2DWW [30], which is used to measure the relative positions in different isogeodesic stripes from different nasal regions.In Section 3, we have introduced that, based on different centres, the isogeodesic stripes can be obtained.A stripe region is defined by a set of iso-geodesic stripes which have same centre.Following the nasal bridge, we change the centre and achieve different stripes regions.Such regions can be regarded as a chain that cover the whole nasal region.Comparing different stripe regions from two faces, the face matching result can be achieved.The C2DWW can be divided into two levels: stripes measure in the same stripe region and stripe regions measure.
. .Stripes Measure in Same Stripe Region.The iso-geodesic stripes in our framework are represented by discrete point set.The stripes measure method is based on the discrete points' relative positions.The relative positions are represented by different codes in Figure 6 and (4). 1 ( 1 ,  1 ) and  2 ( 2 ,  2 ) are the points in the stripes.In each axis, there are three conditions of the relative positions.On the horizontal axis, the three conditions are right, left, and near.On the vertical axis, the three conditions are above, below, and adjacent.The threshold determines which conditions of the points are near.Using the appropriate threshold can reduce the influence of area change in conformal mapping.The weight code is proposed for certain relative positions measure.In (5), we show the computation of weight code.  and   are two iso-geodesic stripes from one face surface and A and B are indexes of the stripes.N(  ) and N(  ) means density point numbers of stripes   and   .The density point number is determined by discrete the area of the stripes.The detail algorithm is described in the following.First, we set a constant d to fix point numbers in   and   .Second, we assign the points to each triangle mesh according to area ratio between the mesh and the stripes.The area of the mesh and the stripes are corrected by area distortion representation.Finally, we compute the points number of stripes   and   .  are the number of points' pairs that satisfy the encode condition (i, j).The process is based on density point numbers in triangular mesh.In Figure 7, we show the computation process instance of   .According to different combinations of i and j, we achieve a 3 * 3 measure matrix of two stripes.In [30],   of different directions should multiply corresponding parameter to represent different degree of the distribution.In (6), we add the parameters to correct the measure matrix.
. .Stripe Regions Measure.Combining different   from the different stripes of 2D face representation, we propose the stripe regions measure.In (8),  1 and  2 are the stripes in face F and   1 and   2 are the stripes in face   .We achieve the similarity of  and   by measuring the stripes.

𝐷 (𝐹, 𝐹
In fact, there are not only two iso-geodesic stripes in a strip region.Therefore, we extend ( 8) to (11).The similarity measure of ( 11) is mathematical metric.It is determined by (9).  means the weight of the stripes   and   in face similarity measure.The index of the stripes i and j should be adjacent.The reason is that the adjacent stripes' relative positions are robust to facial expressions (see (12)).△ means the change of the points in stripes by facial expression.
= ( 1 , . . .,   ) , Based on the similarity measure between two stripe regions, the C2DWW measure result can be computed.We have introduced that the different stripe regions can be extracted from the nasal regions using different centres.Following the nasal regions, we extract different stripe regions and achieve the similarity measure result from corresponding stripes of two faces, which can be regarded as a "chain" measurement.In (13), the computation of C2DWW measure is proposed. 2 means the C2DWW measure results between two faces F and   .  is stripe region measure of two faces and the stripes region is constructed based on the centre q.

Experiments
Using the C2DWW measure, the face match results are achieved which can be used in face recognition directly.We evaluate the accurate of face recognition in this part.Two public facial databases, Texas3D and BosphorusDB, are used to evaluate our method identification rate.In Texas3D, there are 1149 scans from 118 persons with different facial expressions.Basically, the facial data in Texas3D are frontal face scans.In BosphorusDB, there are 4666 scans from 105 persons with different facial expressions, head poses, and occlusions.The experiments are constructed by three parts.Firstly, some parameters should be determined in our framework.Secondly, we evaluate the identity rate of different face recognition methods in public facial databases.Finally, we evaluate the identity rate in special facial data with different head poses and occlusions.
. .Parameter Configuration.In our framework, some parameters influence the accuracy of the facial surface measure method which determines the structure constructing performance.The number of the iso-geodesic stripes and the weights is shown in (11) and the stripe regions' number is shown in (13).
Naturally, different numbers of iso-geodesic stripes in a stripe region influence the performance of the method.The additional stripes can improve the precision, but the time complexity of the algorithm will increase.When stripes are more than one certain quantity, the improvement of the precision is not obviously.Two dense stripes are increasing sensitivity by facial expressions.On the contrary, insufficient quantity of the stripes cannot address the requirement for facial similarity measure.We select about 200 scans from 100 persons to be the gallery set and 100 scans to be probe set in Texas3D.In Figure 8, we show receiver operating characteristic curves (ROC) results with different number of the stripes (S3 = 3, S4 = 4, S5 = 5, S6 = 6, S7 = 7).For the weights of (9), we choose a set of value (  = 1/3,   =   = 1/4,  0 =  0 = 1/12) [30].We use single strip region which is constructed from the nasal tip.For the weights of (11), the general way is using the learning framework such as linear regression to optimal a set of weights.We just use the same weights in our frameworks because the weights from the learning methods may produce the locally optimal results.In certain applications, the weights can be computed according to the actual situation.
Another parameter that should be determined is the number of the stripe regions.Following the centre points in nasal bridge, the different number of stripe regions can be selected in the face recognition framework.In Section 3, we have introduced that the centres are achieved from the geodesic path between nasal tip and eyebrows tip which can be regarded as the nasal bridge curve.We divide the curve into equal segments; the equal points can be regarded as the centres.In Figure 9, we also show facial recognition ROC results in Texas3D with different number of the equal points (P1 = 1, P2 = 2, P3 = 3, P4 = 4, P5 = 5, P6 = 6.P1 means there have only one strip region in face and the centre is nasal tip.P2 means there are two stripe regions and the centres are nasal tip and eyebrows tip.Pn means there are n stripe regions and the centres include nasal tip, eyebrows tip, and n-2 end points in nasal bridge curve).In summary, the parameters with best identification rate are achieved (S5 and P4).
. .Facial Identification in Different Facial Expressions.In face recognition, the influence of facial expressions should be considered seriously.We evaluate the identification performance of our framework to other methods ( [8,11,24,28]), which also use the geodesic stripes or geodesic curves to construct the face recognition framework.The test sets are constructed from Texas3D and BosphorusDB.For Texas3D, we select the same test set which is introduced in parameter configuration part.For BosphorusDB, we select 6 samples from each person (totally 630) with different expressions to be the gallery set and 2 samples from each person (totally 210) without obvious facial expressions to be the probe set.We compute the Cumulative Matching Characteristics (CMC) and ROC result using different methods in the test sets.In Figure 10, we show the CMC and ROC results by different methods in the test set of Texas3D.In Figure 11, we show the CMC and ROC results by different methods in the test set of BosphorusDB.In Table 1, we show the evaluation results for different methods.The results show that our method can achieve better recognition performance for the facial data with different expressions.
. .Facial Identification in Different Head Poses and Occlusions.For facial data in BosphorusDB, there are different influence factors such as head poses and occlusions (hand and hair).Such influence factors should be considered in face recognition framework.For some facial scans with large head poses and occlusions, the facial data just have half facial surface for recognition process.In our framework, the C2DWW can achieve reasonable measure result from half face.The reason is that the face data has symmetry character and the relative positions in half face have similar value to whole face.We construct a subtest set from BosphorusDB to evaluate the performance of our method in half face.We select 6 scans from each person in BosphorusDB (totally 630) which include different head poses and occlusions to be the probe set and gallery set which are discussed in previous part.We extract the half face data from the probe set to evaluate the identification performance.In Figure 12, we show the CMC and ROC results.
The test set is constructed from Texas3D and Bospho-rusDB.In Table 2, we show the evaluation results for different methods in subtest set.The results show that our method can achieve better recognition performance for the facial data with different head poses and occlusions.
. .Performance Evaluation.In this part, we evaluate the performance of face matching speed by different methods.
The methods [11,24,28] extract geodesic curves from facial surface to be facial features directly.The time cost of different curves matching is huge for face recognition process in a large facial database.The method [8] constructs geodesic stripes in facial surface and computes the relative positions of vertexes in different stripes.The geometric features are coded into a low dimensional feature vector and the face match process can be transferred to linear vectors computation.Our framework inherits the advantage of fast computation from method [8] and removes the influence of head poses  and some occlusions.In Table 3, we show the average face searching time cost by different methods in different facial databases (the time cost do not include the preprocess time consume).

Conclusion
We propose a 3D face recognition method which is based on local conformal parameterization and iso-geodesic stripes analysis.Using the local conformal parameterization, we achieve the 2D face representation which remove the influence of head poses.Based on the iso-geodesic stripes, we extract the facial features from the 2D face representation and use the C2DWW to achieve the face match results.The advantages of our method are robust to different head poses, facial expressions, and some kinds of occlusions.However, the biggest limitation of our method is sensitive to occlusions in nasal region.The occlusions in nasal region break the nasal geometric features which affect the accurate of local conformal parameterization.
In future work, we will find a solution to solve the problem such as using a face modeling reconstruction to repair the geometric information of nasal region.We will consider more facial information such as eyes' region and mouth's region in face recognition framework.We will attempt to find more convenient facial features to construct face representation which does not require the facial surface searching such as geodesic path computation.

Figure 1 :Figure 2 :
Figure 1: The pipeline of our face recognition framework.

Figure 4 :
Figure 4: The 2D mapping result of 3D facial surface by local conformal mapping.

Figure 9 :
Figure 9: The ROC result with different number of the stripes in Texas3D.

Figure 10 :Figure 11 :
Figure 10: The CMC and ROC results for identification by different methods in Texas3D (the left picture is CMC result; the right picture is ROC result).

Figure 12 :
Figure 12: The CMC and ROC results for identification by different methods in subtest set of BosphorusDB (the left picture is CMC result; the right picture is ROC result).

Table 1 :
Evaluation of face recognition for different methods (Rank 1 and FAR0.1).

Table 2 :
Evaluation of face recognition for different methods in subtest set of BosphorusDB (Rank 1 and FAR0.1).

Table 3 :
Average face searching time cost by different methods in different facial databases (second).