Mathematical Problems in Engineering
Volume 2016 (2016), Article ID 2370919, 14 pages
http://dx.doi.org/10.1155/2016/2370919

Research Article

An Improved Surface Simplification Method for Facial Expression Animation Based on Homogeneous Coordinate Transformation Matrix and Maximum Shape Operator

Juin-Ling Tseng

Department of Computer Science and Information Engineering, Minghsin University of Science and Technology, No. 1, Xinxing Road, Xinfeng Township, Hsinchu County 304, Taiwan

Received 28 September 2015; Revised 2 January 2016; Accepted 10 January 2016

Academic Editor: Fazal M. Mahomed

Copyright © 2016 Juin-Ling Tseng. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract.

Facial animation is one of the most popular 3D animation topics researched in recent years. However, when using facial animation, a 3D facial animation model has to be stored. This 3D facial animation model requires many triangles to accurately describe and demonstrate facial expression animation because the face often presents a number of different expressions. Consequently, the costs associated with facial animation have increased rapidly. In an effort to reduce storage costs, researchers have sought to simplify 3D animation models using techniques such as Deformation Sensitive Decimation and Feature Edge Quadric. The studies conducted have examined the problems in the homogeneity of the local coordinate system between different expression models and in the retainment of simplified model characteristics. This paper proposes a method that applies Homogeneous Coordinate Transformation Matrix to solve the problem of homogeneity of the local coordinate system and Maximum Shape Operator to detect shape changes in facial animation so as to properly preserve the features of facial expressions. Further, root mean square error and perceived quality error are used to compare the errors generated by different simplification methods in experiments. Experimental results show that, compared with Deformation Sensitive Decimation and Feature Edge Quadric, our method can not only reduce the errors caused by simplification of facial animation, but also retain more facial features.

1. Introduction

Concomitant with the rapid advancements in information technology, the application of 3D animation has become increasingly popular in fields such as the movie industry [1, 2], gaming [3–5], arts [6, 7], and education [8, 9]. Among the various 3D animation technologies, facial animation is the most commonly used technique. However, because facial animation contains a variety of expressional contents, the surface relief of 3D models in facial animation undergoes considerable changes.

Parke [10] was the first to propose the concept establishing facial animation. Subsequently, many related studies and techniques, including Head Pose Estimation [11], Facial 3D Shape Estimation [12], Head Shop [13], The Digital Emily Project [14], Automatic Generation [15], Kinect-Based Facial Animation [16], Real-Time Facial Animation [17–21], and 3D Facial Similarity Measure [22], emerged.

However, in order to elaborate facial animation, numerous triangles are needed, which significantly increases storage costs. Thus, in order to reduce the number of triangles needed to describe the face animation, many experts and scholars have proposed simplified 3D animation methods, including Deformation Sensitive Decimation (DSD) [23, 24], Feature Edge Quadric (FEQ) [25], Facial Features Region Partition [26], and MPEG-4 Quadric-based LoD Simplification [27]. However, most of them are derived from QSlim [28], proposed by Garland and Heckbert, and DSD [23], proposed by Mohr and Gleicher.

QSlim [28] is one of the most famous 3D model simplification methods [29]. This method can not only execute rapidly, but also reduce the errors caused by model simplification. Unfortunately, QSlim can only simplify 3D static models, whereas 3D animation typically contains various frame models. Consequently, in order to utilize QSlim for simplification of 3D animation, Mohr and Gleicher proposed the DSD method [23]. The method calculates and aggregates the error matrix of each vertex in different frame models to serve as a basis for simplification. However, DSD can cause destruction of the appearance characteristics of the model because it lacks complete homogeneous coordinate transformation.

In order to preserve the facial expression features when simplifying face animation, Kim et al. [25] established 32 feature points and adopted FEQ to preserve facial animation features, as shown in Figure 1; Wang et al. [26] used Facial Features Region Partition to set a scope on the face, which is intended to preserve the desired facial features. Its simplification results are shown in Figure 2. However, in these methods, the feature regions must be set by the user before the shapes of the regions are preserved. In other words, these methods can only preserve general facial features, such as the eyes and nose. For nongeneral features, such as wrinkles, it is most likely that they are not in the set range and their triangles can easily be removed, resulting in damage to the features of the nonsetting regions.

Figure 1: Simplification results using Feature Edge Quadric [25].

Figure 2: Simplification results using Facial Features Region Partition [26].

In order to overcome this limitation of feature regions having to be set by users and to preserve the facial features during the simplification process, this paper proposes a DSD-based method that uses Homogeneous Coordinate Transformation Matrix (HCTM) to solve DSD’s homogeneous coordinate transformation problems and uses Maximum Shape Operator (MSO) to estimate the changes in each vertex point in the various frame models. In this way, the intensity values of each facial characteristic are automatically quantified and the characteristics of facial animation are moderately preserved during the simplification process.

2. Related Works

2.1. Deformation Sensitive Decimation (DSD)

QSlim [28] is a Quadric Error Metrics- (QEMs-) based 3D model simplification method. This method first calculates the matrix of the adjacent triangular piece of each vertex: = . In this formula, and the matrix is based on the calculation of the distance of any point from the plane, . The sum of distances of any point to its adjacent triangle is computed by adding all the matrices of all adjacent triangular patches. This sum of distances, which can be obtained using formula (1), is the error matrix values resulting from the simplification of vertex pairs. In the QSlim process, the simplification is performed in an orderly manner from the vertex pair of the lowest error to the highest, as shown in Figure 3. Consequently, this method can efficiently generate a simplified low error model as follows:

Figure 3: Surface simplification using Quadric Error Metrics [28].

In this formula,

QSlim is utilized to simplify 3D animation models by DSD [23]. This method mainly calculates the matrix of each vertex in different frame models of animation, as shown in formula (3), to estimate the errors each vertex pair () produces during the simplification process and then to decide which vertex pair should be simplified first, as shown in formula (4). This method inherits the speedy execution of QSlim but lacks complete homogeneous coordinate transformation [30]. Consequently, it cannot generate the best vertex point after the simplification of vertex pairs, as illustrated in Figures 4 and 5. Consider

where is the number of frame models in 3D animation; refers to the new vertices, and , generated in the th frame model after simplification; and and are the respective error matrices of and in the overall 3D animation model.

(a) Original model (16842 triangles)

(b) Simplifying by the DSD (2000 triangles)

Figure 4: The shape of horse hoof is destroyed gradually when simplifying by the DSD [30].

(a)

(b)

Figure 5: DSD cannot generate the best vertex point after the simplification of vertex pairs. (a) Original model (57835 triangles); (b) simplifying by the DSD (5000 triangles).

In addition, the DSD simplification method is not aimed at analyzing the surface of the 3D animation model. Consequently, it is relatively easy for the important features of the face to be damaged during the facial animation simplification process. To solve this problem, Kim et al. proposed Feature Edge Quadric method with the aim of defining the important feature regions of the face and further preserving the main facial animation features during the simplification process.

2.2. Feature Edge Quadric (FEQ)

FEQ was proposed by Kim et al. for preservation of facial features. In the proposed method, the feature points of 10 parts in the face, including tip of the nose, top of the head, left/right side of the face, top/bottom of the nose, left/right eye socket, bottom left/right of the nose, lip contact line, top/bottom of the lip, chin, and throat, are set as shown in Figure 1. For error estimation, this method contains basic quadric error and Feature Edge Quadric. The former is based on QSlim and calculates the error matrix generated by each vertex pair () during the simplification process, as shown in the following formula:

where is the set of triangles adjacent to , is the set of triangles adjacent to , is the area of triangle , and is the error matrix of triangle .

In the latter, the error matrix of the adjacent feature edges of each feature point is calculated to properly preserve feature edges during the facial animation simplification process.

To calculate the error matrix of feature edge , FEQ first computes the planes , which are orthogonal to the edge and can be calculated by and the average normal vector of its two adjacent triangles, , as shown in the following formula:

where , where and are the normal vectors of two triangles adjacent to .

Through the integration of basic quadric error and Feature Edge Quadric, the formula of the error matrix can be deduced, as shown in

where , where is the set formed by the adjacent feature edge of vertex point , and is the weight value (0-1) customized by users (0.5 by default).

Although the FEQ simplification method aims to preserve important features of the eyes, nose, mouth, and so forth, there are many more facial features than these organ features. The forehead, cheeks, and other areas also change significantly with changes in facial expressions. However, FEQ easily ignores the expression features and causes a rapid increase in the simplified errors during the process of simplification. Therefore, our paper proposes a novel simplification method that integrates MSO and HCTM and properly takes into account the overall facial features, thereby reducing errors generated during the simplification of facial animation.

3. Homogeneous Coordinate Transformation Matrix (HCTM)

Because 3D facial animation is composed of various different expression models, the tangent plane and normal vector of the same vertex point also vary with changes in expressions, as shown in Figure 6. In other words, the local coordinate system for the same vertex point varies in different frame models. Therefore, if the coordinate transformation system is not accurate, errors can be easily caused by the simplification of the facial animation models. For this reason, the DSD cannot accurately calculate the optimal vertex position after the simplification of vertex pairs. This paper adopts Theorems 1 and 2 and proposes HCTM to solve this problem.

Figure 6: The local coordinate system for the same vertex point varies in different expression models.

Theorem 1. The Homogeneous Coordinate Transformation Matrix that corresponds to a translation by is given by the following formula [31]:

Theorem 2. The Homogeneous Coordinate Transformation Matrix that corresponds to a rotation of about the axis in the direction of the unit vector is given by the following formula [31]:

where and .

In this paper, HCTM sets the local coordinate space for each vertex point in the first expression model of the facial animation as the main coordinate space. Then, the vertex points in all other expression models are transformed into the corresponding local coordinate space in the first expression model.

Suppose that is the vertex point of in frame ; is the Mesh formed by the adjacent vertex points of ; and and are the tangent plane and normal vector in . HCTM must convert from its original local coordinate system to the local coordinate system in the first expression model. This process contains two steps: translation of to the coordinate system that takes as the origin and and as the tangent plane and normal vector and rotation of the tangent plane and normal vector to overlap with and . The transformation matrix in step one can be defined by Theorem 1 as follows:

In this formula,

The transformation matrix in step two can be defined by Theorem 2 as in formula (12). In this formula, and is the angle included between and . Consider

Integrating formulas (10) and (12), HCTM can be defined, as is shown in (13). Mesh and the error matrix can be converted from one to the other with (13), as is shown in formulas (14) and (15) as follows:

With HCTM, the same Mesh in different expression models can be converted to the local coordinate system in the first expression model, and its error calculation can be revised using formula (3) defined by DSD, as shown in formula (16). This method can solve the error value increase problem resulting from incomplete homogeneous coordinate transformation in DSD as follows:

4. Maximum Shape Operator (MSO)

To detect the shape features of the facial animation model, this paper presents a shape operator that estimates the changes in the 3D surface. In general, shape operator is mainly used to calculate the shape change of the specific tangent direction of the 3D object’s surface, as in Definition 3. However, in this paper, the shape change of the 3D object’s surface, on which Mesh lies, is estimated by the edge of the Mesh.

Definition 3. Let be a regular surface, and let be a surface normal to defined in a neighborhood of a point . For a tangent vector to at , we put

Then, is called the shape operator [32, 33].

Shape operator refers to the change degree of the normal vector field of surface ’s vertex point in its tangent direction . To calculate the adjacent surface changes of the vertex points in the 3D model, in this paper, shape operator is extended to estimate and integrate the shape operator values of the vertex point and its adjacent vertex points. Suppose that the number of adjacent vertex points of vertex point is , called , and its tangent vector to each vertex point is, respectively, , and , as shown in Figure 7. Consequently, , which signify the shape operator value of vertex in the , and directions can be generated according to the definition of shape operator, as shown in formula (18). After integrating the shape operator in each of these different tangent directions, the surface change of the region in which the vertex lies is shown in formula (19) as follows:

Figure 7: Shape operators of point using the tangent vectors to each neighboring point to estimate local surface variation.

Shape operator can automatically analyze the surface changes of the facial expression model and extract the important features, such as eyes, nose, and mouth, as shown in Figure 8. This method covers the drawback in FEQ of each feature region having to be defined by the user. However, facial animation is not static. As time passes, the face may produce various facial features, including nasolabial folds and forehead wrinkles. Therefore, in order to preserve facial expression features during simplification of the facial animation, this paper uses MSO to extract the facial animation feature. This method calculates the shape operator value of each vertex point in (the number of expression models) expression models and takes the maximum value as the eigenvalue for appearance analysis of facial animation. The MSO is defined as follows:

where is the shape operator value of vertex in frame . Figure 9 shows the results of the surface changes of the face animation extracted by MSO. It can be seen from the result that nasolabial folds are obvious in expressions of surprise, grin, laugh, and smile, but not obvious in expressions of fury, anger, rage, and sadness. However, if the features of those expressions are ignored during the face animation simplification process, a rapid increase in simplification errors can easily occur. With MSO, the proposed method can extract the features of nasolabial folds to reduce the errors caused by the simplification of the facial animation.

(a)

(b)

(c)

(d)

Figure 8: Shape operator can automatically analyze the surface changes of the facial expression model. (Red indicates the high-variation surfaces. Yellow indicates medium-variation regions. Green represents low-variation areas.)

Figure 9: This method calculates the shape operator value of each vertex point in every expression model and takes the maximum value as the eigenvalue for appearance analysis of facial animation.

5. The Proposed Algorithm

The algorithm proposed in this paper is mainly based on DSD, with the introduction of MSO and HCTM to reduce the errors caused by the simplification of the face animation model. The main steps in this algorithm are as follows.

(1)Calculate the QEM of vertex in each facial expression model:

 where is the set of triangles adjacent to in facial expression model and is the matrix formed in plane , as is shown in formula (1).(2) Calculate HCTM and update QEM to obtain the following:

(3) Estimate the MSO of each vertex point :

 where is the number of vertex points adjacent to in facial expression model , is the vertex point adjacent to in facial expression model , and is the shape operator of .(4) Sum all QEMs of and import MSO into the QEM:

 where is the MSO value of , is the mean value of MSO, and is the standard deviation of MSO. The original MSO value is between zero and 1,241.77, which is scattered and has some outliers. In order to balance the effect of QEM and MSO, in this paper, the MSO is standardized to , and the outliers that are higher than are set to the maximum value of one.(5) Calculate the minimum simplification error of each vertex pair with the following conditions:

(a) is an edge; or(b) is not an edge, but its distance is smaller than the threshold value set by users.

(6) Choose the vertex pair with the smallest error in step (5) for simplification.(7) Simplify the vertex pair into a vertex point , and update the QEM of , where .(8)Update all the information adjacent to the vertex points or .(9) Repeat steps (5) to (8) until the number of triangles has been reduced to the designated value.

6. Experimental Results

This study used an Intel Core 2.2 GHz CPU with 1 GB RAM as the main execution environment. The eight experimental facial animation models included fury, surprise, anger, grin, laugh, rage, sadness, and smile. This experiment compared the differences between this method and DSD and FEQ and used root mean square (RMS) error and perceived quality to analyze the distortion errors caused by these methods so as to verify the superiority of this method over DSD and FEQ.

The eight original expression models adopted in this experiment are shown in Figure 10. Each model had 29,299 vertex points and 57,836 triangles. When the number of triangles in the facial animation model is reduced to 5,000, 2,000, 1,000, and 500 through DSD, the generated RMS error values are shown in Table 1.

Table 1: RMS error comparison between our method and DSD (unit: 10⁻²).

Triangles
The DSD
Our method
Improvement rate

Face 01 (fury)

5000
5.7313
0.8427
85.30%

2000
11.3899
1.6240
85.74%

1000
22.9495
2.6013
88.67%

500
42.7859
4.7140
88.98%

Face 02 (surprise)

5000
4.8805
1.4011
71.29%

2000
9.6882
2.7791
71.31%

1000
21.3109
4.2362
80.12%

500
37.8799
6.6230
82.52%

Face 03 (anger)

5000
4.6126
1.1371
75.35%

2000
9.1516
2.3843
73.95%

1000
19.3625
3.9031
79.84%

500
35.8760
6.6498
81.46%

Face 04 (grin)

5000
5.2081
1.3187
74.68%

2000
10.2029
2.4012
76.47%

1000
21.2078
3.8791
81.71%

500
39.2217
6.6697
82.99%

Face 05 (laugh)

5000
4.8700
1.2308
74.73%

2000
10.1936
2.4448
76.02%

1000
21.3471
3.8759
81.84%

500
39.3825
6.5277
83.42%

Face 06 (rage)

5000
5.3178
1.1113
79.10%

2000
10.2902
2.3494
77.17%

1000
21.0917
3.6617
82.64%

500
37.7458
5.6601
85.00%

Face 07 (sad)

5000
4.6088
1.4588
68.35%

2000
9.9824
2.9477
70.47%

1000
21.0213
4.4211
78.97%

500
38.0176
7.1686
81.14%

Face 08 (smile)

5000
4.6887
1.1734
74.97%

2000
9.8362
2.4347
75.25%

1000
21.4011
3.7970
82.26%

500
39.8993
5.9372
85.12%

Figure 10: The original expression models.

With changes in the expression model, the error caused by DSD and this method will be different. Take face 01 (fury) as an example: when the number of triangles in the facial animation model is reduced to 5,000, 2,000, 1,000, and 500 through DSD, the generated RMS error values are 5.7313 × 10⁻², 11.3899 × 10⁻², 22.9495 × 10⁻², and 42.7859 × 10⁻², respectively. However, the error can be reduced to 0.8427 × 10⁻², 1.6240 × 10⁻², 2.6013 × 10⁻², and 4.7140 × 10⁻² using this method, resulting in an improvement rate of between 85.30% and 88.98%. For other expression models, this method is better than the DSD method, with an improvement rate of more than 68.35%. In addition, with the reduction in the number of triangles, the improvement rate is increasingly better. Take face 01 (fury) as an example: when the number of triangles is reduced to 5,000, the improvement rate of this method is 85.30% compared with the DSD; and when the number is 500, the improvement rate is 88.98%. In other words, this method is better than DSD in low triangle numbers and can retain more facial shapes than DSD.

The average errors generated by all the facial expressions in the facial animation are shown in Table 2 and Figure 11. The table shows the errors generated from simplifying the entire 3D facial animation using DSD and this method, respectively. In Table 2, it can be seen that when the number of triangles is simplified to 5,000, 2,000, 1,000, and 500, the errors generated by DSD are 4.9897 × 10⁻², 10.0919 × 10⁻², 21.2115 × 10⁻², and 38.8511 × 10⁻², whereas those of this method are 1.2092 × 10⁻², 2.4207 × 10⁻², 3.7969 × 10⁻², and 6.2438 × 10⁻², which are improved by 75.77% to 83.93% compared with DSD.

Table 2: Average RMS error comparison between our method and DSD (unit: 10⁻²).

Triangles
The DSD
Our method
Improvement rate

5000
4.9897
1.2092
75.77%

2000
10.0919
2.4207
76.01%

1000
21.2115
3.7969
82.10%

500
38.8511
6.2438
83.93%

Figure 11: RMS error comparison between our method and DSD.

In addition to the use of QEM for simplification like DSD, FEQ also estimates the simplification errors of important facial features such as the eyes, nose, mouth, and ears. Therefore, FEQ is also better than DSD in terms of simplification results. However, the features of facial animation include not only the eyes, nose, mouth, and ears, but also wrinkles on an angry forehead, protuberant cheeks, and obvious nasolabial folds on a smiling face.

The data in Table 3 and Figure 12 indicate that when the number of triangles is simplified to 5,000, 2,000, 1,000, and 500, the errors generated by FEQ are 1.2658 × 10⁻², 2.7134 × 10⁻², 5.2143 × 10⁻², and 10.6767 × 10⁻², which are better than DSD’s but are only limited to general facial features such as eyes, nose, mouth, and ears, having no obvious improvement in other expression features like wrinkles in forehead and nasolabial folds in cheeks.

Table 3: Average RMS error comparison among our method, FEQ, and DSD (unit: 10⁻²).

Triangles
The DSD
The FEQ
Our method

5000
4.9897
1.2658
1.2092

2000
10.0919
2.7134
2.4207

1000
21.2115
5.2143
3.7969

500
38.8511
10.6767
6.2438

Figure 12: Average RMS error comparison among our method, FEQ, and DSD.

The error comparison results in Table 4 indicate that when the number of triangles is simplified to 5,000, 2,000, 1,000, and 500, the respective errors generated by this method are 1.2092 × 10⁻², 2.4207 × 10⁻², 3.7969 × 10⁻², and 6.2438 × 10⁻², which are all better than EFQ’s. The improvement rate reaches 4.47% when the number of triangles is simplified to 5,000, and it even reaches 41.52% when the number is 500.

Table 4: Average RMS error comparison between our method and FEQ (unit: 10⁻²).

Triangles
The FEQ
Our method
Improvement rate

5000
1.2658
1.2092
4.47%

2000
2.7134
2.4207
10.79%

1000
5.2143
3.7969
27.18%

500
10.6767
6.2438
41.52%

In the preservation of expression features, as shown in Figure 13, when the model is simplified to 5,000 triangular pieces, as shown in Figure 13(b), the FEQ method retains 129 triangles in the forehead wrinkled area; however, this method has 123 fully covered and 37 half-covered triangles in the same area. If we use 0.5 as a unit for each half-covered triangle, then our method obtains 141.5 triangles in the forehead wrinkled area. In other words, for this feature area, this method retains 10% more triangular pieces than the FEQ. Moreover, when the model is reduced to 1,000 triangles, as shown in Figure 13(c), the FEQ only has 15.5 triangles in the forehead wrinkled area (including 11 fully covered and 9 half-covered triangles), but this method has 28.5 triangles (20 fully covered and 17 half-covered triangles); 84% more triangles are retained compared with the FEQ.

(a) Original model

(b) The simplification results of forehead wrinkles using FEQ (left) and our method (right) when the model is simplified to 5,000

(c) The simplification results of forehead wrinkles using FEQ (left) and our method (right) when the model is simplified to 1,000

Figure 13: Comparison between FEQ and our method when the number of triangles in the face 03 (anger) model is simplified to 5,000 and 1,000.

In addition, in the cheeks and the nasolabial fold areas, as shown in Figure 14, when the number of triangles in the model is reduced to 1,000 using the FEQ, these areas retain 61 triangles. However, by using the proposed method to simplify the model, 113.5 triangles are retained (including 97 fully covered and 33 half-covered triangles), which is approximately 86% more triangles as compared to the FEQ.

(a) FEQ

(b) Our method

Figure 14: Comparison between FEQ and our method when the number of triangles in the face 04 (grin) model is simplified to 1,000.

From the simplified results of forehead wrinkles, cheeks, and nasolabial fold areas, it can be seen that this method retains a greater number of triangles than the FEQ. It also shows that, after simplification, this method gets closer to expressing the original facial model in these feature areas than the FEQ method.

To verify the effectiveness of our method, we also adopted perceived quality to further compare it with DSD and FEQ. Perceived quality is mainly used to compare visual differences. We used tensor-based perceptual distance measure (TPDM) [34], proposed by Torkhani et al., to evaluate the perceived quality values of the model before and after simplification. The perceived quality value is between zero and one. A value closer to one signifies that the appearance of the simplified model is very similar to the original, whereas a value closer to zero signifies that the appearance of the simplified model is very different from the original.

In Table 5, it can be seen that the perceived value generated by our method is 1.7413 × 10⁻¹ to 3.0275 × 10⁻¹, results that are clearly superior to DSD’s 2.4633 × 10⁻¹ to 4.6055 × 10⁻¹, with its improvement rate of 29.31% to 37.30%. In Table 6, it can be seen that the perceived value generated by FEQ is 1.7413 × 10⁻¹ to 3.0275 × 10⁻¹. Thus, this method generates a lower perceived value than FEQ and can achieve an improvement rate of 2.33% to 12.88%, as shown in Figure 15.

Table 5: Perceived quality comparison between our method and DSD (unit: 10⁻¹).

Triangles
DSD
Our method
Improvement rate

5000
2.4633
1.7413
29.31%

2000
3.2981
2.2074
33.07%

1000
4.0961
2.5682
37.30%

500
4.6055
3.0275
34.26%

Table 6: Perceived quality comparison between our method and FEQ (unit: 10⁻¹).

Triangles
FEQ
Our method
Improvement rate

5000
1.7828
1.7413
2.33%

2000
2.3440
2.2074
5.83%

1000
2.8950
2.5682
11.29%

500
3.4751
3.0275
12.88%

Figure 15: Perceived quality comparison among our method, FEQ, and DSD.

In addition, in terms of time cost, given that this improved method is based on DSD, it inherits benefits such as lower computation time. In order to perform the time cost analysis, this paper divided the entire implementation process into four phases, namely, the setup time, initialization time, simplification running time, and output time.

In the setup time phase, the main work is to input the original facial models, including vertex coordinates and triangle information. In the initialization time phase, this method calculates relevant information needed to simplify the model, including QEM, HCTM, and MSO. In the simplification running time phase, the method mainly records the execution time needed to simplify the model until it reaches a specific required number of triangles.

In the final output time phase, it records the time needed to output the simplified facial model. The average time required for each execution process is shown in Table 7. As is evident from the table, the execution speed of this method is significantly fast. The overall simplification time is only about 8 seconds, in which the initialization time for HCTM and MSO is just over 2-3 seconds, accounting for about 37% of the total simplification time. Undoubtedly, this method is very efficient in model simplification.

Table 7: Time analysis for each execution process in facial model simplification (unit: second).

Triangles
Setup time
Initialization time
Running time
Output time
Total time

5000
2.95
2.92
1.95
0.22
8.04

2000
2.78
3.05
2.09
0.17
8.09

1000
2.53
2.97
2.13
0.11
7.74

500
2.78
2.88
2.19
0.09
7.94

Average
2.76
2.96
2.09
0.15
7.95

7. Conclusions

In order to analyze the shape changes of facial animation and to reduce simplification errors, this paper proposed HCTM to modify the homogeneity of the local coordinate system for different models and adopted MSO to automatically analyze the degree of facial animation shape change, to locate the region with most expression changes and to rectify the drawback wherein feature regions such as the eyes, nose, mouth, and ears had to be defined by users. In experiments conducted, RMS and perceived quality errors were utilized to compare the simplification results of the proposed method with those of DSD and FEQ. The experimental results show that the errors caused by this method are lower than those of DSD and FEQ. Furthermore, this method can not only properly retain the facial features of fixed positions such as the eyes, nose, mouth, and ears, but also preserve more triangles than other methods in other important feature regions such as wrinkles on the forehead, cheeks, and nasolabial folds. Thus, it satisfies the requirement that the simplified facial animation should be as elaborate and natural as possible.

Conflict of Interests

The author declares that there is no conflict of interests regarding the publication of this paper.

Acknowledgments

The author thanks Professors Chassery, Wang, and Torkhani for providing the TPDM codes. Additionally, this research was supported by the Minghsin University of Science and Technology, Taiwan, under Grant MUST-104CSIE-2.

References

S. Karunaratne and H. Yan, “3D animated movie actor training using fuzzy logic,” in Proceedings of the IEEE Computer Graphics International, pp. 23–30, IEEE, Hong Kong, July 2011.

M. Salvati, M. Kinoshita, Y. Katsura, K. Anjyo, T. Yotsukura, and H. Uchibori, “Developing tools for 2D/3D conversion of Japanese animations,” in Proceedings of the ACM SIGGRAPH Talks (SIGGRAPH '11), Article no. 14, ACM, Vancouver, Canada, August 2011.

M. Song and P. Grogono, “Application of advanced rendering and animation techniques for 3D games to softbody modeling and animation,” in Proceedings of the 2nd Canadian Conference on Computer Science and Software Engineering (C3S2E '09), pp. 89–100, ACM, Montreal, Canada, May 2009.

M. Zhao and J. Zhang, “Rapidly product and optimize facial animation methods for 3D game,” in Proceedings of the International Conference on Internet Computing in Science and Engineering (ICICSE '08), pp. 136–139, IEEE, Harbin, China, January 2008.

L. Bangquan and M. Yun, “A facial animation based on emotional model for characters in 3D games,” in Proceedings of the IEEE International Conference on Computer Science and Information Processing (CSIP '12), pp. 1304–1307, Xi'an, China, August 2012.

I. Conradi and Y. Y. X. Ivan, “The undiscovered country: the art of pictorial 3-D stereo animation,” in Proceedings of the International Conference on Cyberworlds, pp. 181–188, IEEE, Ontario, Canada, October 2011.

V. Janarthanan, “Innovations in art and production: sound, modeling and animation,” in Proceedings of the IEEE 9th International Conference on Information Technology: New Generations (ITNG '12), pp. 879–882, IEEE, Las Vegas, Nev, USA, April 2012.

M. Aoki, W. Koning, A. Miyai, and T. Kamihira, “3D animation education in the US and Japan: different environments, similar issues,” in Proceedings of the SIGGRAPH Asia Sketches (SA '11), ACM, Hong Kong, December 2011.

H. Croft, R. Rasiah, J. Cooper, and K. Nesbitt, “Comparing animation with video for teaching communication skills,” in Proceedings of the ACM Conference on Interactive Entertainment (IE '14), pp. 1–10, Newcastle, Australia, December 2014.

F. I. Parke, “Computer generated animation of faces,” in Proceedings of the ACM Annual Conference (ACM '72), vol. 1, pp. 451–457, ACM, Boston, Mass, USA, August 1972.

Y. Cai, M. Yang, and Z. Li, “Robust head pose estimation using a 3D morphable model,” Mathematical Problems in Engineering, vol. 2015, Article ID 678973, 10 pages, 2015.

U. Musti, Z. Zhou, and M. Pietikainen, “Facial 3D shape estimation from images for visual speech animation,” in Proceedings of the 22nd International Conference on Pattern Recognition (ICPR '14), pp. 40–45, Stockholm, Sweden, August 2014.

K. Kähler, J. Haber, H. Yamauchi, and H.-P. Seidel, “Head shop: generating animated head models with anatomical structure,” in Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation, pp. 55–63, San Antonio, Tex, USA, July 2002.

O. Alexander, M. Rogers, W. Lambeth, M. Chiang, and P. Debevec, “Creating a photoreal digital actor: the Digital Emily project,” in Proceedings of the 6th European Conference for Visual Media Production (CVMP '09), pp. 176–187, IEEE, London, UK, November 2009.

A. Maejima, H. Yarimizu, H. Kubo, and S. Morishima, “Automatic generation of head models and facial animations considering personal characteristics,” in Proceedings of the 17th ACM Symposium on Virtual Reality Software and Technology (VRST '10), pp. 71–78, ACM, Hong Kong, November 2010.

T. Weise, S. Bouaziz, H. Li, and M. Pauly, “Kinect-based facial animation,” in Proceedings of the SIGGRAPH Asia Emerging Technologies (SA '11), Article No. 1, Hong Kong, December 2011.

T. Weise, S. Bouaziz, H. Li, and M. Pauly, “Realtime performance-based facial animation,” ACM Transactions on Graphics, vol. 30, no. 4, article 77, 2011.

S. Bouaziz, Y. Wang, and M. Pauly, “Online modeling for realtime facial animation,” ACM Transactions on Graphics, vol. 32, no. 4, article 40, 2013.

C. Cao, Y. Weng, S. Lin, and K. Zhou, “3D shape regression for real-time facial animation,” ACM Transactions on Graphics, vol. 32, no. 4, article 41, 2013.

C. Luo, J. Yu, C. Jiang, R. Li, and Z. Wang, “Real-time control of 3D facial animation,” in Proceedings of the IEEE International Conference on Multimedia and Expo (ICME '14), pp. 1–6, IEEE, Chengdu, China, July 2014.

C. Cao, Q. Hou, and K. Zhou, “Displaced dynamic expression regression for real-time facial tracking and animation,” ACM Transactions on Computer Systems, vol. 33, no. 4, article 43, 2014.

J. Zhao, C. Liu, Z. Wu et al., “3D facial similarity measure based on geodesic network and curvatures,” Mathematical Problems in Engineering, vol. 2014, Article ID 832837, 17 pages, 2014.

A. Mohr and M. Gleicher, “Deformation sensitive decimation,” Tech. Rep., University of Wisconsin, 2003.

S. Zhang and E. Wu, “Deforming Surface Simplification Based on Feature Preservation,” in Entertainment Computing—ICEC 2007, vol. 4740 of Lecture Notes in Computer Science, pp. 139–149, Springer, Berlin, Germany, 2007.

S.-K. Kim, S.-O. An, H.-T. Kim, and M. Hong, “Efficient modeling for animating human face,” in Proceedings of the IEEE International Symposium on Ubiquitous Multimedia Computing (UMC '08), pp. 35–38, IEEE, October 2008.

A.-B. Wang, B. Yu, and Z.-J. Liu, “Mesh simplification based on facial features region partition,” in Proceedings of the International Conference on Image Analysis and Signal Processing (IASP '09), pp. 269–272, IEEE, Taizhou, China, April 2009.

R. L. P. Duarte, A. El Rhalibi, C. Carter, S. Cooper, and M. Merabti, “An MPEG-4 quadric-based LoD simplification for facial animation,” in Proceedings of the International Conference on Multimedia Computing and Systems (ICMCS '12), pp. 743–748, Tangier, Morocco, May 2012.

M. Garland and P. S. Heckbert, “Surface simplification using quadric error metrics,” in Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques, pp. 209–216, Los Angeles, Calif, USA, August 1997.

B.-S. Jong, J.-L. Tseng, and W.-H. Yang, “An efficient and low-error mesh simplification method based on torsion detection,” The Visual Computer, vol. 22, no. 1, pp. 56–67, 2006.

J. L. Tseng, “Surface Simplification of 3D animation models using robust homogeneous coordinate transformation,” Journal of Applied Mathematics, vol. 2014, Article ID 189241, 14 pages, 2014.

M. D. Adams, Multiresolution Signal and Geometry Processing: Filter Banks, Wavelets, and Subdivision, University of Victoria, 2013.

A. Gray, E. Abbena, and S. Salamon, Modern Differential Geometry of Curves and Surfaces with Mathematica, Chapman & Hall, CRC Press, 2006.

J.-L. Tseng and Y.-H. Lin, “3D surface simplification based on extended shape operator,” WSEAS Transactions on Computers, vol. 12, no. 8, pp. 320–330, 2013.

F. Torkhani, K. Wang, and J.-M. Chassery, “A curvature-tensor-based perceptual quality metric for 3D triangular meshes,” Machine Graphics and Vision, vol. 23, no. 1-2, pp. 59–82, 2014.