The Impact of Asymmetric Left and Asymmetric Right Face Images on Accurate Age Estimation

1Department of Electrical Engineering, Mirpur University of Science and Technology, Mirpur 10250, Pakistan 2Department of Software Engineering, Mirpur University of Science and Technology, MUST, Mirpur 10250, Pakistan 3Department of Computer Science, Government College University Faisalabad, Pakistan 4School of Computer Science and Engineering, Korea University of Technology and Education, 1600 Chungjeolno, Byeogchunmyun, 31253 Cheonan, Republic of Korea


Introduction
One can glean multiple information from human face including identity, age, gender, and race.With the passage of time, the facial appearance changes with increasing age.The rate of change of facial aging variations is different for different population [1].One can infer the facial age directly using distinct appearance patterns such as wrinkles, moles, and scars.However, facial appearance is closely related to the physical changes that arise with the aging process.After the growth termination around 20 years of age, face shape stays to change, predominantly in late adulthood [2].Facial asymmetry is one of such changes which increases with age and affects the facial appearance differently for both sides of face [3,4].Facial asymmetry causes dramatic changes with age, including asymmetric appearance of facial landmarks, wrinkles, and sagging.Since the perception of age depends on facial appearance, thus left and right half face images of a given face image show different aging patterns and perceived or estimated age.In this paper we focus on this important yet previously neglected topic of facial age estimation using left and right asymmetric faces.
Previously, facial asymmetry was used in recognizing face images, estimating age group, and demographic estimation effectively.Quantification of facial asymmetry has been for expression invariant human identification in [5].Asymmetric half face images have been used for face recognition in [6].Facial asymmetry based features in conjunction with local and global features were used in [7] for face recognition.Bilateral asymmetric measurements were used in [8] to estimate the accurate age group.Face images are subsequently recognized based on age group estimation.Deeply learned asymmetric face features have been used to estimate the facial demographic information.Facial asymmetry based bilateral measurements were used in [9] to train a regression model to predict the most discriminative dimensions among the subjects belonging to different populations.Previously, Li et al. [10] employed asymmetric left and asymmetric right face images for twin identification task.
Different from above presented methods, we analyze the impact of using asymmetric left and asymmetric right face images on age estimation.Being an individual's facial characteristic, facial asymmetry differs in perceptible ways even between two sides of a face image.Figure 1 illustrates the inherent intrinsic facial asymmetry present in face images of two distinct individual subjects.For each subject, the original face is shown in the left most image of a row while the central image shows the asymmetric left face image which is obtained by horizontal concatenation of left half and its mirror image.Similarly the right most image in each row shows the asymmetric right face image obtained by the horizontal concatenation of right face image and its mirror counterpart.From Figure 1, it is evident that the facial asymmetry exhibited by the left and right half face images is not the same; i.e., the left asymmetric image is different from the right asymmetric image and these both are different from original image as well.This difference is also manifested when original face image is compared with left or right asymmetric face images.Thus, the use of asymmetry based features to estimate the age of an identical subject is possible and is the basis for the age estimation approach we are going to present in this work.
In this study, we try to answer the following questions: (i) Do asymmetric left or asymmetric right facial halves age differently?(ii) Which facial half is affected more by asymmetry based aging variations?(iii) What is the effect of asymmetric left or asymmetric right half face on age estimation?
Based on the existing methods to facial age estimation, this study offers the following contributions: (i) To the best of our knowledge, this is the first study of its kind considering the asymmetric left or right face images for facial age estimation on two challenging databases.(ii) A detailed evaluation of the performance of a deep CNN architecture in estimating facial age using the original, left asymmetric, right asymmetric face images.(iii) Per decade analysis of age estimation using asymmetric left or right face images.
The structure of this paper is as follows.Existing approaches to age estimation are reviewed in the next Section.Materials and procedure are described in Section 3. Experimental results are presented out in Section 4. Results related discussion is offered in Section 5, whereas the conclusions are presented in the last Section.

Literature Review
There has been a renowned interest in accurate age estimation owing to its significance in numerous applications such as face recognition, deduplication of identity cards, locating missing persons, and controlling pedophilia.Since the inaccurate age estimation in human-computer interaction could lead to undesired results, such as illegal access to the Internet by certain age groups, accurate age estimation is an important yet challenging research problem owing to imbalanced aging effects on left and right half of a given face image.Before the success of deep learning methods, hand-engineered features were used for facial age estimation of a query face image.Most of these methods extracted visual features which were then applied to appropriate classifiers for age estimation.Biologically inspired features (BIF) were used in [11] for age estimation.Variants of BIF have been consistently used in age estimation, such as [12,13].Neighborhood based local features have been used to estimate the facial age in [14].Local Binary Patterns (LBP) based features in conjunction with principle components of BIF, active appearance model (AAM), and principle components analysis (PCA) have been used for age estimation in [15].Most recently, Liu et al. [16] employed global and local features for fast genre classification of web images.
After the success of deep learning in numerous computer vision and artificial intelligence applications, numerous deep learning based age estimation approaches have been presented in the existing works.Huerta et al. [17] presented a deep analysis on age estimation using CNNs.Hu et al. [18] proposed a learning scheme using weakly labelled data in conjunction with CNNs.Wang et al. [19] employed deep models to learn discriminative features for age estimation.Recently, CNNs have been used for license plate recognition in [20].
Different from the above presented works, in this paper we propose to evaluate the effectiveness of asymmetric left and asymmetric right face images compared to the original face images in age estimation using a deep CNN model.

Data Collection.
To study the impact of asymmetric left and asymmetric right face images on age estimation, we require labelled face images.Concerning the age estimation, the most commonly used datasets are the MORPH [21] and FERET [22].Both the MORPH and FERET are publicly available longitudinal databases containing the age-separated face images with gender and ethnicity ground truth information.Table 1 summarizes the key indicators of these two databases.To build our dataset, we collected 10,000 full frontal face images of 5,000 male and 5,000 female subjects from the MORPH dataset [21].Similarly 1,000 face images are collected from FERET dataset belonging to 578 male and 422 female subjects.

Preprocessing.
The preprocessing involves the normalization of extrinsic variations and determination of left asymmetric and right asymmetric face images for each original face image as described in the following two steps.
(1) Normalization of extrinsic variations: all the face images are eye-aligned and corrected for unwanted    example, nose) in asymmetric right or asymmetric left face image will lead to increased or decreased estimated age of a person [23].This is because the size of facial parts is visual cues for apparent age perception [24].

Experimental Protocol and Evaluation.
We conducted this series of experiments to test the hypothesis that estimated age of asymmetric left and asymmetric right faces is different from the original face image.For this purpose, we use an age estimator based on a deep architecture called the GoogLeNet [25].The network is selected due to its best performance for the age estimation task as reported in the previous studies [18].The network is more complex and deep than previous dCNNs.It has two convolution, two pooling, and nine inception layers.An inception layer concatenates filters of different sizes and dimensions into a single new filter.More precisely, GoogLeNet has a directed acyclic graph structure.Unlike its peer networks, the inception module in GoogLeNet allows performing convolution or pooling operations in parallel.A total of 9 inception modules are used resulting in very deep network architecture.
Overall, the flexible layered structure results into improved performance and computational efficiency.Figure 3(a) shows the architecture of GoogLeNet while Figure 3(b) shows the corresponding inception-3a module.
We train the deep network as an age estimator using original face images selected from MORPH database.The trained model should provide estimated ages for left asymmetric and right asymmetric face images in the testing stage.
Following the works of Geng et al. [26,27] and Hu et al. [18], we define the age estimation loss function in terms of label distribution.The choice of label distribution is motivated by the fact that human aging is a slow and continuous process.At closer ages the similarity between face images of a particular person is quite high and vice versa.Considering a Gaussian distribution to model the label distribution of age the loss for a given face image f is defined as where   ∈ RR  , for  number of ground truth ages, and    is the probability that image  belongs to age .
To train the deep network, the parameters are optimized using the backward propagation [28] as gradient computing methodology in conjunction with stochastic gradient decent (SGD).Recall that SGD is a commonly used iterative method employed in deep CNNs to optimize the network parameters to minimize the loss function.In our experiments on the MORPH dataset, the number of neurons in the softmax layer is equal to number of ground truth ages .The deep CNN model is trained using the original face images while tested on the left asymmetric and right asymmetric face images.In our age estimation experiments, the preliminary learning rate is set to 0.001 and subsequently decreased by a factor of ten after every 50 epochs.This is because, originally the learning rate is set to a high value and then it is decreased in appropriate steps once the loss function starts decreasing until it reaches a global loss minimum.Weight decay is set to 0.0001, which prevents the undesired growth of the weights.
For performance evaluation of the deep CNN, we use mean absolute error (MAE) and cumulative score (CS) as appraisal metrics used in the previous related studies [18,29].Each of these evaluation metrics is described below.
(1) MAE: the MAE is defined in terms of the absolute error between estimated age for  number of testing images and ground truth age as follows: where   and   represent the ground truth labels and estimated age of ℎ face image.(2) CS: cumulative score defines the percentage of correctly estimated ages with respect to absolute error and is expressed as shown in where  ≤ represents the number of testing images for which the error e does not exceed a given number of years .

Experiment 1.
In the first series of experiments, we test the age estimation accuracy of the original face images using the trained deep CNN.We use 70% images in training and 30% in testing.To ensure balance between training and test accuracies, fivefold cross-validation approach is used.Following this setting, there are 24,000 face images in training and 6,000 in testing for the MORPH dataset.For the same experimental protocol, we use 1,000 images in in testing for FERET dataset.Table 2 displays the comparative MAE results on the selected datasets for this series of experiments.The CS

Discussion
(i) From the comparative results presented in Table 2, it is observed that performance of deep CNN in estimating age of real face images is better than that for left symmetric face images but less than the asymmetric right face images.
To explicitly disclose the performance of age estimator within a particular age error range, we also report the cumulative scores (CS).From Figures 4(a) and 4(b), it is obvious that, for the MORPH and FERET sets, the age estimation accuracy obtained by deep CNN architecture on original face images is better than the asymmetric left face images, whereas it is less than the asymmetric right face images.This is because original face images contain the asymmetric age features of both the left and right half face images, where left parts of face images are more affected by the aging variations compared to the right sided facial parts which are less affected to the aging variations.From Figures 4(a) and 4(b), it is obvious that age estimation accuracy for right asymmetric face images is better than the asymmetric left face images.This is because MAE of left asymmetric images is higher than the right asymmetric images compared to the original face images as given in Table 2.For example, in case of the MORPH dataset, the MAE of the left asymmetric and right asymmetric images is 23.18% and 6.88%, respectively.This suggests better performance of the right asymmetric face images than the right asymmetric images in estimating the apparent age of a given face image.The better age estimation performance of the right asymmetric face image can be attributed to the fact that this facial half is not effected by aging process inexplicably a suggested in [3].
(ii) In terms of MAE, the deep CNN achieves the best score for asymmetric right face images.For MORPH dataset, MAE of 2.57 years is achieved, whereas MAE of 2.60 years is achieved for FERET dataset.Similarly, the highest age estimation accuracy is achieved in terms of CS by using the right asymmetric face images, both for the MORPH and FERET test sets.The better accuracy of age estimation using right asymmetric face images can be attributed to the lesser effect of facial asymmetry on right sides of face images as suggested by Ercan et al. [3].Moreover, the right sides of face images are smaller in both male and female subjects as verified in [3].Thus concatenating the right half face image and its mirror image horizontally will result into right asymmetric face image with relatively smaller widths of different facial parts resulting in lesser estimated age.Generally, the estimated age of face photos with smaller facial parts is lesser.For example, the nose width in childhood is less than that of adulthood resulting in lesser perceived age of a given face image.
(iii) Age estimation using left asymmetric face images resulted in MAE of 3.40 years and 3.43 years for MORPH and FERET test sets.Similarly, the CS in estimating age using deep CNN architecture is the lowest for asymmetric face images compared to the original and right asymmetric face images.This is because of two main reasons: first, the ratio of asymmetric distances for left side of face is quite high compared to that of right half face, and second, generally the left half face is larger than the right half face as suggested in [3] In Figure 5(c), we observe that change in MAE per decade of increasing age is lower for asymmetric right face images compared to the corresponding changes for original and asymmetric left face images.This is because right half of face is comparatively less affected by facial asymmetry with increasing age as suggested in [3].
(v) Ercan et al. [3] proposed that there is a correlation between the level of facial asymmetry and facial side.Anson et al. [31] explored that facial wrinkles are developed by the compression during sleep which also contribute to facial skin expansion.The experimental results presented in our study support these notions, by showing that bilateral facial asymmetry based asymmetric right face images can be used for age estimation of a given face image with reduced MAE compared to its counterparts i.e., the original and left asymmetric face images.This suggests the fact that there is an increase in the apparent age due to bilateral asymmetry of faces as they get aged chronologically.A larger aging effect of asymmetry on chronological age is observed when asymmetric left face images are used in age estimating task.The larger aging effect of asymmetric left face is also obvious in original face images a depicted by the estimation results shown in Table 2.

Conclusion
We have investigated the role of asymmetric left and asymmetric right face images in accurate age estimation.It has been observed that that facial asymmetric aging of two facial halves is associated with estimated age.This study suggests that right asymmetric face images are less inexplicably affected by aging variations compared to the left asymmetric images.This suggests that right asymmetric face images should be used in accurate age estimation.Furthermore, faces with lower facial asymmetry look younger than faces with higher facial asymmetry, and the younger face can be used to estimate the age of a given face image with lower MAE compared to corresponding asymmetric left or the original face image.Thus, varying facial asymmetry for the left and right half faces can be added to the list of known signs of aging.Future studies can utilize this aging cue to study the impact of asymmetric face images on gender or race estimation task.

Figure 1 :
Figure 1: Original face images with corresponding left asymmetric and right asymmetric face images for two subjects from the MORPH dataset.

Figure 5 :
Figure 5: Variation of MAE of age estimation per decade of age progression on (a) original, (b) left asymmetric, and (c) right asymmetric face images on the MORPH and FERET datasets, respectively.

Table 1 :
The key indicators of the MORPH and FERET databases.
metric face image is obtained by horizontal concatenation of left half face and its mirror counterpart.Similarly, the right asymmetric face image is obtained by horizontal concatenation of right half face and its mirror image.The proposed preprocessing pipeline is illustrated in Figure2.asymmetric left and asymmetric right face images of the same subject may be different.Transplanting the corresponding mirror images into left and right half faces will lead to generation of younger and older face images of the same individual.Since facial asymmetry affects both sides of face differently, therefore smaller or larger facial parts (for

Table 2 :
MAE for MORPH and FERET test sets using original face images.
Similar to the previous experiments, right asymmetric face images of the subjects used across the 5-fold are used in testing.The comparison results of MAE on the MORPH and FERET test sets are reported in Table 2.The CS curves for right asymmetric face images are shown in Figures 4(a) and 4(b) for the MORPH and FERET test sets, respectively.
Table 2 lists the age estimation results in terms of MAE which reflects the overall performance of deep CNN model on original, left asymmetric and right asymmetric face image test sets on the MORPH and FERET.Using original face images, MAE of 2.76 years is achieved on the MORPH test set.Similarly, MAE of 2.78 years is achieved for original face images from FERET test set.It is worthwhile to note that in all of our experiments on age estimation, chronological age (i.e., the real age) has been used as ground truth.
[30] a consequence, horizontal concatenation of the left half face and its mirror image results in comparatively larger facial components.The larger face components in turn results in increased perceived age of a given face image.This observation is mainly in line with the trends reported by Porcheron et al.[30]for facial contrast and perceived age.(iv) Since facial asymmetry increases with age, we also analyzed the impact of increasing age on age estimation accuracy in terms of change in MAE per decade of increasing age.To this end, we trace MAE against seven age groups ranging from 10 to 70 years in the steps of 10 years in Figures 5(a)-5(c) for the original, left and right asymmetric face images on the MORPH and FERET datasets, respectively.From Figure 5(a), it is obvious that the largest change in MAE occurs for the age groups 20-30 and 30-40 years.This is because the facial growth terminates around 20 years of age, and afterwards facial shape continues to change, predominantly in later age groups representing the adulthood and the old age as observed by Farkas et al. [2]. Figure 5(b) shows that rate of change of MAE with respect to decades of increasing age is higher for asymmetric left face images compared to the original and right asymmetric face images.This can be attributed to the prominent facial asymmetry for the left part of face as reported by Ercan et al. [3].