Facial makeup significantly changes the perceived appearance of the face and reduces the accuracy of face recognition. To adapt to the application of smart cities, in this study, we introduce a novel joint subspace and low-rank coding method for makeup face recognition. To exploit more discriminative information of face images, we use the feature projection technology to find proper subspace and learn a discriminative dictionary in such subspace. In addition, we use a low-rank constraint in the dictionary learning. Then, we design a joint learning framework and use the iterative optimization strategy to obtain all parameters simultaneously. Experiments on real-world dataset achieve good performance and demonstrate the validity of the proposed method.
Digital technology represented by artificial intelligence, Internet of things (IoT), and cloud computing, etc. is developing vigorously for smart cities. A smart city aims at using various kinds of information technology to integrate the system and services of the city, which improves the utilization efficiency of resources and the quality of life of residents [
Due to differences in illumination variations, face angle, posture, and cameras, the face images belonging to the same person may look very different. Particularly, in real-world applications, facial makeup significantly changes the perceived appearance of the face and reduces the accuracy of face recognition. The literatures [
Example of face image pairs: left one is without makeup and right one is with makeup.
Recently, dictionary learning has achieved great success in the field of face recognition. Traditional dictionary learning learns sparse representation and dictionary in the original data space. However, face makeup image verification is not only affected by cosmetics, but also easily affected by illumination and posture. In this study, we develop a joint subspace and low-rank coding method for makeup face recognition (JSLC). We consider finding a feature projection space and project the face images into it. At the same time, we learn a discriminative dictionary in such feature subspace, and each face image is encoded by a discriminative coding. To solve the problem of subspace and dictionary simultaneously, we build a joint learning model for them. In addition, to obtain more discriminative information in the subspace, we consider a low-rank constraint in the dictionary learning. The optimal solution of subspace projection matrix, dictionary, and sparse coefficient can be obtained simultaneously by alternating iterative optimization strategy.
We organize the rest of this paper as follows. Firstly, related work about makeup face recognition is reviewed in Section
In the view of AI, the makeup face recognition contains two stages: feature extraction and classification method. The common used feature extraction methods for face recognition is geometric methods and appearance methods [
Dictionary learning methods can approximate each sample by using a linear combination of a few atoms from the learned dictionary [
The original meaning of equation (
Because the appearance of the person face will change significantly after makeup, in this study we use subspace learning to project the original data samples and preserve the discriminative information in the feature subspace. The subspace learning imbedded into dictionary learning can be represented as
Then, we consider using an affinity matrix
The element
We denote diagonal matrix
In order to obtain more discriminative information in the subspace, we consider a low-rank constraint of
We combine
Obviously equation (
In this subsection, we solve equation ( Update Equation ( where We can obtain Update We use Lagrange dual approach to solve equation ( where Update Obviously, each term in equation ( Equation ( Update
When we obtain the optimal parameters of dictionary
Finally, we can use the closing distance strategy to perform the testing task.
Based on the above analysis, the proposed JSLC method is presented in Algorithm
Input: a dataset of facial images Output: dictionary Initialization: random matrix Repeat Update Update Update Update if converged
In the experiment, we use the widely used face datasets DFW [
Example face images of DWF dataset.
To validate the effectiveness of our approach, our method verified performance with the following methods: LLC [
Table
Performance comparisons of all methods using HOG features.
Methods | Rank 1 | Rank 5 | Rank 10 | Rank 15 |
---|---|---|---|---|
LLC | 59.10 | 73.20 | 80.26 | 85.81 |
LMNN | 63.24 | 77.83 | 85.00 | 91.02 |
PRDC | 63.68 | 80.55 | 85.62 | 92.74 |
NCA | 65.35 | 82.01 | 89.23 | 93.45 |
RDML-CCPVL | 66.68 | 84.00 | 89.74 | 96.51 |
JSLC |
The bold values mean the best performance values.
Performance comparisons of all methods using LBP features.
Methods | Rank 1 | Rank 5 | Rank 10 | Rank 15 |
---|---|---|---|---|
LLC | 59.82 | 74.05 | 80.46 | 85.99 |
LMNN | 64.08 | 78.03 | 85.39 | 91.45 |
PRDC | 64.13 | 80.97 | 86.11 | 93.02 |
NCA | 65.72 | 82.38 | 89.73 | 94.01 |
RDML-CCPVL | 67.17 | 84.22 | 90.16 | 96.99 |
JSLC |
Performance comparisons of all methods using TPLBP features.
Methods | Rank 1 | Rank 5 | Rank 10 | Rank 15 |
---|---|---|---|---|
LLC | 60.10 | 74.81 | 80.93 | 86.26 |
LMNN | 64.42 | 78.83 | 85.79 | 91.91 |
PRDC | 64.09 | 81.12 | 86.68 | 93.54 |
NCA | 65.61 | 82.71 | 89.97 | 94.55 |
RDML-CCPVL | 67.22 | 84.79 | 90.54 | 97.38 |
JSLC |
Figures
Performance of JSLC with different subspace dimensions.
Performance of JSLC with different number of dictionary atoms.
Performance of JSLC using different parameters. (a)
First, we discuss the effect of
In this study, a joint subspace and low-rank coding method is proposed for makeup face recognition. Based on the dictionary learning framework, the subspace learning and low-rank coding is jointly, so that the discriminative information of face images can be exploited. Experiment results on DFW show the good performance of our method. In the future, we will carry out face makeup recognition and verification in more complex datasets and more scenes, such as under various illumination, pose, and expression. How to extract deep features of face images into our method is also our work in the next step.
The labeled datasets used to support the findings of this study are available from the corresponding author upon request.
The authors declare that there are no conflicts of interest regarding the publication of this paper.
The research activities described in this paper have been conducted within the Qinglan Project of Jiangsu Province under Grant no. Q019001, the Scientific Research Project of Changzhou Institute of Technology under Grant no. YB201813101005, Youth Innovation Fund Project of Changzhou Institute of Technology under Grant nos. QN202013101002 and HKKJ2020-37, National Natural Science Foundation of China under Grant no. 61806026, Natural Science Foundation of Jiangsu Province under Grant no. BK20180956, and Project of Jiangsu Education Science in the 13th Five-Year Plan in 2018 under Grant no. B-a/2018/01/41.