Recognition of Augmented Frontal Face Images Using FFT-PCA/ SVD Algorithm

In spite of the differences in visual stimulus of human beings such as ageing, changing conditions of a person, and occlusion, recognition can even be done at a glance by the human eye many years after the previous encounter. It has been established that facial differences like the hairstyle changes, growing of one’s beard, wearing of glasses, and other forms of occlusions can hardly hinder the power of the human brain from making a face recognition. However, the same cannot easily be said about automated intelligent systems which have been developed to mimic the skill of the human brain to aid in recognition. )ere have been growing interests in developing a resilient and efficient recognition system mainly because of its numerous application areas (access control, entertainment/leisure, security system based on biometric data, and user-friendly human-machine interfaces). Although there have been numerous researches on face recognition under varying pose, illumination, expression, and image degradations, problems caused by occlusions are mostly ignored. )is study thus focuses on facial occlusions and proposes an enhancement mechanism through face image augmentation to improve the recognition of occluded face images. )is study assessed the performance of Principal Component Analysis with Singular Value Decomposition using Fast Fourier Transform (FFT-PCA/SVD) for preprocessing face recognition algorithm on face images with missingness and augmented face image database. It was found that the average recognition rates for the FFT-PCA/SVD algorithm were the same (90%) when face images with missingness and augmented face images were used as test images, respectively. )e statistical evaluation revealed that there exists a significant difference in the average recognition distances for the face images with missingness and augmented face images when FFT-PCA/SVD is used for recognition. Augmented face images tend to have a relatively lower average recognition distance when used as test images. )is finding is contrary to the equal performance assessment by the adopted numerical technique. )e MICE algorithm is therefore recommended as a suitable imputation mechanism for enhancing/improving the performance of the face recognition system.


Introduction
Face recognition technology in image processing is the process of identifying one or more people in images or videos by analysing and comparing patterns. In face recognition, an algorithm is used to extract facial features to be compared to a database with the aim of finding the best match. e computational model for face recognition developed by Turk and Pentland [1] created the present awareness in this field of research. According to Sharif et al. [2], viable technology has enhanced face recognition to gain a significant position in image processing but constraints such as ageing, facial expression, and occlusion have not been adequately addressed. Occlusion refers to the situation whereby extraneous objects hinder face recognition, and it represents missing information or holes in the facial images [3]. Cao et al. [4] posited that facial hair such as beard or moustaches and the usage of accessories like sunglasses or scarves can cause occlusion. Also, according to azheena and Aswathy Devi [5], a face can be occluded if part of its area is hidden by wearing objects like sunglasses, a mask, hats, or scarf in the eyes and mouth positions. ey classified occlusion as natural and synthetic. Natural occlusion refers to obstruction of views between objects in an image without any intention. On the other hand, an artificial blockade or intentional covering of an image view is synthetic occlusion.
Some of the different reasons people use such items are outlined in the following examples. Firstly, for safety reasons, medical staff wear surgical masks for protection when treating patients and people on construction sites wear helmet for protection against head injuries. Secondly, people wear veils because of their cultural practices or religious convictions.
irdly, bank robbers, automated teller machine (ATM) fraudsters, shop thieves, and football hooligans use items to cover up their faces when they want to commit illegal actions. Finally, sports men and women use helmets for rugby, swimming caps for swimming, etc. during sports competitions. e use of most of these items changes the appearance of the original face of the people and poses challenges to the recognition system due to distortion of face representations.
According to Gonzalez-Sosa et al. [6], the effect of occlusion is data missingness in a face matrix. is degrades the facial image and reduces the performance of the algorithm for recognition, hence the need to recover such portions to obtain the entire face.
Min et al. [7] proposed an efficient approach which consists of first detecting the presence of scarf/sunglasses and then processing the nonoccluded facial regions only. e occlusion detection problem was approached using Gabor wavelets, PCA, and support vector machines (SVM), while the recognition of the nonoccluded facial part is performed using block-based local binary patterns. eir experiments on AR face database showed that the proposed method yields significant performance improvements compared to some existing works for recognizing partially occluded and also nonoccluded faces. eir proposed method gave an average recognition rate of 94.58%, 92.08%, and 75.83% when used to recognize nonoccluded face, scarf, and sunglass occlusions, respectively.
Miyakoshi and Kato [8] proposed an imputation method using a Bayesian Network (BN) with weighted learning to impute missing values in face images.
eir system successfully classified test samples with missing values. e missing values were imputed by the proposed method, with better success than some conventional imputation methods (Normal BN, Weighted KNN, Mean, and Support Vector Regression). e weighted Bayesian Network gave an accuracy rate of 83.6%, 65.6%, 76.5%, and 58.5% when the test image had no occlusion, brows occlusion, eye occlusion, and mouth occlusion, respectively.
Wright et al. [9] developed robust face recognition via sparse representation. e method required a proper harnessing of sparsity for proper choice of features. Sufficiently, a large number of features were needed to obtain correct computations. e drawback in this method was to ascertain the number of features that would be sufficient for correct computation. A Sparse Representation Classification (SRC) was proposed from the theory of sparse representation to predict how much occlusion can be handled by a recognition algorithm in addition to the choice of training images so as to maximize robustness to occlusion.
Further investigations proved that the Collaborative Representation (CR) that makes the sparse representation classification powerful was ignored in the earlier works. Hence, a Collaborative Representation Classification (CRC) was combined with Regularized Least Square (RLS) to obtain a new classification method (CRC_RLS) [10]. Although the CRC_RLS was an improvement to the SRC, the Laplacian or Gaussian distributions restrict the coding coefficient and the coding errors statistically.
Asiedu et al. [11] leveraged on the property of bilateral symmetry for natural objects to reconstruct frontal face images from left and right segmented images. ey further evaluated the performance of FFT-PCA/SVD algorithm on the reconstructed face image database. e use of symmetry can have a limitation in a situation where the same area in both the right and the left parts of the face are occluded. ese constraints in the aforementioned method necessitate the need to explore other alternatives in face recognition system to recognize face images with occlusions. e goal is to enhance the recognition module to yield high precision. To this end, this study seeks enhancing the face recognition system by imputing occluded parts of the face (face augmentation).

Source of Data.
e study adopted the Massachusetts Institute of Technology (MIT) (2003)(2004)(2005) database to benchmark the face recognition algorithm. It is a secondary database which is made of ten subjects, with the subjects captured under different angular poses (0 ∘ , 4 ∘ , 8 ∘ , 12 ∘ , 16 ∘ , 24 ∘ , 28 ∘ , and 32 ∘ ). For this study, we concentrated on the face images captured under the straight pose (0 ∘ ). e trainimage database contains twenty frontal face images. Ten of these images were 0 ∘ straight pose from the MIT (2003)(2004)(2005) database. e images captured into the trainimage database are denoted as train images and are used to train the algorithm. Figure 1 shows images captured into the train image database.
Ten frontal images were acquired by creating random missingness (5%) in each image. ese were captured into the test image database 1 labeled "occluded face images." Figure 2 contains the images captured into the test image database 1.

Multivariate Imputation with Chain Equation (MICE).
One frequently used imputation method with a wide application for dealing with missing data in statistics is the MICE.
is method is also known as the sequential regression or Fully Conditional Specification (FCS) multiple imputation [12]. It is a very flexible method because it can handle different variable types such as discrete and continuous. Continuous variables are modeled through the method of linear regression whereas categorical variables are modeled through logistic regression.
It has three different phases which are similar to any other multiple imputation method: imputation, analysis, and pooling. It creates multiple imputations to overcome the limitation of single imputation. MICE uses Fully 2 Applied Computational Intelligence and Soft Computing Conditional Specification (FCS) to preserve unique features such as bounds, skip patterns, interactions, and bracketed responses in the data [12]. e MICE operation is based on the assumption of missing at random (MAR) with the implication that missing value probability does not depend on the unobserved values but only on the observed values [12]. MICE can handle large datasets through the use of chain equations as compared to other imputation methods that use joint models [13]. is makes it a powerful and flexible multiple imputation method that uses a number of regression algorithms.
In this study, ten frontal face images were acquired through augmentation of the images with missingness using the Multivariate Imputation with Chain Equations (MICE) algorithm. ese images were captured into the test image database 2 labeled "augmented images." Figure 3 contains the images capture into the test image database 2.
To keep the data uniform, captured images were digitized into gray-scale precision and resized into 200 × 200 dimensions and the data types changed into double precision for preprocessing.
is made the images (matrices) conformable and enhanced easy computations [11].

Research
Design. In the recognition system, the train images are preprocessed using the adopted mean centering and Fast Fourier Transform (FFT) mechanisms. e important features of the preprocessed face images are extracted using the PCA/SVD algorithm. e extracted features are stored in memory as a created knowledge for recognition.
is study considered two test image databases: frontal face images with random missingness (shown in Figure 2) and frontal face images acquired through augmentation of the images with missingness using MICE algorithm (shown in Figure 3). e test images are also preprocessed using the mean centering and Fast Fourier Transform (FFT) mechanisms and their unique features are extracted using PCA/  ese features are passed to the classifier where they are matched with the knowledge created from the train images for recognition. It is worthy of note that only one test image database is used in the recognition module along with the train image database at a time. Figure 4 shows a design of the recognition module.

Preprocessing.
Preprocessing is an initial stage of face recognition where the quality of the images is enhanced. According to Asiedu et al. [11], the preprocessing techniques are used to denoise the images making them better conditioned for recognition. Preprocessing of an image basically removes the acquired noise and suppresses unwanted distortion of image feature. is improves the quality of the image for feature extraction. Mean centering and Fast Fourier Transformation mechanisms were adopted for preprocessing. Details of the Fast Fourier Transformation and mean centering mechanisms used for the preprocessing of the face images are presented in Section 2.3.1 and Section 2.3.2, respectively.

Fast Fourier Transform.
Fast Fourier Transform (FFT) was adopted as a noise reduction mechanism. According to Glynn [14], the FFT algorithm reduces the computational burden to O(N log N) arithmetic operations. Zhang et al. [15] and Asiedu et al. [16] demonstrated that the application of FFT in the image preprocessing stage improves the recognition system. e DFT of a column vector, a jk , is represented mathematically as where s � 0, 1, . . . , p − 1 and j � 1, 2, . . . , n. a jk is the k th column of the image matrix, A j [11].
e Gaussian filter was adopted for filtering the face images after the Discrete Fourier transformation because of the Gaussian nature of illumination variations [11].
(2) e output after the inverse transformation is usually complex. e real components are extracted to be used at the feature extraction stage whereas the imaginary component is discarded as noise. Figure 5 shows the stages in FFT preprocessing of an image.

Mean Centering.
Given the image sample X � (X 1 , X 2 , . . . , X n ), whose elements are the vectorised form of the individual images in the study, the mean centering is performed by subtracting the mean image from the individual images under study. e mean is given by and the mean centering of the jth image is given by and W � (w 1 , w 2 , . . . , w n ) is the mean-centered matrix of the face space.

Feature Extraction.
e storage of face image requires the reduction of the dimensions of original images using feature extraction methods. is is due to the large nature of face image space. We adopt Principal Component Analysis (PCA) as a dimensionality reduction algorithm to extract the   Applied Computational Intelligence and Soft Computing 5 most significant components or those components which are more informative and less redundant, from the original data. PCA can be used to find lower dimensional subspace which identifies the axes with maximum variance [17]. As stated earlier, the FFT-PCA/SVD algorithm was used to train the image database to extract unique face features for recognition.
We now present the mathematical underpins of the face feature extraction mechanism, PCA/SVD, as described by Asiedu et al. [16].
A set of n orthonormal vectors, e j , which best describes the distribution of the data, is required for feature extraction. e t th vector e t is chosen such that is a maximum subject to the orthonormality constraints e vectors e t and scalars λ t are the eigenvectors and eigenvalues, respectively, of the variance-covariance matrix C given as rough Singular Value Decomposition (SVD) of the covariance matrix C, the eigenvalues and their corresponding eigenvectors are extracted. e SVD decomposition gives two orthogonal matrices U and V and a diagonal matrix Σ. e eigenfaces are then computed from the following equation: where u j is the j th column vector of U. e extracted principal components from the training set are given as and Γ T � [c 1 , c 2 , . . . , c n ]. ese are the created knowledge which is stored in memory for recognition.

Recognition Process.
We may recall that there are two test image databases: frontal face images with random missingness (shown in Figure 2) and frontal face images acquired through augmentation of the images with missingness using MICE (shown in Figure 3). When a test image is passed through the recognition module, its unique features are extracted as  Figure 6 presents the recognition distances and matches face image with missingness and augmented face images. It can be seen in Figure 6 that there was one mismatch when the face images with missingness were used as test images for recognition. Also, there was one mismatch when the augmented face images were used as test images for recognition.

Numerical Evaluations.
e average recognition rate was adopted as the numerical assessment criteria for assessing the performance of the algorithm on both databases. e average recognition rate, R avg , of an algorithm is defined as where t run is the number of experimental runs, n i cr is the number of correct recognition in the i th run, and n tot is the total number of faces being tested in each run [16]. e average error rate, E avg , is defined as 100 − R avg . e total number of correct recognition 10 i�1 n i cr for the study algorithm is 9. e total number of experimental runs t run � 10, and the total number of images in a single experimental run n tot � 2. Now, using the face images with missingness as test images, the average recognition rate of study algorithm (FFT-PCA/SVD) is e average error rate is Using the augmented face images as test images, the average recognition rate of study algorithm (FFT-PCA/ SVD) is e average error rate is

Statistical Evaluation.
e recognition distances were obtained from the same subjects in two different treatments (images with missingness and augmented images with MICE); hence, a paired sample test is suitable for statistical evaluation of the experiment. Let x j1 denote the recognition distance recorded using face images with missingness as test images and x j2 denote the recognition distance using the augmented images as test images for the j th individual; then the paired differences should reflect the differential effects of the treatments. Now assuming the difference X j , j � 1, 2, . . . , n, are independent observations from N(ψ, σ 2 d ) distribution, then the statistic where and has a t-distribution with n − 1 degrees of freedom. Consequently, an α-level test of the hypothesis H 0 : ψ � 0, against H 1 : ψ ≠ 0, is conducted by comparing |t| with t n−1 (α/2) and the upper 100(α/2) percentile of the t-distribution with n − 1 degree of freedom. Note that H 0 : ψ � 0 implies the Mean difference of recognition distances from face images with missingness and augmented face images is zero.
To make a decision as to whether or not to reject the H 0 , p value corresponding to the computed t statistic is compared to the level of significance. A 100(1 − α) confidence interval for the mean difference in recognition distance ψ � E(X j1 − X j2 ) is constructed as Some assumptions underpinning the paired sample ttest are that the observation must be paired and independent and the observed differences should be normally distributed.
Each observation of one treatment (recognition distances for face image with missingness) has a match (paired) with a corresponding observation in the other treatment (recognition distances for augmented face images). is satisfies the assumption of paired observations. e assumption of independence of the observations is attained since different subjects were captured in the study database. Table 1 shows the sample statistics of recognition distances for the subjects used in the study when face images with missingness and augmented faces images are used as test images.
It is evident from   is means the distribution of the observed difference is the same as the expected distribution (normal). Upon violation of the normality assumption, we would have resorted to the nonparametric counterpart (Related Samples Wilcoxon Signed Rank Test) of the paired sample test. e paired samples correlation between the recognition distance for the face image with missingness and augmented face images is 0.611 with a p value of 0.06. is indicates that there exists a moderate positive linear relationship between the recognition distance for the face images with missingness and recognition distance of the augmented face images. e p value of 0.06 indicates that this relationship is not significant at 5% level of significance. e results of the paired sample t test are shown in Table 2.
It can be seen from Table 2 that the average observed difference between the recognition distances for face images with missingness and recognition distances for augmented face images (MD and AD, respectively) is 68.5751. e test statistic value from the paired sample test is 3.979 with a corresponding p value of 0.003. It is evident from the p value that there is a significant difference between the average recognition distance for face images with missingness and recognition distances for augmented face images. It can be inferred that the average recognition distance for face images with missingness is significantly higher (at 5% level of significance) than the average recognition distance for augmented face images.

Conclusion and Recommendation
e FFT-PCA/SVD algorithm produced average recognition rates of 90% each when face images with missingness and augmented face images were used as test images, respectively. It is evident from the results of the numerical assessment that the recognition algorithm has equal performance when face images with missingness and augmented face images are used as test images. e numerical results (average recognition rate of 90%) of this study were appreciable and consistent with those of Min et al. [7] and Miyakoshi and Kato [8] despite the differences in occlusion criteria (random occlusions; brows, eye, and mouth occlusions; scarf and sunglass occlusions) and the database used to benchmark the recognition/classification systems. e statistical assessment revealed that there is a significant difference between the average recognition distance for face images with missingness and recognition distances for augmented face images. e average recognition distance for face images with missingness is significantly higher than the average recognition distances for augmented face images. is means the enhancement of the face recognition module through augmentation with MICE algorithm is a viable improvement as it provides relatively lower recognition distances. A relatively lower recognition distance is desired as it signifies close match. e numerical assessment mechanism failed to reveal the hidden effect of the enhancement with MICE algorithm whereas the statistical evaluation procedure was able to uncover this effect. is can be attributed to the fact that the statistical evaluation mechanism is a more data-driven approach to assess the performance of the recognition algorithm.
is may also be as a result of the low level of missingness (5%) created. e superiority MICE augmentation mechanism may be demonstrated better in subsequent studies by increasing the level of missingness created in the face images.
From the findings of the study, the MICE algorithm is therefore recommended as a suitable imputation mechanism for enhancing/improving the performance of recognition modules/systems when used to recognize occluded face images.
Data Availability e PGM image data supporting this research are from previously reported studies and datasets, which have been cited. e processed data are available from the corresponding author upon request.

Conflicts of Interest
e authors declare that there are no conflicts of interest.