Feature-Level vs. Score-Level Fusion in the Human Identification System

. The design of a robust human identiﬁcation system is in high demand in most modern applications such as internet banking and security, where the multifeature biometric system, also called feature fusion biometric system, is one of the common solutions that increases the system reliability and improves recognition accuracy. This paper implements a comprehensive comparison between two fusion methods, named the feature-level fusion and score-level fusion, to determine which method highly improves the overall system performance. The comparison takes into consideration the image quality for the six combination datasets as well as the type of the applied feature extraction method. The four feature extraction methods, local binary pattern (LBP), gray-level co-occurrence matrix (GLCM), principle component analysis (PCA), and Fourier descriptors (FDs), are applied separately to generate the face-iris machine vector dataset. The experimental results highlighted that the recognition accuracy has been signiﬁcantly improved when the texture descriptor method, such as LBP, or the statistical method, such as PCA, is utilized with the score-level rather than feature-level fusion for all combination datasets. The maximum recognition accuracy is obtained at 97.53% with LBP and score-level fusion where the Euclidean distance (ED) is considered to measure the maximum accuracy rate at the minimum equal error rate (EER) value.


Introduction
Due to the digital revolution in the last decades, the high and accelerating demands on automated systems motivate developers, including biometric system designers, to put plans and thoughts towards improving the recognition accuracy and enhancing the system performance. So, the robustness of the biometric system became the main and vital aspect for many developers. One of the most reliable approaches is employing multifeature biometric system rather than unimodal systems; therefore the demand for more biometric traits like a fingerprint, face, iris, palm print, retina, hand geometry, voice, signature, and gait has been raised up. is paper aims to build a reliable biometric system by fusing the face and iris traits into one multimodal system. Meanwhile, the recognition results for the two fusion methods have been compared extensively to determine the best method that satisfies the highest accuracy at minimum EER value. e two fusion techniques have been run by six combination datasets and four feature extraction methods, LBP, GLCM, PCA, and FDs. Each combinational dataset contains 40 subjects, so the overall test subjects are 240. e diversity in applying multifeature extraction methods and databases will support our deep investigation in determining the maximum system accuracy to the corresponding fusion method, feature extraction method, and face-iris combination datasets.
Ijar and Nisha [1] presented a model for recognizing facial expression based on the support vector machine (SVM) algorithm.
e Univalue Segment Assimilating Nucleus (USAN) approach was utilized as a feature extraction after resizing the image into 64 × 64 pixels, where 125 images were used for detecting the face region. e recognition accuracy was 90% with a 16% low error rate. Hamd and Mohammed [2] applied LBP, GLCM, FDs, and PCA feature extraction methods with feature-level fusion technique to improve the recognition accuracy of human identification system using face-iris traits. ree databases and two combinational datasets were utilized for testing the work. e maximum accuracy rate was measured using the Euclidean distance measurement; it was 100% using LBP and GLCM method, while the PCA and FDs method achieved only 97.5%.
Sharifi and Eskandari [3] applied three fusion methods: score level, feature level, and decision level to fuse the face and iris traits using the Log-Gabor method; moreover, backtracking search algorithm (BSA) was employed to improve the recognition accuracy by electing the optimized weights for feature-level and score-level fusion. e verification rates showed a valuable improvement of the three fusion techniques over multimodal and unimodal systems. Zou et al. [4] applied a fusion method on the extracted local and global features for the scene classification approach using a collaborative-based classification. A fusion of LBP and Gabor filters satisfied significant results as the experimental results have approved. Luo et al. [5] presented a fusion of multifocus images for integrating the partially focused images into only one. e singular values of the high-order decomposition and edge intensity were both applied to implement the proposal.
Hamd [6] applied three neural network classifiers and Euclidian distance to optimize the iris feature vector with respect to maximum accuracy. e maximum accuracy correspondence to the optimum machine vector was satisfied with probabilistic and backpropagation rather than radial basis function and Euclidean distance measurements. Dwivedi and Dey [7] implemented an evaluation modal using three virtual databases. e integration of Dempster-Shafer (D-S) theory based on the decision-level fusion, cancellable modalities for iris-fingerprint features, and score-level fusion was proposed in this work. e meanclosure weighting (MCW) score level and D-S theory were both applied to minimize the boundaries in the individual score techniques. e test results of the proposed hybridfusion framework for three traits were very satisfied and robust than individual fusion methods (score level, feature level, and decision level).
Gunasekaran et al. [8] involved the local-derivative ternary pattern and contourlet transform for extracting the high variation coefficients. e multimodal fusion of irisface and fingerprint satisfied significant improvements in the accuracy rate. Islam [9] developed a new combined approach of feature fusion and score fusion based on the multibiometric systems. Features from left and right iris images were fused, and a log-likelihood ratio based scorelevel fusion was applied to find the score of iris recognition. e output of multiple classifier selection (MCS) such as the individual modalities of each iris (left and right), feature fusion based modality, and log-likelihood ratio based modality was combined. e discrete hidden Markov model (DHMM) was utilized as a classifier in the modal, and the PCA was applied as an iris feature reduction approach. Each iris vector has 9600 feature values, where only 550 values have been considered. e proposed system performed a 90% enhancement to the accuracy rate over the existing iris recognition approaches under natural lighting conditions at various noise levels. e D-S theory was applied for fusion at score level by Nguyen et al. [10]. e motivation for applying D-S theory to multibiometric fusion is to take advantage of the uncertainty concept in D-S theory to deal with uncertainty factors that impact biometric accuracy and reliability in less constrained biometric systems. A unified framework was developed to improve the recognition accuracy, where the combination of classifiers performance and quality measures for the input data was weighted and then fused at score level after selecting the significant factors that generate an overall quality score. e proposed approach is robust against image quality variation and classifier accuracy; furthermore, it enabled the multimodal biometric to operate in less constrained conditions. Typically, three masses were considered when evaluating a biometric match: the mass of a genuine enrolment attempt, the mass of an impostor enrolment attempt, and the mass of the uncertainty (either genuine or impostor) of the acquired data and/or classifier. e multimodal biometric fusion of D-S approaches has verified promising performance after appropriately modeling the uncertainty factors. e N classifier accuracy values are also supplied in the form of an EER to strengthen the fusion. ese N trios were combined to output a fused score and a fused quality score. When only high-quality data was accepted, the proposed approach achieved a competitive performance (at or close to 100% and with 1% EER) compared with other conventional fusion approaches. e bio-secure DS2 benchmark database [11] was considered to conduct the tests. It consists of 3 face channels, 6 optical and 6 thermal fingerprint channels, and 2 iris channels (not applied). ose 17 channels contain the data match scores and quality measures information. High quality of face and optical fingerprint achieved high accuracy than low quality face and thermal fingerprint. where the EERs varied from 0.08 to 0.09 for HR face optical fingerprint and to 0.33 for thermal fingerprint. e work did not make comparisons with Bayesian fusion approaches, and its results were only compared with other techniques that have similar data requirements. It also differs from our proposal in introducing the uncertain factors in computing the accuracy that utilized biometric traits other than the iris feature in score-level fusion approach.
An investigation was made by [12] to compare the performance of three multimodal approaches for recognizing a fusion of iris and fingerprint traits using sum rule, weighted sum rule, and fuzzy logic method. A multimodal biometric system of combined iris-fingerprint features was implemented and compared with the proposed unimodal systems. e scores of both traits were fused together in two phases: the matching score phase and the decision level phase, where the iris trait has more weight in fusion than a fingerprint. e small weight value was added to the matching distance using the fuzzy membership function to mimic human thinking and providing enhanced results. e experimental results showed that the fuzzy logic method introduced by Zadeh for fusion at the decision level is the best followed by the weighted sum rule and the classical sum rule in order.

2
Applied Computational Intelligence and Soft Computing e accuracy rates, error rates, and matching time were reported as three items for performance evaluation at zero false acceptance rate (FAR) to the corresponding false rejection rate (FRR). e iris and fingerprint features were combined from an equivalent number of CASIA-Iris (V1 and V2) and fingerprints FVC2004 database, and as assumed by this work, the fusion of fingerprint and iris is more reliable than other biometric like face [13]. Hereby, it differs from our proposal by considering the iris-fingerprint features rather than face-iris combinational features. Furthermore, the proposed multibiometric system design is more complicated as it applied three different matching algorithms, and it converts the iris and fingerprint scores to fuzzy sets (fuzzification), which means the fuzzy inference system produces bad recognition, very bad, medium, good, very good, or excellent state. e experimental outcomes satisfied the best compromise between FAR and FRR (0% FAR and 0.05% FRR) with a 99.975 accuracy rate and 0.038 EER value with matching time equal to 0.1754 sec. Gender classification using a modified LBP was proposed by [14] to handle disadvantages of basic LBP with a new theory of nonlinear gender classification utilizing Tani-Moto metric as distance measure. e compared results with some state-of the-art algorithms showed the high quality of the proposed approach in terms of accuracy rate. Hammad et al. [15,16] integrated information from two biometric modalities using convolution neural network (CNN) to improve the performance and make a robust system. Moreover, Q-Gaussian multisupport vector machine (QG-MSVM) was proposed with decision-level fusion and feature-level fusion for [15] and score-level fusion for [16] to complete the classification stage and achieve high accuracy. e proposed systems were tested on several databases for electrocardiogram (ECG) and fingerprint like PTB and LivDet2015 to show their efficiency, robustness, and reliability in contrast to existing multimodal authentication systems.
e proposed system can be deployed in real applications according to their advantages. e performance of different classification methods and fusion rules was studied by El_Rahman [17] in the context of multimodal and unimodal biometric systems utilizing the MIT-BIH for electrocardiogram (ECG) database and FVC2004 for fingerprint databases with 47 subjects from virtual multimodal database. e performance of the proposed unimodal and multimodal systems is measured using receiver operating characteristic (ROC) curve and area under the ROC curve (AUC). e experimental results indicated that AUC is 0.985 for sequential multimodal system and 0.956 for parallel multimodal system in contrast to the unimodal systems that achieved only 0.951 and 0.866 for the ECG and fingerprint databases, respectively. e proposed work has concluded that the overall performance of the multimodal systems is better than that of the unimodal systems tested on different classifiers and different fusion rules.
Consequently, few listed papers worked on the face-iris based multimodal human identification system which applies four different feature extraction methods besides two comparative fusion methods. Moreover, our proposal is carried out on six combinational datasets: three for face database and two for iris database. e quality of the database varies from good to low, where the low quality facial/ iris images will add more challenges to the system performance as the blur, low illumination, different poses, and partial occlusion will eventually affect the feature extraction result and recognition accuracy outcomes.

The Proposed Methods
e two multimodal biometric systems have been designed and evaluated individually; then their performances are compared to determine the best-multifeature model that satisfies maximum accuracy corresponding to each feature extraction method. e three face and two iris databases are combined into six combination datasets. is variety in the image sources and qualities will support the accurate evaluation of the overall system performance. An example of how the four feature extraction methods are implemented by a facial or iris image is illustrated in the next sections.

Face-Iris Datasets.
e specifications of the five standard databases are tabulated in Table 1 [18-20]. e subject must hold two biometric traits (face-iris) using one-to-one relationship. Each database contains 40 subjects, and the multimodal system will classify 240 subjects (one test image with different number of training images) for six combination datasets as explained in the following equation: (1) e two databases, UFI and MMU-1, contain the most difficult images (partial occlusion, different poses, varying contrast, and illumination) that represent the biggest challenges for any biometric system during the detection and classification phases.

Iris Preprocessing.
e image of the iris is preprocessed before applying the feature extraction methods. ese processes are pupil localization, iris localization, and iris normalization. Pupil localization uses the connected component labelling algorithm (CCLA) to detect the region of connected pixel. Moreover, Canny detector and circular Hough transform are both applied for iris localization. e final step is the iris normalization which is implemented using Daugman's Rubber Sheet model. After that, the four feature extraction methods are applied for iris feature extraction and machine vector generation [9].

LBP.
e LBP operation is based on the difference between the threshold value (center pixel) and its eight neighbours.
e LBP-code is set to zero if the deference Applied Computational Intelligence and Soft Computing result is less than zero; otherwise, the LBP-code is set to one [21], where the LBP circle is preconstructed to produce a flexible number of P neighbours with R radius. e LBP operations are presented in equations (2) and (3) [2,21]: where g c is the gray value of center pixel, g p is the gray value of P neighbours (P � 0 to P − 1), and f is the intensity difference of every pixel in the image. e resulted codes are classified into uniform and nonuniform patterns depending on the number of transitions between 1s and 0s bits. e uniform pattern provides two advantages: it selects the important features like lines, corners, end edges, and it saves storage by reducing the code length from 2^P to P(P − 1) + 3. Uniform LBP 8,1 is represented in this work with a radius of 1 pixel and 8 neighbours, where the increase or radius length leads to a decrease in the recognition accuracy as mentioned in [22]. e LBP method has been modified into local ternary pattern (LTP) and improved local ternary pattern (ILTP) for more pattern representation, where the LTP has three levels of coding scheme: positive value, negative value, and zero. is technique is very useful for noisy image applications. e ILTP patterns are classified into two uniform or nonuniform groups, where the extracted patterns are tagged according to the degree of uniformity. e occurrence probability of those tags is extracted as features. is method is very suitable for rotation invariant and noise-resistant image applications [23][24][25].

GLCM.
e gray-level co-occurrence matrix is a second-order feature extraction method. It can work in different angles: vertical 90°, horizontal 0°, diagonal 45°, and 135°, where each direction determines a specific relationship. If the directional information is not important in the feature extraction, the four angles can be applied equally without any concern, and this is still true if the image is isotropic [26]. Based on the GLCM, more than six features can be obtained: contrast, homogeneity, energy, entropy, correlation, variance, and autocorrelation [27]. It is started by counting specific intensity pairs with specific distances and directional angles over a subimage. e result would be a 2D square matrix, and its size equals the number of intensity levels. An example of GLCM computation for three gray level values (1-3), 0°angle, and radius equal to 1 is explained in Figure 1.

PCA.
e PCA is a statistical method that depends on computing the covariance matrix of the feature vectors, then the eigenvalues and eigenvectors. is method is a feature extraction and dimension reduction approach, where the dimension of the templates is reduced with maintaining the important features [28]. e mathematic computations of PCA are determined as steps in Equations (4)- (7). First, calculate the mean(A) for each image vector a i .
Second, let (a i − A i ) (b i − B i ) be represented as a meancentered image for each vector obtained from subtracting the image vector from image mean, and then compute the covariance matrix vectors cov as in the following equation: where A i and B i represent the mean vector value and the two parameters a i and b i are the present values of a and b. n is the number of rows. en, the eigenvalues are calculated from cov matrix as in the following equation: det cov (a,b) − I � 0.
Finally, for each eigenvalue λ, the eigenvector V is obtained as follows:

FDs. FD is a modified frequency-domain method of
Fourier transform. It is applied to describe the shape in the image, where the described object is invariant to the position, rotation, and scale change. Fourier descriptor provides mainly two components, the DC which represents the x-y coordinate of the center point in the border and the radius of the circle that fits the border points. ere are three steps for implementing the FDs procedure. First, the x-y coordinates of the border are converted to complex numbers. Second, the shape signature is performed by computing the centroid distance using equations (8)- (10). Finally, the Fourier coefficients are calculated by equation (10) [8]. x(t), where (x t , y t ) are the coordinates of N samples on the boundary of an image region for t � 0, 1, . . ., N−1, (x d , y d ) is the center point of region, r(t) represents the described boundary shape, and FD n is the Fourier transform coefficient.

Feature-Level
Fusion. e fusion of face and iris traits improves the stability and reliability of the recognition system performance by converting the two unimodal systems into one, known multimodal biometric system. e features of each trait are extracted separately, and then the serial rule is applied to implement the fusion technique, where a concatenation of face and iris features is created sequentially to produce a new pattern for the classification step and make the final decision. Based on the four feature extraction methods, four multimodal biometric systems are produced as shown in Figure 2. e concept of the serial rule is explained in the following equation [29,30]: where X Fq refers to the facial features with vector size q and Y Im refers to the iris features with vector size m, where m and q are not equal.

Score-Level
Fusion. e fusion at the score level is commonly utilized in multibiometric systems [1,2]. In this technique, the recognition results are calculated for each unimodal system separately, and then the recognition score results are fused into one multimodal system to enhance the overall system performance as explained in Figure 3. First, the score vectors of the classification process for both traits (face and iris) are calculated separately and normalized as in Equation (12) at minimum EER value. e sum rule in Equation (13) is applied for fusing the face-iris scores in the second step. Finally, a decision process is obtained using the desired threshold that satisfied the maximum fused system performance [31,32].
where SC i represents the score normalization of the face and iris biometric sample i, min sci and max sci are the minimum and maximum value in the score vector of sample i, respectively, SC fi and SC ri are the score value of the face and iris biometric sample i, respectively, and M refers to the number of biometric systems that have been used. Figure 3 shows the steps of building four multimodal systems based on score-level fusion technique.

Results and Discussion
e multimodal system performances have been evaluated by six combination datasets, four feature extraction methods, and two fusion methods. e maximum recognition rate is measured by Euclidian distance at the minimum EER value that resulted from the intersection of FAR and FRR curves [9].

Recognition Accuracy
From Table 2, the combination dataset UFI-CASIA-V1 has satisfied maximum recognition rate of 85.2885 using scorelevel fusion and LBP method, while the feature-level fusion obtained a rate of 85.0641 using the same feature extraction method. It is clearly noticed that these rates are better than 67.7885 and 83.0449 that have been obtained by feature-level and score-level fusion, respectively, when the UFI-MMU-1 combination dataset is utilized. e low accuracy value of 67.7885 belongs to GLCM approach, and feature-level fusion is much affected by the quality and blurring images in the UFI-MMU-1 dataset, while the LBP and score-level fusion have maintained their 83.0449 recognition rate with those low quality and difficult combinational datasets. Table 3 presents the ORL-CASIA-V1 and ORL-MMU-1 recognition accuracies. Generally, the score-level fusion has achieved maximum recognition rates for both combination datasets. It satisfied 97.5321 and 97.3077 rates using LBP and PCA, respectively. Despite the blurred iris images in ORL-MMU-1, the LBP and PCA are very competitive acquiring maximum accuracies with score-level fusion technique. It is worth mentioning that feature-level fusion achieved excellent recognition rates 97.4359 and 95.4808 using PCA with the same combination datasets, respectively. Finally, from Table 4, the LBP satisfied the most maximum accuracies corresponding to the two fusion methods that have been carried out on AR-CASIA-V1 and AR-MMU-1 datasets. e maximum rate is 95.9615, and it is registered for AR-CASIA-V1 with score-level fusion technique. erefore, the scorelevel fusion and LBP method have the most stable and highest performance for recognizing the face-iris traits in our multimodal system.

Maximum Performance
is section describes analytically and graphically the behaviour of FAR, FRR, EER, and a threshold for the maximum recognition accuracies that have been satisfied by LBP and PCA methods in Table 3. ose good results for the two fusion methods are tabulated in Table 5. ey reflect the highest competitive performance between the two fusion methods as well as the four feature extraction methods that are carried out on good quality ORL-CASIA-V1 combination dataset. Moreover, the score-level fusion has outperformed the feature-level fusion under all circumstances.
However, the shaded cells in Table 5 are graphically represented in Figures 4 and 5. ese figures represent the maximum recognition rates in our multimodal system, and they clearly show the minimum intersection (EER) values between the FAR and FRR at 0.025 and 0.024 for PCA and LBP methods, respectively. e two threshold values for those EERs are 0.06 and 0.02 as explained in Figures 4 and 5, respectively. For more analysis and clear comparison, the recognition accuracies for the two fusion methods and four feature extraction approaches carried out by six combination datasets are graphically represented through Figures 6-11. ese graphs provide full detail about the behaviour of our proposed multimodal systems under deep tests. ey show       Applied Computational Intelligence and Soft Computing that score-level fusion scored 22 times out of 24 in the highest rates against feature-level fusion.

Conclusions
ere is no doubt a multifeature biometric system can improve the performance of a unimodal system and increases its reliability and robustness.
is research has motivated the design philosophy one step farther towards the right selection of fusion rule that produces a strong methodical vision to the interested developers. e quality of the combinational dataset and the applied feature extraction method were the two factors that have been taken into consideration besides the fusion technique to study the system performance for classifying 240 subjects. e experimental results show that score-level fusion has outperformed feature-level fusion in satisfying maximum accuracy rates applied on four feature extraction methods. As a result, the score-level fusion and the LBP method are both candidates for a reliable and robust human identification system using the face-iris traits as a multifeature biometric system. is work can be more improved if a decision-level fusion is considered to calculate the system performance in the classification stage.

Conflicts of Interest
e author declares that there are no conflicts of interest concerning the publication of this paper. Applied Computational Intelligence and Soft Computing 9