Research and Development of Palmprint Authentication System Based on Android Smartphones

The security problems of online transactions by smartphones reveal extreme demand for reliable identity authentication systems. With a lower risk of forgery, richer texture, and more comfortable acquisition mode, compared with face, ﬁngerprint, and iris, palmprint is rarely adopted for identity authentication. In this paper, we develop an eﬀective and full-function palmprint authentication system regarding the application on an Android smartphone, which bridges the algorithmic study and application of palmprint authentication. In more detail, an overall system framework is designed with complete functions, including palmprint acquisition, key points location, ROI segmentation, feature extraction, and feature coding. Basically, we develop a palmprint authentication system having user-friendly interfaces and good compatibility with the Android smartphone. Particularly, on the one hand, to guarantee the eﬀectiveness and eﬃciency of the system, we exploit the practical Log-Gabor ﬁlter for feature extraction and discuss the impact of ﬁltering direction, downsampling ratio, and discriminative feature coding to propose an improved algorithm. On the other hand, after exploring the hardware components of the smartphone and the technical development of the Android system, we provide an open technology to extend the biometric methods to real-world applications. On the public PolyU databases, simulation results suggest that the improved algorithm outperforms the original one with a promising accuracy of 100% and a good speed of 0.041 seconds. In real-world authentication, the developed system achieves an accuracy of 98.40% and a speed of 0.051 seconds. All the results verify the accuracy and timeliness of the developed system.


Introduction
Mobile Internet technologies have brought humans a new lifestyle. e emergence of smartphones has better accelerated people's daily life with popular social networks such as online transactions [1]. However, frequent online transactions by smartphones often bring security problems. Specifically, to facilitate the payment, the "mobile wallet" should be linked with a cash card. en, phenomena like Internet fraud and network "phishing" begin to spread extensively, which leads consumers to be still distrustful of the reliability of mobile payment. Moreover, when people trade online, they have to set different complex passwords on various occasions, which may be forgettable. ese facts indicate that mobile payment still demands for reliable mechanisms extremely [2]. Over the last years, fingerprint authentication has drawn significant attention. However, fingerprint can be easily damaged due to physical work and is not informative enough for identity authentication. For iris authentication, users have to suffer from an uncomfortable acquisition mode.
Recently, many researchers focus on face biometrics. Nevertheless, face information is always exposed to the environment and can be easily stolen for identity forgery. As for hand geometry biometric, its physical uniqueness needs further discussion [1]. All in all, with the advantages of abundant textures, comfortable acquisition mode, and lower risk of forgery, palmprint is more acceptable to users in some situations, compared with the other biometrics [1]. e earliest palmprint authentication system was developed on an embedded device by Zhang and Shu team in 1999. ey acquired palmprint by scanning the printed palmprint templates [3]. e offline operating mode renders the system not only time-consuming but also unacceptable to users. To make improvement, in 2003, they utilized CCD to acquire images and set pegs to restrain the palm position. However, the improved equipment is immovable for its large size and still unacceptable for its contacting acquisition mode [4]. In 2010, Zhang et al.'s team further developed a new palmprint device with a smaller size compared with the previous ones, but this device remains to suffer from the contact-based acquisition mode [5]. In 2015, Aykut and Ekinci developed a contactless palmprint authentication system that obtained good usage experience whereas it is inconvenient for mobile applications. In summary, the existing palmprint authentication systems are still not mature enough for general application and deserve in-depth research. Table 1 presents a total comparison among the abovementioned systems.
From Table 1, the contact-based palmprint authentication systems are unacceptable for the uncomfortable acquisition mode, while the contactless systems are not robust to the real disturbance [7]. Besides, the widely adopted feature extractor, Log-Gabor filter, in these systems has a high computation cost, which can greatly reduce the timeliness of the system [8]. Furthermore, the adopted Quadrant coding is unsuitable to make full play of the discrimination of the extracted features [9]. In current years, the Android operation system-based smartphones have occupied 76% of the smartphone market in the world. So, it is meaningful to further study an efficient contactless palmprint authentication system with the application on the Android smartphone.
According to the aforementioned analysis, we design an overall system framework with complete functions for palmprint authentication. Specifically, a vital acquisition scheme is established to ensure the image quality in the contactless acquisition process. Some improvements are introduced to the Log-Gabor filter and the original coding method to reduce the computational complexity and enhance the discriminability of the coding results. On the basis, a system with user-friendly interfaces and good compatibility is developed for the application on the Android smartphone.
Most systems only focus on the palmprint authentication method. Actually, as the primary work, it is crucial to establish an acquisition scheme. us, the image quality and accurate ROI segmentation can be ensured for more reliable identity authentication. Besides, the authentication algorithm is the core issue of our system. Considering the practicality and effectiveness, the Log-Gabor filter is the most suitable one for our system. However, to guarantee the timeliness of our system, we should consider reducing the computational complexity of the Log-Gabor filter. Moreover, a proper coding method should be exploited to enhance the discriminability of the extracted features, which helps the system distinguish one class from the others more easily. Finally, all the procedures should be programmed into modules and linked to the well-designed functional interfaces to facilitate the operation. Depending on the above motivations, the overall system framework proposed in this paper is displayed in Figure 1. Table 2 lists all the involved  evaluation parameters and their definitions in advance. e main contributions of our work are summarized as follows: (i) A palmprint acquisition scheme is proposed against the potential disturbances such as the variation of palm posture and illumination. en, we present a complete set of procedures for palmprint ROI extraction, such as key points location, palmprint angle correction, and the final ROI segmentation.
(ii) To guarantee the efficiency of the system and obtain more discriminative feature coding results, we discuss the impact of filtering direction and downsampling to the performance of Log-Gabor filter and improve the original coding scheme [10,11].
(iii) A palmprint authentication framework with complete functions is designed. We explore and show the details of developing the corresponding system with user-friendly interfaces and good compatibility with Android smartphones.
(iv) Our work provides an open technology to extend the palmprint authentication algorithms to realworld applications, which bridges the algorithmic studies and applications of biometrics. e rest of the paper is organized as follows: Section 2 defines an acquisition scheme and provides a method for palmprint ROI extraction. Section 3 describes the improvements for palmprint authentication. Section 4 introduces the hardware components of the smartphone and the software realization of the palmprint authentication system. Section 5 performs experimental verifications. Section 6 concludes this paper.

Palmprint Acquisition and ROI Extraction
2.1. Acquisition Scheme. In our system, ROI segmentation is finished according to three key points among fingers. However, palm gesture and the position of the palm can affect the key point locating [12,13]. Taking some cases into consideration. e palm deformation often produces visional textures. e large gaps among fingers can lead to imprecise locating results. A long distance between palm and camera can lead the acquired image to have too much background. All of these potential disturbances will bring difficulties for key points locating. So, it is necessary to define an acquisition scheme.
Considering the aforementioned potential disturbances, the acquisition scheme in this paper is determined as follows: five fingers extended and the other four fingers close to each other except for the thumb. A red line and a center point are marked on the interface to conduct the position of the finger root and palm center.  In the reserved subimage, the gaps between fingers root are deemed as the key points and denoted as P 1 , P 2 , and P 3 , respectively, as shown in Figure 2(c).

Columns of Key Points.
e average gray value versus the column of the reserved subimage can be calculated by where F y and F x denote the gradient versus column and row, respectively. (i, j) denotes the pixel coordinate. e three local minima are shown in Figure 3, which correspond to the column indexes of the three key points.

Rows of Key Points.
e three subimages (100 × 20) containing three key points are, respectively, cropped from Figure 4(a) shown in Figure 4(b). For each subimage, the row index of the key point can also be obtained according to formula (1). e average gradient value versus the rows of the first subimage is shown in Figure 5. e X-coordinate of the maximal value corresponds to the row index of the key point.

ROI Segmentation.
e acquired images often contain redundant information such as fingers and background. Palmprint ROI refers to the palm center that contains three flexor lines [14,15]. Here, we segment ROI according to the above located key points.
Since the size of a smartphone screen is generally different from the size of a palmprint image and there exists rotation between them, it is necessary to correct the scale and angle of the palmprint image before ROI segmentation.

Scale Correction.
e key point lying in the original image with a size of w × h is denoted as P i (x i , y i )(i � 1, 2, 3). e corresponding position in the smartphone interface with a size of w ′ × h ′ is denoted as P i ′ (x i ′ , y i ′ )(i � 1, 2, 3). en, the transformation can be defined as Equates to the point at which the FAR and FRR cross D 1 Interval length of the intraclass hamming distance distribution D 2 Interval length of the interclass hamming distance distribution Dis e distance from the upper bound of D 1 to the lower bound of  Mobile Information Systems

Rotation Correction.
e purpose of the rotation is to place P 1 and P 3 on a horizontal line to facilitate ROI segmentation. e calculation of the rotation angle is shown as follows: where (x 1 ′ , y 1 ′ ) and (x 3 ′ , y 3 ′ ) are the coordinates of P 1 ′ and P 3 ′ , respectively. e corresponding coordinates after rotation are denoted as P i ″ (x i ″ , y i ″ )(i � 1, 2, 3). We define the rotation transformation as follows: where [ ]reserves the integer value, while the line P 1 ″ P 3 ″ is set as the x-axis, and its vertical line is set as the y-axis. en, the coordinate system is established in the palmprint image. e length of P 1 ″ P 3 ″ is denoted as d. Moving it down alongside the y-axis by d/3, a square is built up with the side length of d, as shown in Figure 6(a). To obtain a standard ROI, we scale the segmented square to a size of 128 × 128, shown in Figure 6(b).

Palmprint Authentication
In experiments, we find that image preprocessing has a little effect on ROI segmentation results. To shorten the running time of the system, we perform image preprocessing after ROI segmentation.

Image Preprocessing
e pixel value of the original ROI image is denoted as F old (i, j), and the pixel value after the preprocessing is denoted as F new (i, j). e gray enhancement is defined as [16] where low in,high in,low out, and high out are four given constants, according to which the original pixel value between [low in, high in] is then transformed between [low out, high out].

Gray Normalization.
To balance the uneven illumination, we normalize the gray image by the following formulas [17]: where M(F) and VAR(F) signify the mean function and the variance function, respectively. e preprocessed ROI images are shown in Figure 7(c), from which we observe that the three flexor lines and the surrounding details are more obvious and abundant.

Discussion on Log-Gabor Filter.
e three principal lines in palmprint are not sufficient to signify the uniqueness of individuals. So, the feeble wrinkles around them should be amplified to provide more valuable information. What is more, direction information in palmprint should be characterized. e Log-Gabor filter is capable of supporting us to extract such a kind of features. By setting various factors and filtering directions, we can obtain more representative abundant features through the Log-Gabor filter [18,19]. Feature coding is the other important process, in which a more discriminative feature coding method helps to further improve the performance of the system. Note feature extraction corresponding multiple factors and directions can result in a large computation. To guarantee the timeliness and the effectiveness of our system, we will take the discussion on the original Log-Gabor filter and feature coding method (we call the original algorithm in the later content).

Log-Gabor Filter.
e transfer function of the Log-Gabor filter in the linear frequency domain is defined as [20] where f 0 and σ denote the central frequency domain and the bandwidth, respectively. To maintain the shape of the filter, the ratio σ/f 0 is usually fixed [21]. e output of the Log-Gabor filter is related to the filtering direction. Here, we set θ � [0°, 45°, 90°, 135°] to discuss the influence of filtering direction to feature extraction results. e scale wavelength is set as 3, and the ratio is fixed by σ/f 0 � 0.6. e filtered results are shown in Figure 8.
en, a feature matrix with the dimension of 4 × 5 × 128 × 128 � 327680 is obtained by the original algorithm, which leads to a computation disaster [7]. To this end, we bring some improvements to the Log-Gabor filter.
In detail, the ROI image and extracted features will be downsampled to get rid of the redundant information while reducing the computational time. To obtain distinguishable coding results for the final identity matching, we adopt the Gray code to replace Quadrant code to enhance the discriminability of features. e specifics are shown as follows: Step 1: ROI image (128 × 128) is downsampled to the size of 64 × 64. e pixel value before and after filtering is denoted as i(x, y) and I(x, y), respectively. G m (x, y)(m � 1, 2, 3, 4) denotes the Log-Gabor filter Step 2: the features are downsampled with a ratio of 1 : 2 to obtain a feature matrix with a dimension of 4 × 32 × 32. Let I m (x, y) and I m ′ (x, y) denote the pixel value before and after the feature downsampling. e feature downsampling can be performed by the following formula: Step 3: according to the real part and imaginary part of the features, Quadrants coding achieves two kinds of coding results, while Gray coding achieves four kinds of coding results. Hence, Gray coding is conducive to enhance the discriminability of features. Denoting the coding result as Co de I m ′ � [Co de I m ′ R, Co de I m ′ I], then Gray coding can be defined as follows: All the above steps are repeated regarding five scales to obtain a matrix with a dimension of 4 × 5 × 32 × 32 × 2 � 40960. e flow chart of the improved algorithm is shown in Figure 9.

Feature Matching.
Hamming distance is exploited for feature matching to judge whom the acquired palmprint image belongs to. By the bitwise XOR operation on two feature vectors, the rate of the number "1" is defined as Hamming distance. If the samples come from the same user, the Hamming distance can be small enough. Otherwise, the Hamming distance can be large. e calculation of Hamming distance is defined as where P(i) and Q(i) denote the ith elements in two vectors and N denotes the vector dimension. eoretically, when P and Q are exactly the same, HD � 0. Otherwise, HD � 1. ere exist no palmprint images that are totally the same or different in practice, so generally HD ∈ (0, 1). e final authentication is achieved by comparing the Hamming distance with the predetermined matching threshold.

Structure of the Hardware and Software
4.1.1. Hardware. In recent years, Android smartphones have occupied a large market share in the world. Among the most mainstream models in 2018, HUAWEI P20 is the most popular one that carries a host processor HiSilicon Kirin 970, 6 GB RAM, 128 GB ROM, and a 6.1-inch touchscreen with a resolution ratio of 2240 × 1080. Its operating system is Android 8.1. Among these, the camera and GPU are the key components. Four mainstream brands released in the same period with HUAWEI P20 are listed in Table 3 [22].
For the CMOS camera, the larger the camera numbers and the pixel size, the more information can be captured. e smaller the aperture size, the more light could be passed to enhance the image. What is more, G72MP12 is more powerful than Adreno540 and 418 for image processing. After the synthetic comparison, HUAWEI P20 is chosen as the platform of our system. e main hardware structure of HUAWEI P20 is shown in Figure 10.  Figure 1, the system development can be finished through three steps: middle layer development, application layer development, and Android APK generation [23]. e entire development structure is displayed in Figure 11. e details are described as follows: (i) Underlying layer development: in the QT environment, we first develop executable files to call the hardware of the smartphone to finish each procedure in the designed framework. Procedures, like palmprint acquisition, ROI segmentation, feature extraction, feature coding, and feature matching, are all separately programmed into executable functional files by C++ language. (ii) Application layer development: using Android SDK, we develop interfaces to call the executable functional files. rough user-friendly interfaces, the user can operate the system in real time. e development in this stage is finished in Eclipse Java EE environment.
(iii) Android APK generation: we use NDK to compile the functional files developed in the underlying layer. e compiled files are packaged with the JNI standard to facilitate the callback from the application layer. en, dynamic link libraries are generated and packaged into an APK to install on the smartphone.

System Function Design.
Activity is the functional unit in the application layer. Since an activity occupies a certain part of memory, to save the resource, we integrate the interrelated functions into one activity. It is also convenient for us to debug the code and maintain the software. Our APK is composed of four activities: main activity (main interface), database activity (database interface), authentication activity (authentication interface), and acquisition activity (acquisition interface).

Main Activity.
e main activity provides entrances for the other activities. Additionally, the initialization and     storage of the database are finished in this activity. e developed main interface is shown in Figure 12(a).

Database Activity.
Users finish identity registration in this activity. ey can append their identity information by clicking "Add" button and input their palmprint images for the later authentication (5 samples are necessary) by clicking "INPUT SAMPLES" button. Features extraction is finished by clicking "DATA GENERATION" button. Meanwhile, we design "Delete" and "Update" buttons to facilitate database management.
e database interface is shown in Figure 12(b).

Authentication Activity.
Identity authentication is achieved in this activity. After choosing the identity to be confirmed in the identity list, users can click the "Browse" button or "Take photo" button to select the test sample. When clicking the "Start" button, the system will display authentication result, elapsed time, and encourage users to evaluate the authentication result. Users can examine the correct authentication number and the accuracy by clicking the "Accuracy" button. Also, they can click the "ROI Compare" button to check the difference between two segmented ROI images. e authentication interface is shown in Figure 12(c).

Acquisition Activity.
When users enter the real-time acquisition, the rear camera is driven to work. Two modes, "long press" and "click," are designed for the "Take" button. When it is long pressed (about 2 seconds), the located key points and ROI are displayed on the screen. Tips for distance and angle adjustment are given on the top to guide the acquisition. Users can click the "Take" button (about 0.5 seconds) to finish the final acquisition. e developed acquisition interface is shown in Figure 12(d), where the red line and point are set to conduct the palm position.

Statistical Analysis.
e blue spectrum samples in the PolyU multispectral palmprint database containing 6000 samples from 500 volunteers are chosen to verify the improved algorithm [18]. 12 samples of each volunteer are divided into two equal parts. e first part is regarded as the training samples, while the second part is used for the test.
Each volunteer is tested 6 times, and the total intraclass test times reach 3000. For interclass verification, test samples of each volunteer are, respectively, utilized to match each training sample of a selected volunteer. e minimum Hamming distance is deemed as the final matching result. If it is less than the threshold, they are regarded as the same class. Otherwise, they are regarded as different ones [24].
In our simulation, the intraclass Hamming distances distribute in interval [0.1317, 0.2265], while the interclass Hamming distances distribute in interval [0.3118, 0.5271]. e distribution density curves are shown in Figure 13. Based on the above discussion, AA, FRR, and FAR are brought to evaluate the improved algorithm. We set the threshold th ∈ [0.10, 0.55], and the step length is 0.01. e FAR and FRR threshold curves are drawn in Figure 14. It can be seen that the FAR and FRR curves have no intersection, which means EER is 0. Note Figure 14 is more intuitive for us to find the optimum threshold interval, compared with the ROC curve [20].
A comprehensive comparison between the improved algorithm and the original one is listed in Table 4. e evaluation items include the intraclass and interclass Hamming distance, indicator L, feature dimension, and the authentication time. For Hamming distance and indicator L, compared with the original algorithm, the improved algorithm presents more obvious separation between the intraclass and interclass Hamming distance distributions, which greatly reduce the ERR. In addition, we find that the feature dimension is reduced by 6.25%. is guarantees the efficiency of our system.

Comparison with the Other Biometrics.
Apart from palmprint, there are some other biometrics that can be used for identity authentication. is subsection carries out a comparison among the proposed system and some other biometrics-based methods. We particularly consider the new emerging biometrics, EEG [25] and ECG [26][27][28].
e comparison results are shown in Table 5. e collected data in Table 5 indicates that the methods based on the new biometrics all achieved promising accuracy. Recent years have witnessed the great application value of the EEG and ECG biometrics based on the brain wave and heartbeat pattern [25,26]. Meanwhile, the good performance presented in [25][26][27][28] also manifests the great potential of the deep neural networks to learn valuable pattern information in the complicated biometric signals. All of these facts motivate us to further study identity authentication using the new biometrics by deep neural networks.

Complexity Analysis.
As is introduced in Subsections 3.2.2 and 3.2.3, given an ROI image y ∈ R N×M , the entire authentication procedure includes three stages, that is, feature extraction, feature coding, and feature matching [29,30]. e first stage involves image downsampling and Log-Gabor filtering. e computational complexity of image downsampling is N × M. After that, the image size becomes (N/2) × (M/2). According to formula (8), the filtering process should be repeated along 4 filtering directions, respectively, with 5 different scale factors [31,32]. Hence, its computational complexity is

Test of Palmprint Acquisition.
To demonstrate the effectiveness of the defined acquisition scheme, we acquire the palmprint samples during a random period in one day and randomly rotate the palm within a finite angle range [33]. e palmprint acquisition interface is shown in Figure 15.     Under the constraint of the acquisition scheme, most potential disturbances are suppressed. Meanwhile, with the conduction of the set marks, palm appears on the screen with a proper scale. After long pressing the "Take" button (about 2 seconds), key points locating and ROI segmentation are all finished. e side length and rotation angle of ROI are calculated and displayed on the top to suggest users to make the final acquisition.

Test of Palmprint Authentication.
We recruited 500 samples acquired from 50 volunteers in the real world to verify the effectiveness of the developed system. At the beginning, each volunteer was required to register identity with 5 training samples. en, they are required to implement 5 times intraclass real-time authentication and 5 times interclass real-time authentication, respectively. Since the palmprint database is established in real application [34], the variations of palm posture, background, and illumination commonly exist in the samples. Here, we set the threshold as 0.30. Specifically, after selecting the identity to be confirmed from the identity list, users can click the "Take photo" button to finish real-time acquisition for the test or click the "Browse" button to choose a previously acquired sample for the test. e system will display the storage path of the selected sample. e intraclass and interclass authentication operations are shown in Figures 16(a)-16(d), respectively. Users can click the "Start" button to trigger authentication. en, the system will display authentication result and running time and encourage the user to make an evaluation. Additionally, we can check ROI of the test sample and training sample by clicking the "ROI compare" button. We can also view the total authentication times, correct authentication number, and authentication accuracy, by click the "Accuracy" button, as shown in Figure 17(c).
After a 250-time test, we find that the authentication accuracy of the developed system is close to that in MATLAB simulation. Due to the unavoidable disturbances in the acquired sample, the authentication accuracy in real-time acquisition is reduced to 98.40%. Note the authentication in MATLAB simulation takes about 0.0278 seconds for average, while it is 0.051 seconds on the smartphone. All the test results on MATLAB and smartphone are listed in Table 6.
Meanwhile, we find that if the acquisition process follows the defined constraint scheme, we can obtain more promising accuracy. Otherwise, the accuracy will be impacted.
is indicates the importance of the established acquisition scheme. We also find that when the light intensity changes within a limited degree, the authentication accuracy can be  nearly not reduced, which demonstrates that our system shows robustness to some extent.

Conclusion
In recent years, there emerges an extreme demand for reliable identity authentication system for secure mobile transactions. is paper presents an effective and efficient palmprint authentication framework towards the application on the Android smartphone. It contains all the required functions during palmprint authentication, including palmprint acquisition, ROI segmentation, feature extraction, and the final identity authentication. More importantly, our method provides an open technology to extend the palmprint authentication algorithms to realworld applications by exploring the development of a user-friendly palmprint authentication system based on the Android smartphone, which well bridges the algorithmic study and the real-world application of palmprint authentication. is paper presents our exploratory work of propelling the application of palmprint authentication on the mobile smart terminals. Considering the timeliness of the system, we choose the practical and effective Log-Gabor filter for feature extraction. However, when the palmprint is subjected to corruption or occlusion, the Log-Gabor filter is unable to extract robust features anymore. In the next stage, we will manage to enhance the robustness of our system concerning more complicated application scenarios. Furthermore, we will also consider achieving identity authentication using the new emerging EEG and ECG biometrics with the powerful deep neural networks.

Disclosure
is paper is an extended version of an earlier conference paper: Research of Palmprint Authentication Arithmetic Based on Smartphone, in proceedings of the 2018 VII International Conference on Network, Communication and Computing, Taipei City, Taiwan, Dec. [14][15][16]2018 Conflicts of Interest e authors declare no conflicts of interest.