Latent Fingerprint Segmentation Based on Ridge Density and Orientation Consistency

1Department of Instrument Science and Engineering, School of EIEE, Shanghai Jiao Tong University, Shanghai 200240, China 2Shanghai Engineering Research Center for Intelligent Diagnosis and Treatment Instrument, School of EIEE, Shanghai Jiao Tong University, Shanghai 200240, China 3School of Educational Science, Minnan Normal University, Zhangzhou, Fujian 363000, China 4Department of Automation, School of EIEE, Shanghai Jiao Tong University, Shanghai 200240, China


Introduction
Latent fingerprints are the finger skin impressions, which are unintentionally left on the surface of a crime scene by deposits of oils and/or perspiration from the finger [1].This impression is usually invisible to the naked eye but it can be captured with some special techniques.Latent fingerprints have been used as an important evidence to identify criminals in law enforcement agencies for more than a century.Different from the plain and rolled fingerprints which are captured with good quality in the controlled environment and widely used in commercial area, latent fingerprints are usually of low image quality caused by unclear ridge structure, uneven image contrast, and various overlapping nonfingerprint patterns like lines, printed letters, handwritings, etc. as shown in Figure 1.Samples of rolled, plain, and latent fingerprint images are shown in Figures 1(a), 1(b), and 1(c), respectively.Due to the poor quality, the performance of automated latent fingerprint identification system (AFIS) (matching latent fingerprint to full template fingerprints) is much lower than that of matching full to full fingerprints [1].Thus, it is necessary to perform the segmentation and enhancement processes for latent fingerprints before input to AFIS.However, the latent fingerprints are usually corrupted by noise and the valid fingerprint area is small like Figure 1(c).These problems cause the difficulties in segmenting the fingerprint ridge area from the latent image.
Fingerprint segmentation is an important processing step to segment the fingerprint foreground with the interleaved ridge and valley structure from the complex background with nonfingerprint patterns for more accurate and efficient feature extraction and identification.There are many methods proposed for fingerprint segmentation in the literature [5][6][7][8].
In these methods, the fingerprint image is uniformly divided into blocks and some local image features such as mean and variance of gray values and the orientation consistency are extracted to evaluate the fingerprint quality for segmentation.They can work well for plain and rolled fingerprints, but their performances are not satisfactory for latent fingerprints.The main difficulties may be due to weak ridge structure, and the complex overlapping irrelevant patterns and various types of noise in latent fingerprint images.The valid fingerprint region is small compared to the complex background.Thus, it is still challenging for segmentation of latent fingerprints.
There are some research efforts made to investigate new methods for improving the latent fingerprint segmentation [2][3][4][9][10][11][12][13][14][15].Zhang et al. [2] proposed an adaptive directional total variation (ADTV) model for latent fingerprint segmentation.This model combines both the anisotropic directional TV term and the spatially adaptive fidelity weight for more effective segmentation of latent fingerprint.Short et al. [9] proposed a method to apply an adaptable ridge template to determine a goodness of the fit score in a local region of fingerprint for segmentation.Choi et al. [10] proposed to combine both ridge orientation and frequency features for latent fingerprint segmentation.In that method, the orientation tensor is used to obtain the symmetric patterns of fingerprint ridge orientation and the local ridge frequency is estimated by the local Fourier analysis.These two features are combined for segmentation.Karimi-Ashtiani and Jay Kuo [11] proposed a latent fingerprint segmentation algorithm based on a quasi-global fingerprint model, which did not rely on the local gradients for estimation of orientations and frequencies and thus is robust to the gradient deviations.Cao et al. [12] proposed a method for latent fingerprint segmentation based on a ridge quality measure which was defined as the structural similarity between the fingerprint patch and its dictionary based reconstruction.A learning based method was proposed for latent fingerprint image segmentation based on fractal dimension features and weighted extreme learning machine ensemble [3].The fractal dimension features were computed to characterize the texture patterns of image patches and persistent in geometric quantities.Instead of using the local image features, the linear density, which is a global feature, was computed by detection of line segments for latent fingerprint segmentation [4].A clustering-based approach was investigated for latent fingerprints segmentation [13].A latent fingerprint segmentation method was proposed based on orientation and frequency features [14].Recently, a deep artificial neural network (DANN) model was proposed for segmentation of latent fingerprints by using a stack of restricted Boltzmann machines (RBMs) to learn the features of fingerprint image patches [15].
This paper proposes a latent fingerprint segmentation method based on combination of the ridge density and orientation consistency, which are extracted to characterize both the global and local texture patterns of fingerprint image.First, a total variation (TV) model is used to decompose the latent image into texture and cartoon components.The cartoon component is removed as the noise while the texture component consisting of the fingerprint pattern is used for further processing.Second, to characterize the interleaved ridge and valley structure of fingerprint, we propose to detect the ridge segments and then compute the ridge density and orientation consistency to evaluate the quality of fingerprint image.Finally, a segmentation mask is generated by thresholding and morphological processing for each latent image.The proposed method is tested on NIST SD27 latent fingerprint database and is compared to some existing methods to demonstrate its performance.
The rest of this paper is organized as follows.In Section 2, we will present the proposed latent fingerprint segmentation algorithm in detail.Section 3 provides the experimental results and comparisons.Finally, Section 4 concludes this paper.

The Proposed Method
In this section, we present the proposed segmentation algorithm for latent fingerprints in detail.The valid fingerprint pattern is often characterized as a number of parallel   The total variation (TV) model was proved to be an efficient model for regularizing images without smoothing the boundaries of targets [16].To reduce these structured noises for latent fingerprint segmentation, the TV model is used to decompose the latent image  into texture and cartoon components, i.e.,  =  + V. Texture component V contains the oscillatory and noise component, while cartoon  consists of the piecewise smooth component.Mathematically, the decomposition is obtained by minimization of total variation [16]: where  and V denote the image cartoon and texture components, respectively; TV() denotes the total variation of ; and ‖ ⋅ ‖ 2 is a two norm.The parameter  ∈ [0,1] is used to balance the tradeoff between the fidelity of  and the minimization of TV.Small  will produce weak texture and strong cartoon image and vice versa.In our experiments,  is set to 0.4 to decompose the texture component of latent image for fingerprint segmentation.After decomposition, not only the structure noise but also the varying illuminations have been significantly reduced from the texture image, which will facilitate latent fingerprint segmentation.

Detection of Ridge Segments.
After image decomposition by TV model, a texture image is obtained and used for further processing while the cartoon image with most of the structured noises is removed as nonfingerprint pattern.It will be easier to extract features from the texture image for fingerprint segmentation.The texture component of latent fingerprint is characterized by oscillatory patterns with interleaved and parallel ridge and valley structure.In this work, we propose to detect the ridge segments from the texture image to capture the fingerprint patterns.The ridge segment detection procedure can be divided into image binarization with noise removal, image thinning, and orientation computation for each ridge point, detailed as follows.
First, the texture image is a gray scale image of [0, 255] which is thresholded to a binary image with noise removal.For simplicity, the threshold is set to 128.In the binary image, the black points are the background pixels while the white points are the candidate points of ridge segments.Since the latent fingerprint is often corrupted by various types of noise, it is necessary to remove the noisy points from the binary image for more reliable detection of ridge segments.The noisy points usually occur as the isolated points and are removed according to the following formula: ( Figure 4(a) shows the binary image after removing the noisy points from the thresholded binary image of texture image in Figure 3(c).In addition, to enhance the detection of ridge segments, we also reverse the thresholded binary image with the black points changed into white points and vice versa.The noisy points are removed with the same procedure as above.These two binary images with noise removal are generated for better characterization of the interleaved ridge and valley pattern.Figure 4(b) shows the reversed binary image after removing the noisy points.
Second, the above two binary images are thinned with the thinning method in [17] and these two thinned images are superimposed into one binary image to form ridge segment points.However, there are some noisy ridge segment points, which are usually characterized to be short, closed, and crossed.To remove the noisy ridge points for reliable detection of ridge segments, we delete the short ridge segments with the length smaller than 4 and delete the closed ridge circles with the number of end points less than 2. For each ridge point, we compute the number of ridge points in its 3 × 3 neighborhood.If it is larger than 3, the point is the crossed point, and this point and all the ridge segment points in its 3 × 3 neighborhood are changed to black points.In this way, we go through all ridge points and obtain a set of connected and continuous ridge segments, which are denoted as { 1 ,  2 , . . .,   }.Finally, we compute the orientation for each ridge point.Each ridge segment is composed of a set of connected points.Thus, the orientation of each ridge point is computed based on its neighboring connected ridge points.Assume that the ridge segment   consists of  connected points, denoted as { 0 ,  1 , . . .,   }, where  0 ,   , and   are the starting, middle, and ending points of the ridge segment   , respectively.The orientation of point   is computed as follows: where (   ,    ) is the coordinates of point   .Thus, we have obtained both the location and orientation for each ridge segment point of the binary image.

Computation of Ridge Density and Orientation Consistency.
From Figure 4(c), we can see that the valid fingerprint area should have more parallel ridge segments than the nonfingerprint area.Thus, after detection of the ridge segments and computation of the orientation for each ridge point, we propose to compute a ridge density measure, which evaluates how densely the ridge points are distributed in a local region with the specific orientation.In addition to the locations, the orientations of ridge points are also considered because the neighboring ridge segments are parallel.Thus, the whole orientation range [0 ) is equally divided into 9 bins as { 1 ,  2 , . . .,  9 }, i.e.,  1 ∈ [0, /9),  2 ∈ [/9, 2/9), . . .,  9 ∈ [8/9, ).To compute the ridge density, we divide the binary image with the ridge segment points into the blocks of 16 × 16 pixels.Let   denote the total number of ridge points in each block and     ( = 1, 2, . . ., 9) denote the number of ridge points in each block with the orientation in the bin   .The interleaved ridge and valley flows of fingerprint change smoothly except for a few singular points.Thus the ridge points are densely distributed in a local fingerprint region with consistent orientations.The ridge density for block (, ) is computed by sum of the weighted distribution of the ridge points with respect to different orientations as follows: where  is the area of the block.The ridge density Rd(, ) is high if the ridge points are densely distributed in a block and concentrated in one orientation bin.Otherwise, the ridge density is low.
In addition, the orientations of ridge segments usually change slowly in a local neighborhood.The orientation of each block is defined as the orientation with the maximum number of ridge points.We compute the orientation consistency to evaluate how consistent the orientation of each block is with its 8 neighboring orientations in the 3 × 3 neighborhood.The orientation consistency for block (, ) is computed as are combined to generate an image quality measure, which is computed for each block as Figure 5(c) shows the image quality map of Figure 4(c).
Thresholding is applied to generate a binary segmentation mask for each fingerprint.To fill the gaps and eliminate the islands, morphological operations are further applied to obtain the biggest and continuous binary mask.The convex hull of the biggest mask is computed as the final segmentation mask.Figures show the fingerprint segmentation results by thresholding, the biggest mask, its convex hull, and the final mask overlapped with the texture image, respectively.

Experimental Results
In this section, we conduct experiments on the NIST SD27 latent fingerprint database to test the performances of the proposed algorithm.NIST SD27 consists of 258 latent fingerprints which are manually assigned into three different levels of qualities: "good", "bad", and "ugly" [18].The numbers of "good", "bad", and "ugly" fingerprints are 88, 85, and 85, respectively.Our algorithm was implemented with MATLAB software on a computer of 3.1 GHz Intel Core i5-4440, 4 G RAM, and 64-bit Windows OS.For the latent fingerprint images in NIST SD 27, the average computation time of our algorithm is about 5 seconds for one image.The manual markups of ROI (Region of Interest) by experts provided in [19] are used as the ground truths for NIST SD 27 in our experiments.We compare the segmentation results of the proposed algorithm with the manual markups and other published methods [2][3][4]10].First, we test the proposed algorithm on the same sample latent fingerprints of different image qualities from NIST SD27 database as those in [3].The segmentation results provided in [3] are directly used for comparison.The segmentation results of the method [2] are downloaded from the author's website.Figures 7, 8, and 9 show the comparisons of the segmentation results by the manual markups, the method [2], the method [3], and the proposed method, for good, bad, and ugly latent fingerprint images, respectively.From these results, we can see that the method [2] usually generated a larger segmentation mask which includes the nonfingerprint patterns.There are some gaps and islands in the segmentation results by the method [3].Our proposed method can perform well to separate the fingerprint foreground from the complex background for latent images of different qualities, which is closer to the manual markups.Second, although the segmentation results of latent images can be evaluated visually, we compare the segmentation accuracy of our method with those of some published methods [2][3][4]10].Two measures, the Missed Detection Rate (MDR) and the False Detection Rate (FDR), are computed for performance comparison on NIST SD27.The manual segmentation masks are used as the ground truth.MDR is the average percentage of foreground pixels misclassified as background, while FDR is the average percentage of background pixels misclassified as foreground.Table 1 shows the comparison of the segmentation accuracy in the proposed method with those of the existing methods [2][3][4]10].The segmentation results are directly used from those reported in the paper.In the comparison, the computations of MDR and FDR and the ground truth of segmentation masks by the proposed method are same as those in [2,4,10].The MDR and FDR of our method are 13.03% and 23.17%, respectively.These results show the effectiveness of our segmentation method compared to the other methods.It seems that the method [3] performed best in comparison.But it should be noted that the segmentation ground truth in method [3] was generated by the author of the paper and is different from our proposed method and the computations of MDR and FDR measures were based on image patches instead of pixels.The ultimate goal of segmentation is to improve the performance of latent fingerprint identification.Thus we conduct the latent fingerprint identification experiments to test the effectiveness of the proposed segmentation algorithm finally.The foreground of segmented texture image is the input of latent image matching.The commercial software VeriFinger SDK 6.5 is integrated to perform the feature extraction and matching for latent fingerprint identification.To make the latent matching experiment more realistic and challenging, the template database includes both the 258 template fingerprints of NIST SD27 and the 27,000 fingerprints of NIST SD14.Each latent fingerprint is matched against all the fingerprints of template database and a matching score is generated between the latent and a template fingerprint.The matching scores are sorted in descending order and the identification performance is evaluated by Cumulative Match Characteristic (CMC) curve.A CMC curve plots the identification accuracy rates with respect to different rank-k ( = 1, 2, . . ., 20).
We compare the performances of the proposed algorithm with those of the linear density based method [4] and the manual markup as well as those without segmentation, which are denoted as "linear density based method", "proposed method", "manual markup", and "without segmentation", respectively."Without segmentation" means the texture image is directly used as input for latent fingerprint identification.Figures 10(a), 10(b), 10(c), and 10(d) show the comparisons of CMC curves on all latent, "good", "bad", and "ugly" quality fingerprints of NIST SD27, respectively.We can see that the identification of latent fingerprints with segmentation can achieve much better performance than that without segmentation for all latent fingerprints.For bad latent images, the proposed algorithm performs better than the linear density based method [4] and even better than the manual markup.The performance is just slightly worse than the manual markup for latent fingerprints of good quality.For all fingerprints, the proposed algorithm has better performance than the linear density based method [4] and achieves comparable performance to the manual markup.These results and comparison demonstrate the effectiveness of the proposed segmentation algorithm for latent fingerprints.

Conclusion
This paper proposes a latent fingerprint segmentation algorithm based on combination of ridge density and orientation consistency.First, a texture image is generated by image decomposition with a TV model, and the structured noise and illumination variation is greatly reduced from the texture image.Second, we propose to detect the ridge segments and compute the ridge density and orientation consistency, which are combined for latent fingerprint segmentation.Finally, the proposed algorithm is tested on NIST SD27 and compared with other methods and the manual markups.Experimental results and comparisons demonstrate the effectiveness of the proposed segmentation algorithm for latent fingerprints.

Figure 2 :
Figure 2: The flowchart of the proposed latent fingerprint segmentation algorithm.

Figure 3 :
Figure 3: (a) A latent fingerprint image and its TV decomposed components: (b) cartoon and (c) texture.

Figure 4 :
Figure 4: (a) Binary image with noise removal, (b) the reversed binary image with noise removal, and (c) the final detected ridge segment points.
Figures 3(b) and 3(c) show the decomposed cartoon and texture components of the latent image shown in Figure 3(a), respectively.The cartoon component consists of most nonfingerprint patterns and is removed as noise while the texture image contains the weak fingerprint pattern.

Figure 5 :
Figure 5: (a) The ridge density map, (b) orientation consistency map, and (c) the final image quality map.

Figure 4 (
c) shows the final detected ridge segment points from Figure 4(b).

whereFigure 6 :
Figure 6: Fingerprint segmentation results by (a) thresholding, (b) the biggest mask, (c) the convex hull, and (d) the final segmentation result overlapped with texture image.

Figure 10 :
Figure 10: Comparisons of CMC curves by the proposed method, the linear density based method [4] and the manual markup as well as texture image without segmentation for (a) all, (b) good, (c) bad, and (d) ugly latent fingerprints of NIST SD27.

Table 1 :
Comparison of the segmentation accuracies in different methods.