MPE Mathematical Problems in Engineering 1563-5147 1024-123X Hindawi Publishing Corporation 10.1155/2015/384183 384183 Research Article Kernel Fisher Discriminant Analysis Based on a Regularized Method for Multiclassification and Application in Lithological Identification Luo Dejiang 1 Liu Aijiang 2 Cheng Jun 1 College of Management Science Chengdu University of Technology Chengdu 610059 China cdut.edu.cn 2 College of Geophysics Chengdu University of Technology Chengdu 610059 China cdut.edu.cn 2015 1692015 2015 19 07 2014 08 10 2014 10 10 2014 1692015 2015 Copyright © 2015 Dejiang Luo and Aijiang Liu. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

This study aimed to construct a kernel Fisher discriminant analysis (KFDA) method from well logs for lithology identification purposes. KFDA, via the use of a kernel trick, greatly improves the multiclassification accuracy compared with Fisher discriminant analysis (FDA). The optimal kernel Fisher projection of KFDA can be expressed as a generalized characteristic equation. However, it is difficult to solve the characteristic equation; therefore, a regularized method is used for it. In the absence of a method to determine the value of the regularized parameter, it is often determined based on expert human experience or is specified by tests. In this paper, it is proposed to use an improved KFDA (IKFDA) to obtain the optimal regularized parameter by means of a numerical method. The approach exploits the optimal regularized parameter selection ability of KFDA to obtain improved classification results. The method is simple and not computationally complex. The IKFDA was applied to the Iris data sets for training and testing purposes and subsequently to lithology data sets. The experimental results illustrated that it is possible to successfully separate data that is nonlinearly separable, thereby confirming that the method is effective.

1. Introduction

China’s tight clastic rock reservoir is considerably wide, containing sediments that were deposited during the Carboniferous, Permian, Triassic, and Jurassic periods. The reservoirs in Western Sichuan and Erdos are more representative. The West Sichuan depression is located in the Western Sichuan Basin, which belongs to the western depression belt that is located in Yangtze Platform or Longmenshan Fault Zone. A tight gas reservoir was discovered in the Xujiahe and Shaximiao formations that occur in this area. Tight clastic reservoirs are noted for their low porosity, because of their dense, multilayered stacking and strong heterogeneous characteristics that are caused by their complexity and particularity. These characteristics complicate the identification of the lithology, which would enable the prediction of the properties of a reservoir. Previous research [1, 2] identified the reservoir lithology consisting of mudstone, sandstone, and siltstone in the AC area of western Sichuan. The cross plot of sandstone and siltstone shows that these rock types overlap and are mixed together, which is a linearly nonseparable case. The cross plot and mathematical models have been applied extensively in lithology identification in previous studies. For example, Hsieh et al. constructed a fuzzy lithology system from well logs to identify the formation lithology , while Shao et al. applied an improved BP neural network algorithm, based on a momentum factor, to lithology recognition . Zhang et al. used Fisher discrimination to identify volcanic lithology using regular logging data . However, it is very difficult to identify the lithology of tight clastic rock reservoirs with the above method. Thus, this paper proposes the Kernel Fisher discriminant analysis (KFDA) for tight clastic rock lithology identification.

The KFDA has its roots in Fisher discriminant analysis (FDA) and is the nonlinear scheme for two-class and multiclass problems . KFDA functions by mapping the low-dimensional sample space into a high-dimensional feature space, in which the FDA is subsequently conducted. The KFDA study focuses on applied and theoretical research. Billings et al. replaced the kernel matrix with its submatrix in order to simplify the computation [7, 8]; Liu et al. proposed a new criterion for KFDA to maximize the uniformity of class-pair separabilities that was evaluated by the entropy of the normalized class-pair separabilities ; Wang et al. considered discriminant vectors to be linear combinations of “nodes” that are part of the training samples and therefore proposed a fast kernel Fisher discriminant analysis technique [10, 11]. Wang et al. proposed the nodes to be the most representative training samples . Optimal kernel selection is one of the areas of theoretical research that has been attracting considerable attention. Fung et al. have developed an iterative method based on a quadratic programming formulation of FDA . Khemchandani et al. considered the problem of finding the data-dependent “optimal” kernel function via second-order cone programming . The use of KFDA in combination with strong nonlinear feature extraction ability is becoming a powerful tool for solving identification or classification problems. Hence, it has been applied widely and successfully in many areas. Examples of the application of KFDA are face recognition, fault diagnosis, classification, and the prediction of the existence of hydrocarbon reservoirs .

The principle that underlies KFDA is that input data are mapped into a high-dimensional feature space by using a nonlinear function, after which FDA is used for recognition or classification in feature space. KFDA requires factorization of the Gram matrix into the kernel within-class scatter matrix Kw and the between-class scatter matrix Kb. KFDA can finally be attributed to the solution of a generalized eigenvalue problem Kbα=λKwα. As the matrix Kw is often singular, a regularized method is often used to solve the problem, which is transformed into a general eigenvalue problem by choosing a smaller positive number μ, in which case Kw is replaced with Kw+μI. Previous studies have shown the classification ability of KFDA to depend on the value of μ; therefore, the appropriate values for KFDA are very important. In many practical applications, the parameter μ is specified according to experience or experimental results.

This paper proposes a new approach for the selection of the regularized parameter μ to gain the best classification results, and KFDA is improved for both Iris data sets and lithology data sets. The paper is organized as follows. Section 2 summarizes kernel Fisher discriminant analysis. In Section 3, a numerical method for finding an optimal parameter μ is proposed by introducing a regularized method for KFDA. The experimental results are given in Sections 4 and 5, while Section 6 presents the concluding remarks.

2. Kernel Fisher Discriminant Analysis

Let S={x1,x2,,xN} be the data set that contains K classes in the d-dimensional real space Rd. Let Ni samples belong to the jth class, (N=N1+N2++Nk). FDA is used for lithology identification by searching the optimal projection vectors w, and then a different class of lithology samples has minimum within-class scatter. FDA is given by the vector wRd that maximizes the Fisher discriminant function as(1)JW=wTSbwwTSww,where Sw is the within-class scatter matrix and Sb is the between-class scatter matrix. FDA is essentially a linear method, which makes it very difficult to separate the nonlinear separable sample.

KFDA significantly improves the classification ability for the nonlinear separable sample of FDA via the use of a kernel trick. To adapt to nonlinear cases, φ(x) is mapped from the lower dimensional sample space into a high-dimensional feature space. Note that ϕ(xij)(i=1,2,,Nj,j=1,2,,K) represents the ith projection value in the class ωj. Let mϕ be the mean vector of the population, and let mjϕ be the mean vector of class ωj. In the feature space F, the total scatter matrix St, the within-class scatter matrix Sw, and the between-class scatter matrix Sb can be defined as(2)Stϕ=1Ni=1Nϕxi-mϕϕxi-mϕT,Swϕ=1Nj=1Ki=1Njϕxij-mjϕϕxij-mjϕT,Sbϕ=j=1KNjNmjϕ-mϕmjϕ-mϕT.

Lithology identification by KFDA can be attributed to the optimization of kernel Fisher criterion function as follows:(3)Jv=vTSbϕvvTSwϕv,where v represents the different optimal projection vector. The high dimension of feature space F and the infinite dimension make it impossible to directly calculate the optimal discriminant vector v. A solution for this problem is to use the kernel trick as follows:(4)Kxi,xj=ϕxi,ϕxj.

According to the theory of reproducing a kernel , any solution v must lie in the feature space F, which spans ϕ(x1),ϕ(x),,ϕ(xN) as follows:(5)v=i=1Nαiϕxi.In F, any test samples can be projected into w to give the following equation:(6)vTϕx=i=1Nαiϕx,ϕxi=αTϕxϕx1,ϕx,ϕx2,,kkkkkikϕx,ϕxN=αTKx,x1,Kx,x2,,Kx,xN.In F, the kernel within-class scatter matrix Kw and the between-class scatter matrix Kb can be defined as(7)μ0=1Ni=1NKx1,xi,1Ni=1NKx2,xi,,kkkkkk1Ni=1NKxN,xiT,(8)μi=1Nij=1NiKx1,xji,,1Nij=1NiKxN,xji,(9)Kb=i=1KNiNμi-μ0μi-μ0T,(10)Kw=1Ni=1Kj=1Niξxj-μiξxj-μiT,(11)Jv=vTSbϕvvTSwϕv=αTKbααTKwα=Jα.According to the properties of the generalized Rayleigh quotient, the optimal solution vector w is obtained by maximizing the criterion function in (11) by setting it equivalent to the solution of the generalized characteristic equation as follows:(12)Kbα=λKwα.

3. Choosing the Regularized Parameter

If Kw is a nonsingular matrix, then optimal vectors α, obtained by maximizing (11), are equivalent to the feature vectors corresponding to the top m largest eigenvalues [12, 21]. Equation (12) can be described as(13)Kw-1Kbα=λα.

The solution of practical problems requires the use of n training samples to estimate the variance of n-dimensional structure; therefore, Kw is a singular matrix. This means that it is often not possible to use (13). However, it is possible to promote the stability of the numerical method by using regularized method as follows:(14)Kwμ=Kw+μI,where μ is a small, positive number, and I is the identity matrix. Then, (12) can be expressed as(15)Kw+μI-1Kbα=λα.When KFDA is used to solve problems of an applied nature, parameter μ is determined according to the experience or the result of the experiment. This paper uses a numerical analysis method to solve the parameter; hence, the determinant of the value of Kw+μI can be regarded as a function of μ:(16)fμ=Kw+μI.When function f is stable and the value of the function tends to zero (17), the parameter μ is the best classification parameters(17)limμ0fμ0.

4. Experiments 4.1. Experiments Settings

The Iris data set is often used to test the discriminant analysis algorithm . This data set is divided into three classes, which represent three different varieties of the Iris flower: C1, Iris setosa, C2, Iris versicolor, and C3, Iris virginica, which takes petal length, petal width, sepal length, and sepal width as four-dimensional variables. The three classes represent three different varieties of Iris flowers. There are 150 samples in this data set, and there are 50 samples in each class. The results are plotted as scatter grams (Figure 1) and show that classes C1 and C2 and classes C1 and C3 are linearly separable and classes C2 and C3 are nonlinearly separable. The aim was to address this problem by using the KFDA with different values of μ for classification purposes .

Scatter gram plots of Iris data: red (C1, setosa); green (C2, virginica); blue (C3, versicolor).

The following experiments involve a comparative study of the different selection schemes of μ. In this study, the Iris data set is used to conduct algorithm training. Thus, the three sets of samples (based on petal length, petal width, sepal length, and sepal width) were chosen with each of these sets comprising 30 samples. The kernel function employs in KFDA the Gauss kernel function (18), for which the kernel parameter σ is set to be the norm of the covariance matrix of the training samples(18)Kx,y=exp-x-y22σ.

4.2. Experimental Results and Discussions

The experiment was processed within a MATLAB 7.0 environment running on a PC powered by a Pentium 3.3 GHz CPU. The experimental results are shown in Figures 2 and 3. The optimal value of regularized μ is identified in Figure 1, which shows that the function f(μ)=Kw+μI has an obvious inflexion point and that the value of the function f(μ) approaches zero when the parameter is equal to 0.09.

Value of the determinant versus the value of regularized μ.

Classification results with different value of regularized μ.

μ = 0.09

μ = 0.18

μ = 0.045

This is further illustrated in Figure 3, which shows the classification performance when the parameter μ has different values. A simulation was conducted by selecting 30 data points from each of the three classes and the results were constructed in the form of scatter plots that show the distribution of the data by using KFDA. The optimal regularized parameter (μ=0.09) guaranteed separation of classes C2 and C3. However, when the value of the regularized parameter (μ=0.18) increased, it impacted negatively the classification effect.

5. Application to Lithology Data Sets 5.1. The Geological Settings of the AC Region

The AC region is located in the central segment of the West Sichuan depression and was formed during Upper Triassic and Jurassic periods. The Triassic-Jurassic stratum is the part of the thick entity of the Western Sichuan foreland basin, with a total thickness of 6500 m. The AC region is located in a large uplift belt of the West Sichuan depression, which shows NEE trend [1, 2]. This paper uses KFDA for logging parameters identification of lithology, and the purpose of the study horizon is Xujiahe formation. The Xujiahe formation in Chengdu, Deyang, and JiangYou is a typical tight rock formation characterized by low porosity, because of its dense, multilayered stacking and strong heterogeneity. The formation has a typical thickness of 400–700 m, and the thickness of the formation in Anxian county in front of Longmenshan thickness is up to 1000 m.

The Xujiahe formation consists of alternating layers of sandstone, siltstone, mudstone, shale, and coal series, in which the rock types are more complex. According to logging data obtained from wells and the extraction of the physical parameters, the rock in the Xujiahe formation can be divided into mudstone, siltstone, and sandstone based on the physical intersection diagram and histogram analysis of rocks.

5.2. Logging Parameters Lithological Identification

Logging parameters provide a comprehensive reflection of lithology, and their sensitivity is different for lithology identification. The sensitivity of these parameters for lithology identification was studied using the method of correlation analysis and finally determined acoustic (AC), natural gamma ray (GR), density (DEN), and compensated neutron logging (CNL) as the characteristic variables. Training and testing sets in the standard layer, each of which contained 50 samples, were obtained. The cross-sectional plot is displayed in Figure 4. A comparison of Figures 1 and 4 reveals the characteristics that are similar to both, namely, in which the sandstone and siltstone sample data cannot be separated.

Cross-sectional plots of the logging parameters: green O (sandstone); red (siltstone); blue (mudstone).

In the following experiments, FDA and KFDA are compared on logging attribute data sets. The FDA method is used to extract the optimal and suboptimal discriminant vector of the training sets, and then the testing sets of sample are projected onto vectors (Figure 5). As can be seen in Figure 5, the mudstone can be separated from the sandstone and siltstone. The latter two rock types were still mixed together with respect to the siltstone and sandstone data, although the separation degree is higher than indicated by cross plot.

FDA identification result.

Experiments were conducted on logging attribute sets using IKFDA. The kernel function employed in IKFDA is the Gaussian kernel function, and a numerical method was used to obtain the optimal value of parameter μ=0.06. IKFDA was used to obtain the first and second kernel feature vectors, following which the cross plot of the test sets was obtained (Figure 6). As can be seen in Figure 6, it is possible to obtain a good separation between the three types of samples, with each sample forming its own cluster center. The experimental results show that the performance of IKFDA is superior to that of FDA for lithology identification purposes.

IKFDA identification result.

6. Conclusion

The optimal kernel Fisher projection of KFDA can be expressed as a generalized characteristic equation by using a regularized method. For multiclass problems, the value of the regularized parameter is a key factor in the application effect, which is largely influenced by human experience. This paper proposes a novel method to optimize the regularized parameter using a numerical method. Thus, by selecting an optimal value for the regularized parameter, it becomes possible to solve the generalized characteristic equation, thereby eliminating the human factor. The effectiveness of the IKFDA was demonstrated by applying it to the nonlinearly separable Iris and lithology data sets. The experimental results indicated that successful separation was achieved.

In this paper, the selection of the regularized parameter depended on the numerical method. We analyzed the applied validity of the improved Kernel Fisher discriminant analysis without performing deep theoretical analysis. This needs to be addressed by conducting further research in the future.

Conflict of Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.

Acknowledgments

The authors are most grateful for the valuable advice on the revision of the paper from the reviewers. The work was supported by the Foundation of Geomathematics Key Laboratory of Sichuan Province.

Liu A. J. The Rock Physics Research of Typical Tight Clastic Reservoir in Western Sichuan 2010 ChengDu, China Chengdu University of Technology Wang S. Q. Comprehensive Reservoir Evaluation of Xujiahe Formation of the West Sichuan by Log Welling in XC Structure 2009 Chengdu, China Chengdu University of Technology Hsieh B.-Z. Lewis C. Lin Z.-S. Lithology identification of aquifers from geophysical well logs and fuzzy logic analysis: Shui-Lin Area, Taiwan Computers & Geosciences 2005 31 3 263 275 10.1016/j.cageo.2004.07.004 2-s2.0-13844270596 Shao Y. X. Chen Q. Zhang D. M. Kang L. Cai Z. Yan X. Liu Y. The application of improved BP neural network algorithm in lithology recognition Advances in Computation and Intelligence: Proceedings of the 3rd International Symposium, ISICA 2008 Wuhan, China, December 19–21, 2008 2008 5370 Berlin, Germany Springer 342 349 Lecture Notes in Computer Science 10.1007/978-3-540-92137-0_38 Zhang J. Z. Guan Q. S. Tan J. Q. Application of Fisher discrimination to volcanic lithology identification Xinjiang Petroleum Geology 2008 29 6 761 764 Mika S. Ratsch G. Weston J. Scholkopf B. Muller K.-R. Fisher discriminant analysis with kernels Proceedings of the 9th IEEE Signal Processing Society Workshop on Neural Networks for Signal Processing (NNSP '99) August 1999 Madison, Wis, USA 41 48 2-s2.0-0033337021 10.1109/NNSP.1999.788121 Billings S. A. Lee K. L. Nonlinear Fisher discriminant analysis using a minimum squared error cost function and the orthogonal least squares algorithm Neural Networks 2002 15 1 263 270 10.1016/S0893-6080(01)00142-3 2-s2.0-0036195207 Smola A. J. Schökopf B. Sparse greedy matrix approximation for machine learning Proceedings of the 17th International Conference on Machine Learning 2000 San Francisco, Calif, USA 911 918 Liu J. Zhao F. Liu Y. Learning kernel parameters for kernel Fisher discriminant analysis Pattern Recognition Letters 2013 34 9 1026 1031 10.1016/j.patrec.2013.03.005 2-s2.0-84876157849 Xu Y. Yang J. Y. Lu J. F. Yu D.-J. An efficient renovation on kernel Fisher discriminant analysis and face recognition experiments Pattern Recognition 2004 37 10 2091 2094 10.1016/j.patcog.2004.02.016 2-s2.0-4544340626 Zhu Q. Reformative nonlinear feature extraction using kernel MSE Neurocomputing 2010 73 16–18 3334 3337 10.1016/j.neucom.2010.04.007 2-s2.0-78649938823 Wang J. Li Q. You J. Zhao Q. Fast kernel Fisher discriminant analysis via approximating the kernel principal component analysis Neurocomputing 2011 74 17 3313 3322 10.1016/j.neucom.2011.05.014 2-s2.0-80052923712 Fung G. Dundar M. Bi J. Rao B. A fast iterative algorithm for Fisher Discriminant using heterogeneous kernels Proceedings of the 21st International Conference on Machine Learning (ICML '04) July 2004 ACM 264 272 2-s2.0-14344262828 Khemchandani R. Jayadeva Chandra S. Learning the optimal kernel for Fisher discriminant analysis via second order cone programming European Journal of Operational Research 2010 203 3 692 697 10.1016/j.ejor.2009.09.020 MR2559488 2-s2.0-70449732113 Wang G. Q. Ding G. Q. Face Recognition Using KFDA-LLE 2011 Berlin, Germany Springer Li J. H. Cui P. L. Improved kernel fisher discriminant analysis for fault diagnosis Expert Systems with Applications 2009 36 2 1423 1432 10.1016/j.eswa.2007.11.043 2-s2.0-56349114104 Cho H.-W. An orthogonally filtered tree classifier based on nonlinear kernel-based optimal representation of data Expert Systems with Applications 2008 34 2 1028 1037 10.1016/j.eswa.2006.10.029 2-s2.0-36148963325 Zhang Q. Li J. W. Zhang Z. P. Efficient Semantic Kernel-Based Text Classification Using Matching Pursuit KFDA 2011 Berlin, Germany Springer Xu J. H. Zhang X. G. Li Y. D. Application of kernel Fisher discriminanting technique to prediction of hydrocarbon reservoir Oil Geophysical Prospecting 2002 37 2 170 174 Louw N. Steel S. J. Variable selection in kernel Fisher discriminant analysis by means of recursive feature elimination Computational Statistics & Data Analysis 2006 51 3 2043 2055 10.1016/j.csda.2005.12.018 2-s2.0-33750731977 Volpi M. Petropoulos G. P. Kanevski M. Flooding extent cartography with Landsat TM imagery and regularized kernel Fisher's discriminant analysis Computers & Geosciences 2013 57 24 31 10.1016/j.cageo.2013.03.009 2-s2.0-84877029549 Wu K.-L. Yang M.-S. A cluster validity index for fuzzy clustering Pattern Recognition Letters 2005 26 9 1275 1291 10.1016/j.patrec.2004.11.022 2-s2.0-18444403761 Volkovich Z. Barzily Z. Morozensky L. A statistical model of cluster stability Pattern Recognition 2008 41 7 2174 2188 10.1016/j.patcog.2008.01.008 2-s2.0-41549161992 Khan S. S. Ahmad A. Cluster center initialization algorithm for K-means clustering Pattern Recognition Letters 2004 25 11 1293 1302 10.1016/j.patrec.2004.04.007 2-s2.0-23844528211 Zhang X. Performance Monitoring, Fault Diagnosis, and Quality Prediction Based on Statistical Theory 2008 Shanghai, China Shanghai Jiao Tong University