Kernel entropy component analysis (KECA) is a newly proposed dimensionality reduction (DR) method, which has showed superiority in many pattern analysis issues previously solved by principal component analysis (PCA). The optimized KECA (OKECA) is a stateoftheart variant of KECA and can return projections retaining more expressive power than KECA. However, OKECA is sensitive to outliers and accused of its high computational complexities due to its inherent properties of L2norm. To handle these two problems, we develop a new extension to KECA, namely, KECAL1, for DR or feature extraction. KECAL1 aims to find a more robust kernel decomposition matrix such that the extracted features retain information potential as much as possible, which is measured by L1norm. Accordingly, we design a nongreedy iterative algorithm which has much faster convergence than OKECA’s. Moreover, a general semisupervised classifier is developed for KECAbased methods and employed into the data classification. Extensive experiments on data classification and software defect prediction demonstrate that our new method is superior to most existing KECA and PCAbased approaches. Code has been also made publicly available.
Curse of dimensionality is one of the major issues in machine learning and pattern recognition [
To improve performances of the aforementioned approaches to DR, Jessen [
Therefore, the main purpose of this paper is to propose a new variant of KECA and improve the proneness to outliers and efficiency of OKECA. L1norm is well known for its robustness to outliers [
The remainder of this paper is organized as follows: Section
The general L1norm maximization problem is first raised by Nie et al. [
Then a sign function
Initialize
For each
KECA is characterized by its entropic components instead of the principal or variancebased components in PCA or KPCA, respectively. Hence, we firstly describe the concept of the Renyi quadratic entropy. Given the input dataset
We can estimate Equation (
Equation (
Due to the fact that KECA is sensitive to different bandwidth coefficients
The entropic components multiplied by the rotation matrix can obtain more (or equal) information potential than that of the KECA even using fewer components [
In order to alleviate the problems existing in OKECA, this section presents how to extend KECA to its nongreedy L1norm version. For readers’ easy understanding, the definition of L1norm is firstly introduced as follows:
Then, motivated by OKECA, we attempt to develop a new objective function to maximize the information potential (Equations (
Thus, problem (
By singular value decomposition (SVD), then
Algorithm
Initialize
/Phase 1/
Eigen decomposition.
/Phase 2/
Compute the SVD of
This subsection attempts to demonstrate the convergence of the Algorithm
The above KECAL1 procedure can converge.
Motivated by References [
Obviously,
Considering that
Substituting (
According to the Step 3 in Algorithm
Combining (
Jenssen [
More specifically, we are given
This section shows the performance of the proposed KECAL1 compared with the classical KECA [
The experiments are conducted on six datasets from the UCI: the Inonosphere dataset is a binary classification problem of whether the radar signal can describe the structure of free electrons in the ionosphere or not; the Letter dataset is to assign each blackandwhite rectangular pixel display to one of the 26 capital letters in the English alphabet; the Pendigits handles the recognition of penbased handwritten digits; the PimaIndians data set constitutes a clinical problem of diabetes diagnosis in patients from clinical variables; the WDBC dataset is another clinical problem for the diagnosis of breast cancer in malignant or benign classes; and the Wine dataset is the result of a chemical analysis of wines grown in the same region in Italy but derived from three different cultivars. Table
UCI datasets description.
Database 






Ionosphere  351  33  2  30 × 2  175 
Letter  20000  16  26  35 × 26  3870 
Pendigits  10992  16  9  60 × 9  3500 
PimaIndians  768  8  2  100 × 2  325 
WDBC  569  30  2  35 × 2  345 
Wine  178  12  3  30 × 3  80 
The implementation of KECAL1 and other methods is repeated using all the selected datasets with respect to different numbers of components for 10 times. We have utilized the overall classification accuracy (OA) to evaluate the performance of different algorithms on the classification. OA is defined as the total number of samples correctly assigned in percentage terms, which is within
Overall accuracy obtained by the PCAL1, KPCAL1, KECA, OKECA, and KECAL1 using different UCI databases with different numbers of extracted features. (a) Ionosphere, (b) Letter, (c) Pendigits, (d) PimaIndians, (e) WDBC, and (f) Wine.
In software engineering, it is usually difficult to test a software project completely and thoroughly with the limited resources [
Descriptions of data attributes.
Attribute  Description 

WMC  Weighted methods per class 
AMC  Average method Complexity 
AVG_CC  Mean values of methods in the same class 
CA  Afferent couplings 
CAM  Cohesion among methods of class 
CBM  Coupling between Methods 
CBO  Coupling between object classes 
CE  Efferent couplings 
DAM  Data access Metric 
DIT  Depth of inheritance tree 
IC  Inheritance Coupling 
LCOM  Lack of cohesion in Methods 
LCOM3  Normalized version of LCOM 
LOC  Lines of code 
MAX_CC  Maximum values of methods in the same class 
MFA  Measure of function Abstraction 
MOA  Measure of Aggregation 
NOC  Number of Children 
NPM  Number of public Methods 
RFC  Response for a class 
Bug  Number of bugs detected in the class 
This section aims to employ KECAbased methods to reduce the selected software data (Table
Descriptions of software data.
Releases  #Classes  #FP  % FP 

Ant1.3  125  20  0.160 
Ant1.4  178  40  0.225 
Ant1.5  293  32  0.109 
Ant1.6  351  92  0.262 
Ant1.7  745  166  0.223 
Camel1.0  339  13  0.038 
Camel1.2  608  216  0.355 
Camel1.4  872  145  0.166 
Camel1.6  965  188  0.195 
Ivy1.1  111  63  0.568 
Ivy1.4  241  16  0.066 
Ivy2.0  352  40  0.114 
Jedit3.2  272  90  0.331 
Jedit4.0  306  75  0.245 
Lucene2.0  195  91  0.467 
Lucene2.2  247  144  0.583 
Lucene2.4  340  203  0.597 
Poi1.5  237  141  0.595 
Poi2.0  314  37  0.118 
Poi2.5  385  248  0.644 
Poi3.0  442  281  0.636 
Synapse1.0  157  16  0.102 
Synapse1.1  222  60  0.270 
Synapse1.2  256  86  0.336 
Synapse1.4  196  147  0.750 
Synapse1.5  214  142  0.664 
Synapse1.6  229  78  0.341 
Xalan2.4  723  110  0.152 
Xalan2.5  803  387  0.482 
Xalan2.6  885  411  0.464 
Xercesinit  162  77  0.475 
Xerces1.2  440  71  0.161 
Xerces1.3  453  69  0.152 
Xerces1.4  588  437  0.743 
In (
Figure
The standardized boxplots of the performance achieved by PCAL1, KPCAL1, KECA, OKECA, and KECAL1, respectively. From the bottom to the top of a standardized box plot: minimum, first quartile, median, third quartile, and maximum.
This paper proposes a new extension to the OKECA approach for dimensional reduction. The new method (i.e., KECAL1) employs L1norm and a rotation matrix to maximize information potential of the input data. In order to find the optimal entropic kernel components, motivated by Nie et al.’s algorithm [
Although KECAL1 has achieved impressive success on real examples, several problems still should be considered and solved in the future research. The efficiency of KECAL1 has to be optimized for it is relatively timeconsuming compared with most existing PCAbased methods. Additionally, the utilization of KECAL1 is expected to appear in each pattern analysis algorithm previously based on PCA approaches.
The data used to support the findings of this study are available from the corresponding author upon request.
The authors declare that they have no conflicts of interest.
This work was supported by the National Natural Science Foundation of China (Grant no. 61702544) and Natural Science Foundation of Jiangsu Province of China (Grant no. BK20160769).
The MATLAB toolbox of KECAL1 is available.