^{1,2}

^{1,2}

^{1,2}

^{1,2}

^{1,2}

^{3}

^{3}

^{1}

^{2}

^{3}

To overcome the shortcomings of traditional dimensionality reduction algorithms, incremental tensor principal component analysis (ITPCA) based on updated-SVD technique algorithm is proposed in this paper. This paper proves the relationship between PCA, 2DPCA, MPCA, and the graph embedding framework theoretically and derives the incremental learning procedure to add single sample and multiple samples in detail. The experiments on handwritten digit recognition have demonstrated that ITPCA has achieved better recognition performance than that of vector-based principal component analysis (PCA), incremental principal component analysis (IPCA), and multilinear principal component analysis (MPCA) algorithms. At the same time, ITPCA also has lower time and space complexity.

Pattern recognition and computer vision require processing a large amount of multi-dimensional data, such as image and video data. Until now, a large number of dimensionality reduction algorithms have been investigated. These algorithms project the whole data into a low-dimensional space and construct new features by analyzing the statistical relationship hidden in the data set. The new features often give good information or hints about the data’s intrinsic structure. As a classical dimensionality reduction algorithm, principal component analysis has been applied in various applications widely.

Traditional dimensionality reduction algorithms generally transform each multi-dimensional data into a vector by concatenating rows, which is called Vectorization. Such kind of the vectorization operation has largely increased the computational cost of data analysis and seriously destroys the intrinsic tensor structure of high-order data. Consequently, tensor dimensionality reduction algorithms are developed based on tensor algebra [

Furthermore, traditional dimensionality reduction algorithms generally employ off-line learning to deal with new added samples, which aggravates the computational cost. To address this problem, on-line learning algorithms are proposed [

To improve the incremental learning in tensor space, this paper presents incremental tensor principal component analysis (ITPCA) based on updated-SVD technique combining tensor representation with incremental learning. This paper proves the relationship between PCA, 2DPCA, MPCA, and the graph embedding framework theoretically and derives the incremental learning procedure to add single sample and multiple samples in detail. The experiments on handwritten digit recognition have demonstrated that ITPCA has achieved better performance than vector-based incremental principal component analysis (IPCA) and multilinear principal component analysis (MPCA) algorithms. At the same time, ITPCA also has lower time and space complexity than MPCA.

In this section, we will employ tensor representation to express high-dimensional image data. Consequently, a high-dimensional image dataset can be expressed as a tensor dataset

For tensor dataset

The unfolding matrix of the mean tensor along the

For tensor dataset

For tensor dataset

Tensor PCA is introduced in [

Since it is difficult to solve

According to the above analysis, it is easy to derive the following theorems.

For the order of tensor data

For the first-order tensor,

For the order of tensor data

For the second-order tensor,

Although vector-based and 2DPCA can be respected as the special cases of MPCA, MPCA and 2DPCA employ different techniques to solve the projective matrices. 2DPCA carries out PCA to row data and column data, respectively, and MPCA employs an iterative solution to compute

Because

If the dimensions of projective matrices do not change in iterative procedure, then

MPCA can be unified into the graph embedding framework [

Based on the basic knowledge of tensor algebra, we can get the following:

Given initial training samples

The mean tensor of initial samples is

Consequently, the mode-

Given an initial training dataset

For original samples, the mode-

For new samples, the mode-

input: original samples and new added samples,

output:

Computing and saving

For

Processing QR decomposition for the following equation:

Processing SVD decomposition for the following equation:

Computing the following equation:

Then the updated projective matrix is computed as follows:

Repeating the above steps until the incremental learning is finished.

For tensor dataset

Vector-based PCA converts all data into vector and constructs a data matrix

Letting the iterative number be 1, the time complexity to computing the mode-

For ITPCA, it is assumed that

Taking the space complexity into account, if training samples are reduced into low-dimensional space and the dimension is

In this section, the handwritten digit recognition experiments on the USPS image dataset are conducted to evaluate the performance of incremental tensor principal component analysis. The USPS handwritten digit dataset has 9298 images from zero to nine shown in Figure

The samples in USPS dataset.

At first, we choose 70 samples belonging to four classes from initial training samples. For each time of incremental learning, 70 samples which belong to the other two classes are added. So after three times, the class labels of the training samples are ten and there are 70 samples in each class. The resting samples of original training samples are considered as testing dataset. All algorithms are implemented in MATLAB 2010 on an Intel (R) Core (TM) i5-3210 M CPU @ 2.5 GHz with 4 G RAM.

Firstly, 36 PCs are preserved and fed into the nearest neighbor classifier to obtain the recognition results. The results are plotted in Figure

The recognition results for 36 PCs of the initial learning.

The recognition results under different learning stages are shown in Figures

The recognition results of different methods of the first incremental learning.

The recognition results of different methods of the second incremental learning.

The recognition results of different methods of the third incremental learning.

The comparison of recognition performance of different methods.

The time and space complexity of different methods are shown in Figures

The comparison of time complexity of different methods.

The comparison of space complexity of different methods.

This paper presents incremental tensor principal component analysis based on updated-SVD technique to take full advantage of redundancy of the space structure information and online learning. Furthermore, this paper proves that PCA and 2DPCA are the special cases of MPCA and all of them can be unified into the graph embedding framework. This paper also analyzes incremental learning based on single sample and multiple samples in detail. The experiments on handwritten digit recognition have demonstrated that principal component analysis based on tensor representation is superior to tensor principal component analysis based on vector representation. Although at the stage of initial learning, MPCA has better recognition performance than ITPCA, the learning capability of ITPCA becomes well gradually and exceeds MPCA. Moreover, even if new samples are added, the time and space complexity of ITPCA still keep slower increment.

The authors declare that there is no conflict of interests regarding the publication of this paper.

This present work has been funded with support from the National Natural Science Foundation of China (61272448), the Doctoral Fund of Ministry of Education of China (20110181130007), the Young Scientist Project of Chengdu University (no. 2013XJZ21).