Mathematical Problems in Engineering
Volume 2014 (2014), Article ID 572124, 12 pages
http://dx.doi.org/10.1155/2014/572124

Research Article

Generalized Framework for Similarity Measure of Time Series

Hongsheng Yin,¹ Honggang Qi,² Jingwen Xu,2,3 William N. N. Hung,⁴ and Xiaoyu Song⁵

¹China University of Mining and Technology, Xuzhou 221116, China
²University of Chinese Academy of Sciences, Beijing 100049, China
³Institute of Electrical Engineering, Chinese Academy of Sciences, Beijing 100190, China
⁴Synopsys Inc., Mountain View, CA 94043, USA
⁵Portland State University, Portland, OR 97207, USA

Received 9 July 2014; Revised 10 October 2014; Accepted 13 October 2014; Published 3 November 2014

Academic Editor: Yan Liang

Copyright © 2014 Hongsheng Yin et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract.

Currently, there is no definitive and uniform description for the similarity of time series, which results in difficulties for relevant research on this topic. In this paper, we propose a generalized framework to measure the similarity of time series. In this generalized framework, whether the time series is univariable or multivariable, and linear transformed or nonlinear transformed, the similarity of time series is uniformly defined using norms of vectors or matrices. The definitions of the similarity of time series in the original space and the transformed space are proved to be equivalent. Furthermore, we also extend the theory on similarity of univariable time series to multivariable time series. We present some experimental results on published time series datasets tested with the proposed similarity measure function of time series. Through the proofs and experiments, it can be claimed that the similarity measure functions of linear multivariable time series based on the norm distance of covariance matrix and nonlinear multivariable time series based on kernel function are reasonable and practical.

1. Introduction

A complex system typically needs to be described with multiple state variables. These state variables can be obtained by experimental observations and instrumental measures. With these state variables, a set of discrete multivariate time series (MTS) can be constructed. Mathematically, MTS is expressed in matrix form . Its samples are , ; , where and denote the number of observation variables and the number of observation samples of each observation variable, respectively. Obviously, is a matrix. If , the matrix represents MTS. Otherwise, if , is simplified to a univariate time series (UTS), which is a special case of MTS and can be denoted as a -dimensional vector . Time series theory is widely applied in various fields such as electricity, finance, medical, multimedia, meteorology and hydrology, scientific research, and industrial control. Discovery of the hidden information and operating regularity in a time series is a research hotpot in data mining and knowledge discovery. The research on time series includes clustering, classification, similarity search, feature extraction, trend forecasting, and decision support. Similarity measures a fundamental research topic on time series theory.

Most of the existing research on the similarity measure of time series is focused on UTS. The common measure functions are -norms () [1, 2], DTW [3–5], longest common subsequence (LCSS) [6], edit distance on real sequence (EDR) [7], edit distance with real penalty (ERP) [8], spatial assemble distance (SpADe) [9], DISSIM [10], swale [11], and TQuEST [12]. With further research on UTS, we can expect more new methods will be proposed in the future. However, few researches focus on the similarity measure of MTS. Yang and Shahabi [13, 14] calculated the similarity of MTS with the extended Frobenius norm. Xu et al. [15, 16] measured the MTS similarity based on information theoretic learning framework.

The contributions in the research on framework for similarity measure of time series are also less. Liu and Jiang [17] proposed a concept of similarity of time series through analyzing the geometric relation of Euclidean distance of time series in high-dimensional space, which describes the similarity relation between two UTS using both similarity function and transform constraint function to establish an exact concept for similarity of time series.

From existing UTS and MTS literatures, the similarity measure methods of time series are undefined and nonstandardized which makes the research very difficult. Thus, it is very necessary to establish a general concept for uniform similarity of time series. In this paper, we give the definition of similarity measure function of UTS with vector norm and proved that the similarity functions defined in the form of norms are equivalent in original UTS space and linear transformed space. Moreover, we also extend the definitions of similarity of time series and present a uniform theory of similarity measure based on set theory, metric space theory, operator theory, matrix theory, and kernel method. The uniform theory based on distance of vector/matrix norm can be used for measuring the similarity of time series in both original and transformed spaces, whether the time series are univariable or multivariable and the linear transform or the nonlinear transform. The theory analysis and experimental results show that the definition of distance of vector/matrix norm is equivalent in original and transformed spaces.

The rest of this paper is organized as follows. In Section 2, it is proved that the vector norm based definitions of similarity functions are equivalent in UTS original and linear transformed spaces. In Section 3, the theory that matrix norm is used for defining the similarity function of linear MTS is discussed. Then, the theory that kernel function is used for defining the similarity function of nonlinear MTS is discussed in Section 4. In Section 5, all similarity functions proposed in this paper are discussed and analyzed. It is proved that the norm based definition of similarity functions for measuring the similarity of UTS is equivalent in time domain and Fourier transform or wavelet transform domain. Also, the similarity function of linear MTS defined based on covariance matrix norm distance and the similarity function of nonlinear MTS defined based on kernel function are analyzed in this section. The experimental results are shown in Section 6. Finally, we conclude this paper in Section 7.

2. Similarity Measure of Linear Univariable Time Series

For the original data of linear univariable time series, whether they are recoded manually or sampled automatically, it is assumed that they all satisfy Shannon theorem of information theory without any distortion and without considering the data dimension. The linear univariable time series is denoted as set . Considering two UTS samples , wherein is the time series to be observed (observed time series) and is the time series to be referenced (referenced time series). They are represented in vector form , , . To measure the similarity of and , the subseries with the same dimension as needs to be extracted from . All -dimension successive subseries of compose a set , , in which each element represents a -dimension successive time series. The element in set is denoted as , , which is a -dimension vector.

The similarity measure between observed time series and referenced time series is that the subseries of , and the given referenced series are measured in the similarity measure function with the threshold .

Definition 1. If the similarity function of time series , the observed subseries and referenced series are similar.

According to the literatures [1, 2], there are two cases of time series similarities, which are as follows: if and , and are exactly matching; if , , and , the subseries and are subseries matching. Obviously, exactly matching is a special case of subseries matching. In this paper, subseries and similarity measure function are defined for unifying the above two cases of similarity matching of time series and measuring the similarity of and in the same dimension conveniently. The following definitions on similarity of UTS are all based on the unified similarity.

2.1. Common Similarity Measure Functions

In current research, the distance function of time series and , , is commonly used as the similarity measure function . The common distance metrics are as follows [17].

Euclidean distance:

City-block distance:

Chebyshev distance:

Minkowski distance:

Cosine distance:

Correlation distance:

Mahalanobis distance

In formula (7), matrix is the covariance matrix of and . Mathematically, it can be proved that the distance formulas (1)–(3) are special cases of formula (4) with . In these distance functions, the Euclidean distance is frequently used in practice. Obviously, given the same dimension, the more the similarity between two vectors, the smaller the values of similarity measure functions (1)–(7), and vice versa.

2.2. Relevant Concepts of Time Series Transform

Since time series and are high-dimension data, , straightforward analysis and processing of similarity of time series need huge computation burden which is unacceptable on both time and space complexities. Although measuring the similarity of two time series is intuitively straightforward, the result may be not very accurate. Thus, the time series and need to be properly transformed. The transformed time series are denoted as and , and denotes the transformed space, , and their transform factor is . The transform should be lossless or lossy within a very small margin so that no or less accuracy loss of data is introduced by the transform. Through the space transform, the dimension of data is greatly reduced so that the data can be processed with lower complexity. Accordingly, it is very important to select a proper transform for solving this problem. For notational convenience the relevant definition is given as follows.

Definition 2. Let and be two measure spaces. If there exists a mapping from to , and , , then it can be said that and are isometric, and is the isometric mapping from to .

Definition 3. Let be an operator (mapping) of the normed linear space from to . If the following hold:

(1)additive ,(2)homogeneity ,

then it can be said that is a linear operator from to .

Definition 4. represents the all bounded linear operators of the normed linear space from to . Let and let be arbitrary number. If and the following hold:

(1)additive ,(2)homogeneity ,

then it can be said that is a linear space.

Definition 5. Let and be two normed linear spaces and let be the linear operator from to . If and , then it can be said that and are isometric and is the norm preserving isomorphic mapping from to .

2.3. Definition of Similarity Relation in Set Theory

Definition 6. Let be a relation in set . If   and , then it can be said that relation is reflexive. If , , and , then it can be said that relation is symmetrical. If , , , and there is , then it is said that relation is transitive. If a relation is both reflexive and symmetrical, it is a similarity relation.

It should be noted that the similarity relation does not have the transitive property. For example, father and son are similar, and mother and son are also similar, but father and mother are very possible to be dissimilar. According to the above analysis of similarity of time series, the similarity measure functions are proposed based on the similarity relation of set theory.

2.4. Extended Similarity Definition of Time Series

According to the above discussion, the similarity measure function of time series not only satisfies the similarity relation in set theory, but also should be uniformed in the original space and the transformed space . Objectively, the straightforward similarity measure of two time series is more accurate than nonstraightforward similarity measure in which the geometric triangle inequality is used for the similarity measure through the third-party time series. Thus, Definition 1 is extended as follows.

Definition 7. Given a linear operator from to and , are the transforms of by the operator . When the similarity measure function ( is a given threshold of similarity), the time series are similar in the constraints of . Meanwhile, the similarity function should satisfy the following.

(1)Reflexive symmetrical positive definiteness, that is, , if and only if , , and the equation holds.(2)Geometric triangle inequality:

 ,  .

The generalized similarity definition in different spaces is given by Definition 7. The key problem is how to find the proper operator and similarity measure function .

Lemma 8. Any two norms of finite-dimension linear space are equivalent.

Proof. Let and be two norms of linear space and let be a linear basis of . If and , then there are positive integers , which satisfy and . Thus, ; that is, . Finally, it can be proved that and are equivalent.

Theorem 9. Let be the norm preserving isomorphic mapping from to . The time series are transformed as by . has the usual definition of the norm , with which the distance can be derived by the norm. Similarly, the distance also can be derived by the norm . Then, consider the following.

(1)In space , formulae (1), (2), (3), and (4) represent that the similarity measure function is equivalent.(2)In spaces and , the similarity measure functions represented by and are equivalent.

Proof. Since formulae (1), (2), (3), and (4) are 2-norm, 1-norm, -norm, and -norm of the vector difference, respectively, according to Lemma 8, formulae (1), (2), (3), and (4) can be easily proved by norm axioms that they satisfy Definition 7. Thereby it is proved.
It may be known from Definitions 3 and 5 that , and then it is easily proved by norm axioms that they satisfy Definition 7.

3. Similarity Measure of Linear Multivariable Time Series

In previous sections, we mentioned the definitions of similarity of univariable time series. The signal variable time series is represented in vector form mathematically. The more similar the two vectors, the shorter the distance between them. The distance of two identical vectors should be zero. Thus, the similarity function of univariable time series should be defined based on vector norm. Multivariable time series is represented in matrix mathematically. So, the similarity function of multivariable time series should be defined based on matrix norm.

Definition 10. Matrix norms: if the real function , where is any matrix , satisfies the following:

(1)positive definiteness: and , if and only if ,(2)homogeneity: for any real number , ,(3)triangle inequality: for , ,(4)compatibility: for , ,

then it is said that is a matrix norm in , that is, the norm of matrix .

3.1. Commonly Used Matrix Norm

The commonly used matrix norms are given as follows:

(1), the maximum sum of absolute element of each column of matrix , and is also called the column norm of ;(2), the maximum sum of absolute element of each row of matrix , and is also called the row norm of ;(3), wherein denotes matrix transpose of and is the maximum of absolute eigenvalue of , and is also called the 2-norm of , or spectral norm;(4), wherein is conjugate transpose of . is called the Frobenius norm (-norm), which is similar to 2-norm in the form of vector, and also is compatible with the vector norm . Its advantage in the norm invariance after -norm is multiplied by unitary matrix, that is, the following theorem.

Theorem 11. Let and be -rank and -rank unitary matrices, respectively, and then .

Proof. . According to definition, we know that . With the above results, we have .
Same as vector norm, the equivalence of matrix norms has the following similar conclusions. For , any matrix norms of are equivalent. Mathematically, in the above four kinds of norms, and can be easily computed, and and have better properties and are widely applied. However, is more complicated in engineering application and sensitive to the variation of matrix elements. can be computed more easily and thus is widely applied.

3.2. Definition of Similarity Functions of Multiple Variable Time Series

It is assumed that and are MTS, their samples are denoted as , , ; , is the number of observations, and is the size of observations. Obviously, they are matrices, which are denoted as and . Their definition of similarity function is as follows:

4. Similarity Measure of Nonlinear Time Series

We are inspired by support vector machines (SVMs), where a classifier can convert the difficult nonlinear classification problem in input space into simple linear classification problem in feature space using kernel method. The essence is that the difficult classification of unclear similarity of the same class of samples and small difference of different classes of samples are converted from the input space into the feature space in which the similarity of the same class of samples is enhanced and the difference of different classes of samples is enlarged. Thus, in this paper, the kernel method is introduced for the research of the similarity of nonlinear time series.

In input space , the researching time series is a -dimension multivariable with samples, and same as the researching series, the reference time series is a -dimension multivariable with samples. In the feature space , the nonlinear mapping of and is denoted as and .

Definition 12. In the feature space , the inner product and norm are defined as follows:

Definition 13. The similarity function of the researching series and the reference time series are defined as follows:

5. Discussion and Analysis

5.1. Analysis on the Similarity of Linear Transformed Univariable Time Series

According to operator theory, both Fourier transform and wavelet transform are linear operators. Thus, the relevant conclusions on the similarity of time series based on Fourier transform and wavelet transform are obtained as follows.

Lemma 14. Let , and then Fourier transforms of and , , have the following properties:

(1)invariant inner product ,(2)norm preserving .

Proof. Consider the following:

For the above equation, let , and then there is .

Inference 1. Time series are transformed as by discrete Fourier transform (DFT). The similarity functions is represented by norms and are equivalent.

Proof. According to Theorem 9 and Lemma 14, the results are directly deduced.

Analogously, the following conclusions are also obtained. Due to the limited space, the proof is not presented here.

Inference 2. Time series are transformed as by discrete wavelet transform (DWT). The similarity function represented by norms and is equivalent.

5.2. Practice Computing Method of Similarity of Linear Multivariable Time Series

Let matrices and represent MTS and , and let the covariance matrices between the columns in and be and . All eigenvalues of covariance matrices are arranged in descending order and their corresponding standard orthogonal eigenvectors are . Similarly, all eigenvalues of covariance matrices are arranged in descending order , and the corresponding standard orthogonal eigenvectors are . Thus, the eigenvectors of covariance matrices and are and , respectively, and then their extended similarity function is defined as follows:

According to principle component analysis, physical meaning of formula (13) is equivalent to that the distance (similarity) between the linear orthogonal transforms of and can be measured using the norms in the transformed space. Yang and Shahabi [13, 14] use the extended Frobenius norm to compute the similarity of MTS which is a special case of formula (13). Formula (13) can be understood by referring to the definition of similarity function of univariable time series transformed by linear operator.

5.3. Practical Computation of Similarity of Nonlinear Multivariable Time Series

Mathematically, it is not difficult to prove that the norm of formula (10) derived by the inner product of formula (9) satisfies the Parallelogram formula: . Since , it does not need to know the mapping function explicitly. Instead, the kernel function is used to compute the similarity function defined in formula (11); that is,

Formula (14) shows that the nonlinear similarity measure of nonlinear time series in input space can be linearly measured in feature space through kernel method. Currently, commonly used kernel functions are as follows:

(1)linear kernel function:

(2)-order polynomial kernel function:

(3)Gaussian radial basis RBF kernel function:

(4)neural network kernel function:

6. Experimental Results

In our experiments, we take the nonlinear multivariate time series as a typical example without loss of generality for showing the accuracy of the proposed similarity measurement method. The similarity measurement function of formula (1) is used in the input space, and the similarity measurement function of formula (14) is used in the feature space. The Gaussian radial basis RBF kernel function of formula (17) is used for mapping the nonlinear samples to high-dimension space, which is low complexity compared with polynomial kernel function, especially the high rank kernel function. All samples in the original test and training sets are well mixed. The samples are randomly selected to construct the new test and training sets. Because the KNN classifier can decide a sample belong to the class in which more of nearest samples are contains. Based on the proposed similarity measure function to verify the equivalence of time series in different spaces, KNN and kernel KNN classifiers are employed to classify the samples in the new sets in input space and feature space, respectively. Each random experiment is carried out 20 times. Then the classification accuracies in the experiments are analyzed and compared. Moreover, the parameter of Gaussian radial basis of RBF kernel is optimized in the experiments. The is initialized as a small value first. Subsequently, each random experiment for determining each value of is repeatedly carried out 20 times. The is increasingly adjusted according to the average classification accuracy of each experiment until the optimal value of is obtained. In our experiments, five published datasets, Cylinder-Bell-Funnel (CBF), Fish, Face (four) [18], Iris, and Wine [19], are tested.

Presently, there were already some researching work on the classification of time series. One of the most common benchmark datasets is CBF [20] which was used by [21–23]. CBF dataset consists of three time series, Cylinder , Bell , and Funnel , which are generated by the following equations:

where and are drawn from a standard normal distribution , is an integer drawn uniformly from the range , and is an integer drawn uniformly from the range . The three typical curves, Cylinder, Bell, and Funnel, are shown in Figures 1(a)–1(c), respectively. The curve of classification accuracy relative to is shown in Figure 1(d) and the classification accuracy rates of KNN and kernel KNN classifiers are also shown in Figure 1(e). It can be seen from these experimental results that the similarity of time series in original and feature spaces, which are measured with the proposed generalized similarity function defined by the distance of vector or matrix norm, is equivalent regardless of whether the time series are linearly or nonlinearly transformed. The same conclusion is obtained from the experimental results in Figures 2, 3, 4, and 5. Moreover, the average classification accuracies of each experiment repeatedly tested 20 times for five datasets are listed in Table 1, which also confirm our conclusion.

Table 1: Discriminant accuracy comparison.

Datasets
Average accurate rate

KNN
Kernel KNN

CBF
0.862
0.862

Fish
0.793
0.792

Iris
0.953
0.953

Wine
0.728
0.724

Face (four)
0.893
0.893

(a) Class 1

(b) Class 2

(c) Class 3

(d) Accurate rate with different

(e) Discriminant accuracy comparison

Figure 1: Test results of dataset CBF.

(a) Class 1

(b) Class 2

(c) Class 3

(d) Class 4

(e) Accurate rate with different

(f) Discriminant accuracy comparison

Figure 2: Test results of dataset Face (four).

(a) Class 1

(b) Class 2

(c) Class 3

(d) Class 4

(e) Class 5

(f) Class 6

(g) Class 7

(h) Accurate rate with different

(i) Discriminant accuracy comparison

Figure 3: Test results of dataset Fish.

(a) Class 1: Iris setosa

(b) Class 2: Iris versicolor

(c) Class 3: Iris virginica

(d) Accurate rate with different

(e) Discriminant accuracy comparison

Figure 4: Test results of dataset Iris.

(a) Class 1

(b) Class 2

(c) Class 3

(d) Accurate rate with different

(e) Discriminant accuracy comparison

Figure 5: Test results of dataset Wine.

7. Conclusion

Based on set theory, metric space theory, operator theory, matrix theory, and kernel method, the definition of similarity of time series is extended and unified theoretically to establish a generalized framework for the similarity measure of time series. In the generalized framework, the similarity of time series is defined as the distance of unified vector/matrix norm, which is suitable for both time series of univariable and multiple variables in any linear transformed space or nonlinear transformed space. The proposed similarity definition has been proven to be equivalent in original space and transformed space. The experimental results on some published time series datasets confirm that the theoretical deduction on the generality of similarity measure of time series defined in this paper is right.

Conflict of Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.

Acknowledgment

This work is partly supported by the National Natural Science Foundation of China 61379100 and 61472388.

References

C. Faloutsos, M. Ranganathan, and Y. Manolopoulos, “Fast Subsequence matching in time-series databases,” in Proceedings of the ACM International Conference on Management of Data, vol. 23, pp. 419–429, 1994.

R. Agrawal, C. Faloutsos, and A. Swami, “Efficient similarity search in sequence databases,” in Proceedings of the International Conference on Foundations of Data Organization and Algorithms, pp. 69–84, Chicago, Ill, USA, 1993.

D. J. Berndt and J. Clifford, “Using dynamic time warping to find patterns in time series,” Knowledge Discovery and Data Mining Workshop, vol. 10, no. 16, pp. 359–370, 1994.

E. Keogh and C. A. Ratanamahatana, “Exact indexing of dynamic time warping,” Knowledge and Information Systems, vol. 7, no. 3, pp. 358–386, 2005.

D. J. Berndt and J. Clifford, “Finding patterns in time series: a dynamic programming approach,” in Advances in Knowledge Discovery and Data Mining, pp. 229–248, American Association for Artificial Intelligence, Menlo Park, Calif, USA, 1996.

M. Vlachos, G. Kollios, and D. Gunopulos, “Discovering similar multidimensional trajectories,” in Proceedings of the 18th International Conference on Data Engineering, pp. 673–684, San Jose, Calif, USA, March 2002.

L. Chen, M. T. Ozsu, and V. Oria, “Robust and fast similarity search for moving object trajectories,” in Proceedings of the ACM International Conference on Management of Data (SIGMOD '05), pp. 491–502, Baltimore, Md, USA, 2005.

L. Chen and R. T. Ng, “On the marriage of -norms and edit distance,” in Proceedings of the 30th International Conference on Very Large Data Bases, vol. 30, pp. 792–803, Toronto, Canada, 2004.

Y. Chen, M. A. Nascimento, B. C. Ooi, and A. K. H. Tung, “SpADe: on shape-based pattern detection in streaming time series,” in Proceedings of the 23rd International Conference on Data Engineering (ICDE '07), pp. 786–795, Istanbul, Turkey, April 2007.

E. Frentzos, K. Gratsias, and Y. Theodoridis, “Index-based most similar trajectory search,” in Proceedings of the 23rd International Conference on Data Engineering (ICDE '07), pp. 816–825, Istanbul, Turkey, April 2007.

M. D. Morse and J. M. Patel, “An efficient and accurate method for evaluating time series similarity,” in Proceedings of the ACM International Conference on Management of Data, pp. 569–580, 2007.

J. Aßfalg, H.-P. Kriegel, P. Kröger, P. Kunath, A. Pryakhin, and R. Renz, “Similarity search on time series based on threshold queries,” in Advances in Database Technology—EDBT 2006, vol. 3896 of Lecture Notes in Computer Science, pp. 276–294, Springer, Berlin, Germany, 2006.

K. Yang and C. Shahabi, “A PCA-based similarity measure for multivariate time series,” in Proceedings of the 2nd ACM International Workshop on Multimedia Databases (MMDB '04), pp. 65–74, Washington, D.C., USA, November 2004.

K. Yang and C. Shahabi, “An efficient nearest neighbor search for multivariate time series,” Information and Computation, vol. 205, no. 1, pp. 65–98, 2007.

J.-W. Xu, A. R. Paiva, I. Park, and J. C. Principe, “A reproducing kernel Hilbert space framework for information-theoretic learning,” IEEE Transactions on Signal Processing, vol. 56, no. 12, pp. 5891–5902, 2008.

J.-W. Xu, P. P. Pokharel, A. R. C. Paiva, and J. C. Príncipe, “Nonlinear component analysis based on correntropy,” in Proceedings of the International Joint Conference on Neural Networks (IJCNN '06), pp. 1851–1855, Vancouver, Canada, July 2006.

S. Liu and H. Jiang, “Study of the conception of the similarity in time series,” Journal of Huazhong University of Science and Technology, vol. 32, no. 7, pp. 75–79, 2004 (Chinese).

The UCR Time Series Data Mining Archive, http://www.cs.ucr.edu/~eamonn/time_series_data/dataset.zip

UCI Machine Learning Repository, http://archive.ics.uci.edu/ml/

N. Saito, “Local feature extraction and its application us ing a library of bases,” in Topics in Analysis and Its Applications: Selected Theses, pp. 265–451, 2000.

S. Manganaris, Supervised classiffication with temporal data [Ph.D. thesis], Computer Science Department, School of Engineering, Vanderbilt University, 1997.

M. W. Kadous, “Learning comprehensible descriptions of multivariate time series,” in Proceedings of the 16th International Machine Learning Conference, pp. 454–463, Morgan Kaufmann, 1999.

J. J. R. Diez and C. A. Gonzalez, “Applying boosting to similarity literals for time series classiffication,” in Proceedings of the 1st International Workshop on Multiple Classifier Systems, pp. 210–219, 2000.