Generalized Framework for Similarity Measure of Generalized Framework for Similarity Measure of Time Series Time Series

. Currently, there is no definitive and uniform description for the similarity of time series, which results in difficulties for relevant research on this topic. In this paper, we propose a generalized framework to measure the similarity of time series. In this generalized framework, whether the time series is univariable or multivariable, and linear transformed or nonlinear transformed, the similarity of time series is uniformly defined using norms of vectors or matrices. The definitions of the similarity of time series in the original space and the transformed space are proved to be equivalent. Furthermore, we also extend the theory on similarity of univariable time series to multivariable time series. We present some experimental results on published time series datasets tested with the proposed similarity measure function of time series. Through the proofs and experiments, it can be claimed that the similarity measure functions of linear multivariable time series based on the norm distance of covariance matrix and nonlinear multivariable time series based on kernel function are reasonable and practical.


Introduction
A complex system typically needs to be described with multiple state variables.These state variables can be obtained by experimental observations and instrumental measures.With these state variables, a set of discrete multivariate time series (MTS) can be constructed.Mathematically, MTS is expressed in matrix form X = ( 1 ,  2 , . . .,   ).Its samples are   (),   ( = 1, 2, . . ., ;  = 1, 2, . . ., ), where  and  denote the number of observation variables and the number of observation samples of each observation variable, respectively.Obviously, X is a  ×  matrix.If  ≥ 2, the matrix X represents MTS.Otherwise, if  = 1, X is simplified to a univariate time series (UTS), which is a special case of MTS and can be denoted as a -dimensional vector ().Time series theory is widely applied in various fields such as electricity, finance, medical, multimedia, meteorology and hydrology, scientific research, and industrial control.Discovery of the hidden information and operating regularity in a time series is a research hotpot in data mining and knowledge discovery.The research on time series includes clustering, classification, similarity search, feature extraction, trend forecasting, and decision support.Similarity measures a fundamental research topic on time series theory.
The contributions in the research on framework for similarity measure of time series are also less.Liu and Jiang [17] proposed a concept of similarity of time series through analyzing the geometric relation of Euclidean distance of time series in high-dimensional space, which describes the similarity relation between two UTS using both similarity function and transform constraint function to establish an exact concept for similarity of time series.
From existing UTS and MTS literatures, the similarity measure methods of time series are undefined and nonstandardized which makes the research very difficult.Thus, it is very necessary to establish a general concept for uniform similarity of time series.In this paper, we give the definition of similarity measure function of UTS with vector norm and proved that the similarity functions defined in the form of norms are equivalent in original UTS space and linear transformed space.Moreover, we also extend the definitions of similarity of time series and present a uniform theory of similarity measure based on set theory, metric space theory, operator theory, matrix theory, and kernel method.The uniform theory based on distance of vector/matrix norm can be used for measuring the similarity of time series in both original and transformed spaces, whether the time series are univariable or multivariable and the linear transform or the nonlinear transform.The theory analysis and experimental results show that the definition of distance of vector/matrix norm is equivalent in original and transformed spaces.
The rest of this paper is organized as follows.In Section 2, it is proved that the vector norm based definitions of similarity functions are equivalent in UTS original and linear transformed spaces.In Section 3, the theory that matrix norm is used for defining the similarity function of linear MTS is discussed.Then, the theory that kernel function is used for defining the similarity function of nonlinear MTS is discussed in Section 4. In Section 5, all similarity functions proposed in this paper are discussed and analyzed.It is proved that the norm based definition of similarity functions for measuring the similarity of UTS is equivalent in time domain and Fourier transform or wavelet transform domain.Also, the similarity function of linear MTS defined based on covariance matrix norm distance and the similarity function of nonlinear MTS defined based on kernel function are analyzed in this section.The experimental results are shown in Section 6.Finally, we conclude this paper in Section 7.

Similarity Measure of Linear Univariable Time Series
For the original data of linear univariable time series, whether they are recoded manually or sampled automatically, it is assumed that they all satisfy Shannon theorem of information theory without any distortion and without considering the data dimension.The linear univariable time series is denoted as set A. Considering two UTS samples ,  ∈ A, wherein  is the time series to be observed (observed time series) and  is the time series to be referenced (referenced time series).They are represented in vector form X = (  ()   ∈ B, and Sim = ( ()  , ) ≤ , the subseries  ()  and  are subseries matching.Obviously, exactly matching is a special case of subseries matching.In this paper, subseries  ()  and similarity measure function Sim = ( ()   , ) are defined for unifying the above two cases of similarity matching of time series and measuring the similarity of  ()   and  in the same dimension conveniently.The following definitions on similarity of UTS are all based on the unified similarity.

Common Similarity Measure Functions.
In current research, the distance function of time series  ()  and , ( ()   , ), is commonly used as the similarity measure function Sim = ( ()   , ).The common distance metrics are as follows [17].

Relevant Concepts of Time Series Transform. Since time series 𝑥 (𝑏)
and  are high-dimension data,  ()  ,  ∈ X ⊂ R  , straightforward analysis and processing of similarity of time series need huge computation burden which is unacceptable on both time and space complexities.Although measuring the similarity of two time series is intuitively straightforward, the result may be not very accurate.Thus, the time series  ()   and  need to be properly transformed.The transformed time series are denoted as  ()   and   , and X  denotes the transformed space,  ()   ,   ∈ X  ⊂ R  , and their transform factor is T. The transform should be lossless or lossy within a very small margin so that no or less accuracy loss of data is introduced by the transform.Through the space transform, the dimension of data is greatly reduced so that the data can be processed with lower complexity.Accordingly, it is very important to select a proper transform for solving this problem.For notational convenience the relevant definition is given as follows.
Definition 2. Let (X, ) and (X  ,   ) be two measure spaces.If there exists a mapping T from X to X  , and ∀,  ∈ X,   (T, T) = (, ), then it can be said that X and X  are isometric, and T is the isometric mapping from X to X  .Definition 3. Let T be an operator (mapping) of the normed linear space from X to X  .If the following hold: (1) additive T( + ) = T + T (,  ∈ ), (2) homogeneity T() = T, then it can be said that T is a linear operator from X to X  .Definition 4. (X, X  ) represents the all bounded linear operators of the normed linear space from X to X  .Let T, T 1 ∈ (X, X  ) and let  be arbitrary number.If ∀ ∈ X and the following hold: (1) additive (T 1 + T) = T 1  + T, (2) homogeneity (T) = (T), then it can be said that (X, X  ) is a linear space.Definition 5. Let X and X  be two normed linear spaces and let T ∈ (X, X  ) be the linear operator from X to X  .If ‖‖ = ‖T‖  and ∀ ∈ X, then it can be said that X and X  are isometric and T is the norm preserving isomorphic mapping from X to X  .It should be noted that the similarity relation does not have the transitive property.For example, father and son are similar, and mother and son are also similar, but father and mother are very possible to be dissimilar.According to the above analysis of similarity of time series, the similarity measure functions are proposed based on the similarity relation of set theory.

Extended Similarity Definition of Time Series.
According to the above discussion, the similarity measure function of time series not only satisfies the similarity relation in set theory, but also should be uniformed in the original space X and the transformed space X  .Objectively, the straightforward similarity measure of two time series is more accurate than nonstraightforward similarity measure in which the geometric triangle inequality is used for the similarity measure through the third-party time series.Thus, Definition 1 is extended as follows.
Definition 7. Given a linear operator  from X to X  and  ()   , ,  ∈ X ⊂ R  ,  ()   ,   ,   ∈ X  ⊂ R  are the transforms of  ()   , ,  by the operator .When the similarity measure function Sim = ( * , * ) ≤  ( > 0 is a given threshold of similarity), the time series  ()   ,  are similar in the constraints of .Meanwhile, the similarity function Sim = ( * , * ) should satisfy the following.
(1) Reflexive symmetrical positive definiteness, that is, Sim = ( ()  , ) = (, Finally, it can be proved that ‖‖ 1 and ‖‖ 2 are equivalent. Theorem 9. Let T be the norm preserving isomorphic mapping from X to X  .The time series  ()   ,  ∈ X ⊂ R  are transformed as  ()   ,   ∈ X  ⊂ R  by T. X has the usual definition of the norm ‖ ⋅ ‖, with which the distance (X, ) can be derived by the norm.Similarly, the distance (X  ,   ) also can be derived by the norm ‖⋅‖.Then, consider the following.

Similarity Measure of Linear Multivariable Time Series
In previous sections, we mentioned the definitions of similarity of univariable time series.The signal variable time series is represented in vector form mathematically.The more similar the two vectors, the shorter the distance between them.The distance of two identical vectors should be zero.Thus, the similarity function of univariable time series should be defined based on vector norm.Multivariable time series is represented in matrix mathematically.So, the similarity function of multivariable time series should be defined based on matrix norm.

Commonly Used Matrix
Norm.The commonly used matrix norms are given as follows: (1) , the maximum sum of absolute element of each column of matrix A, and ‖A‖ 1 is also called the column norm of A; (2) , the maximum sum of absolute element of each row of matrix A, and ‖A‖ ∞ is also called the row norm of A; (3) ‖A‖ 2 = √ max (A  A), wherein A  denotes matrix transpose of A and  max (A  A) is the maximum of absolute eigenvalue of A  A, and ‖A‖ 2 is also called the 2-norm of A, or spectral norm; (4) .Same as vector norm, the equivalence of matrix norms has the following similar conclusions.For ∀A ∈ R × , any matrix norms of A are equivalent.Mathematically, in the above four kinds of norms, ‖A‖ 1 and ‖A‖ ∞ can be easily computed, and ‖A‖ 2 and ‖A‖ 1 have better properties and are widely applied.However, ‖A‖ 2 is more complicated in engineering application and sensitive to the variation of matrix elements.‖A‖  can be computed more easily and thus is widely applied.

Similarity Measure of Nonlinear Time Series
We are inspired by support vector machines (SVMs), where a classifier can convert the difficult nonlinear classification problem in input space X ⊂ R  into simple linear classification problem in feature space H  using kernel method.The essence is that the difficult classification of unclear similarity of the same class of samples and small difference of different classes of samples are converted from the input space into the feature space H  in which the similarity of the same class of samples is enhanced and the difference of different classes of samples is enlarged.Thus, in this paper, the kernel method is introduced for the research of the similarity of nonlinear time series.

Analysis on the Similarity of Linear Transformed Univariable Time
Series.According to operator theory, both Fourier transform and wavelet transform are linear operators.Thus, the relevant conclusions on the similarity of time series based on Fourier transform and wavelet transform are obtained as follows.
Proof.Consider the following. (1) (2) For the above equation, let  = , and then there is Proof.According to Theorem 9 and Lemma 14, the results are directly deduced.
Analogously, the following conclusions are also obtained.Due to the limited space, the proof is not presented here.

Practice Computing Method of Similarity of Linear Multivariable Time Series.
Let matrices A × and B × represent MTS X = (x 1 , x 2 , . . ., x  ) and Y = (y 1 , y 2 , . . ., y  ), and let the covariance matrices between the columns in A × and B × be COV(A) × and COV(B) × .All eigenvalues of covariance matrices COV(A) × are arranged in descending order  1 ,  2 , . . .,   and their corresponding standard orthogonal eigenvectors are  1 ,  2 , . . .,   .Similarly, all eigenvalues of covariance matrices COV(B) × are arranged in descending order  1 ,  2 , . . .,   , and the corresponding standard orthogonal eigenvectors are  1 ,  2 , . . .,   .Thus, the eigenvectors of covariance matrices COV(A) × and COV(B) × are  = [ 1 ,  2 , . . .,   ] and  = [ 1 ,  2 , . . .,   ], respectively, and then their extended similarity function is defined as follows: According to principle component analysis, physical meaning of formula ( 13) is equivalent to that the distance (similarity) between the linear orthogonal transforms of A × and B × can be measured using the norms in the transformed space.Yang and Shahabi [13,14] use the extended Frobenius norm to compute the similarity of MTS which is a special case of formula (13).Formula ( 13) can be understood by referring to the definition of similarity function of univariable time series transformed by linear operator.

Practical Computation of Similarity of Nonlinear Multivariable Time
Series.Mathematically, it is not difficult to prove that the norm of formula (10) derived by the inner product of formula (9) satisfies the Parallelogram formula: = ⟨Φ(X), Φ(X)⟩ − 2 ⟨Φ(X), Φ(Y)⟩ + ⟨Φ(Y), Φ(Y)⟩, it does not need to know the mapping function explicitly.Instead, the kernel function (x, y) is used to compute the similarity function defined in formula (11); that is, Sim =  (X, Y) = √  (x, x) − 2 (x, y) +  (y, y).(14) Formula (14) shows that the nonlinear similarity measure of nonlinear time series in input space can be linearly measured in feature space through kernel method.Currently, commonly used kernel functions are as follows: (1) linear kernel function: (2) -order polynomial kernel function: (3) Gaussian radial basis RBF kernel function: (4) neural network kernel function:

Experimental Results
In our experiments, we take the nonlinear multivariate time series as a typical example without loss of generality for showing the accuracy of the proposed similarity measurement method.The similarity measurement function of formula (1) is used in the input space, and the similarity measurement function of formula (14)  times.The  is increasingly adjusted according to the average classification accuracy of each experiment until the optimal value of  is obtained.In our experiments, five published datasets, Cylinder-Bell-Funnel (CBF), Fish, Face (four) [18], Iris, and Wine [19], are tested.Presently, there were already some researching work on the classification of time series.One of the most common benchmark datasets is CBF [20] which was used by [21][22][23].CBF dataset consists of three time series, Cylinder (), Bell (), and Funnel (), which are generated by the following equations: where  and () are drawn from a standard normal distribution (0, 1),  is an integer drawn uniformly from the range

Conclusion
Based on set theory, metric space theory, operator theory, matrix theory, and kernel method, the definition of similarity of time series is extended and unified theoretically to establish a generalized framework for the similarity measure of time series.In the generalized framework, the similarity of time series is defined as the distance of unified vector/matrix       norm, which is suitable for both time series of univariable and multiple variables in any linear transformed space or nonlinear transformed space.The proposed similarity definition has been proven to be equivalent in original space and transformed space.The experimental results on some published time series datasets confirm that the theoretical deduction on the generality of similarity measure of time series defined in this paper is right.

[ 16 ,
32], and ( − ) is an integer drawn uniformly from the range [32, 96].The three typical curves, Cylinder, Bell, and Funnel, are shown in Figures1(a)-1(c), respectively.The curve of classification accuracy relative to  is shown in Figure1(d)and the classification accuracy rates of KNN and kernel KNN classifiers are also shown in Figure1(e

Figure 4 :
Figure 4: Test results of dataset Iris.

Figure 5 :
Figure 5: Test results of dataset Wine.
Set Theory Definition 6.Let  be a relation in set A. If ∀ ∈ A and (, ) ∈ , then it can be said that relation  is reflexive.If ∀,  ∈ A, (, ) ∈ , and (, ) ∈ , then it can be said that relation  is symmetrical.If ∀, ,  ∈ A, (, ) ∈ , (, ) ∈ , and there is (, ) ∈ , then it is said that relation  is transitive.If a relation  is both reflexive and symmetrical, it is a similarity relation.
U  UA] =   [A  A] = ‖A‖2  .According to definition, we know that ‖A‖  =      A       .With the above results, we have ‖AV‖ 2  =      AV 1/2, wherein A  is conjugate transpose of A. ‖A‖  is called the Frobenius norm (-norm), which is similar to 2norm in the form of vector, and also is compatible with the vector norm ‖‖ 2 .Its advantage in the norm invariance after -norm is multiplied by unitary matrix, that is, the following theorem.Theorem 11.Let A ∈ R × and U, V be -rank and -rank  =   [(UA)  (UA)] =   [A (17)sed in the feature space.The Gaussian radial basis RBF kernel function of formula(17)is used for mapping the nonlinear samples to high-dimension space, which is low complexity compared with polynomial kernel function, especially the high rank kernel function.All samples in the original test and training sets are well mixed.The samples are randomly selected to construct the new test and training sets.Because the KNN classifier can decide a sample belong to the class in which more of  nearest samples are contains.Based on the proposed similarity measure function to verify the equivalence of time series in different spaces, KNN and kernel KNN classifiers are employed to classify the samples in the new sets in input space and feature space, respectively.Each random experiment is carried out 20 times.Then the classification accuracies in the experiments are analyzed and compared.Moreover, the parameter  of Gaussian radial basis of RBF kernel is optimized in the experiments.The  is initialized as a small value first.Subsequently, each random experiment for determining each value of  is repeatedly carried out 20 ).It can be seen from these experimental results that the similarity of time series in original and feature spaces, which are measured with the proposed generalized similarity function defined by the distance of vector or matrix norm, is equivalent regardless of whether the time series are linearly or nonlinearly transformed.The same conclusion is obtained from the experimental results in Figures2, 3, 4, and 5.Moreover, the average classification accuracies of each experiment repeatedly tested 20 times for five datasets are listed in Table1, which also confirm our conclusion.