Nonlinear Partial Least Squares for Consistency Analysis of Meteorological Data

Considering the different types of error and the nonlinearity of the meteorological measurement, this paper proposes a nonlinear partial least squares method for consistency analysis of meteorological data. For a meteorological element from one automated weather station, the proposed method builds the prediction model based on the corresponding meteorological elements of other surrounding automated weather stations to determine the abnormality of the measured values. For the proposed method, the latent variables of the independent variables and the dependent variables are extracted by the partial least squares (PLS), and then they are, respectively, used as the inputs and outputs of neural network to build the nonlinear internal model of PLS. The proposed method can deal with the limitation of traditional nonlinear PLS whose inner model is the fixed quadratic function or the spline function. Two typical neural networks are used in the proposed method, and they are the back propagation neural network and the adaptive neuro-fuzzy inference system (ANFIS). Moreover, the experiments are performed on the real data from the atmospheric observation equipment operationmonitoring system of Shaanxi Province of China.The experimental results verify that the nonlinear PLS with the internal model of ANFIS has higher effectiveness and could realize the consistency analysis of meteorological data correctly.


Introduction
Meteorological observation is the premise for the correctness of weather analysis, forecast, and severe weather warning [1].For many meteorological factors, there is not any consistent methodology used for the related measuring instruments.Therefore, the consistency analysis is of important practical motivation for the purposes of scientific research and resource management of the meteorological data [2].
Consistency analysis of meteorological data is based on the continuity and the uniformity of the distribution [3].For determining the abnormality of the meteorological data, the observation data of an automated weather station are compared with the prediction values, which are calculated according to the corresponding meteorological data of other surrounding automated weather stations [4].The consistency analysis methods of meteorological data include the spatial interpolation algorithm, the spatial regression test, the Madsen-Allerupt approach, and the climate statistical comparison method [5][6][7][8][9].These methods consider that there exists the linear relationship for the space distribution of meteorological elements; namely, the measured values of a meteorological element of some automated weather stations, whose space location is near, have greater similarity.Since most of the meteorological measurement processes are nonlinear, the adequacy of these linear methods would be affected.A nonlinear autoregressive neural network is presented for consistency analysis of meteorological data and the approach has learning capacity of nonlinear dependencies from a large volume of potentially noisy data [10].Nevertheless, because of the black-box of the neural network, the understandable heuristic knowledge could not be provided [11].Partial least squares (PLS) is a classical regression method [12].PLS synthetically extracts the information for the independent variables and determines the latent variables which have the best interpretation capability.PLS has been widely used in many different domains [13][14][15][16].The core idea of PLS is a kind of linear regression [17].PLS with nonlinear internal model, which uses a polynomial or spline nonlinear function as the internal model, is proposed to improve regression accuracy [18].The forms of the polynomial and spline function are restricted, and the neural network (NN) can The flowchart of the proposed method.
approximate a nonlinear function with arbitrary precision [19].Hence, using NN to build the internal model of PLS would reduce the residual and have better effectiveness.
In this paper, a nonlinear partial least squares method for consistency analysis of meteorological data is proposed.For a meteorological element from one automated weather station, the proposed method builds the prediction model based on the corresponding meteorological elements of other surrounding automated weather stations to determine the abnormality of the measured values.For the proposed method, the latent variables of the independent variables and the dependent variables are extracted by PLS, and then they are, respectively, used as the inputs and outputs of neural network to build the nonlinear internal model of PLS.Two typical neural networks are used in the proposed method, and they are the back propagation neural network and the adaptive neuro-fuzzy inference system.Experiments are performed on the real data from the atmospheric observation equipment operation monitoring system of Shaanxi Province of China.The organization of this paper is as follows.The proposed method is presented in detail in Section 2. In Section 3, the experimental results are discussed.Finally, Section 4 concludes the paper.

The Proposed Method
For the consistency analysis of meteorological data, Y represents the prediction value of a meteorological element of an automated weather station, and X represents the measured values of the corresponding meteorological element of surrounding automated weather stations.The flowchart of the proposed method is shown in Figure 1, where t is the principal component of X, u is the principal component of Y, E is the residual matrix of X, and F is the residual matrix of Y.
The steps of the proposed method are explained as follows.
Step 1. Initialize u, and u = y  , where y  is the th column vector of Y.
Step 2. Calculate w, which is the weight vector of X, and w  = u  X/(u  u).
Step 3. Normalize w, and w  new = w  old /‖w  old ‖.
Step 5. Calculate c, which is the weight vector of Y, and c  = t  Y/(t  t).
Step 6. Normalize c, and c  new = c  old /‖c  old ‖.
Step 8.If the change of t at the th iteration is less than or equal to the threshold, then go to the next step; otherwise, go to Step 2.
Step 9. Calculate p, which is the loading matrix of X, and p  = t  X/(t  t).
Step 10.Calculate q, which is the loading matrix of Y, and Step 11.Use t and u as the inputs and outputs of NN to build the nonlinear internal model of PLS.For training process, the objective function is min J = min ‖u − g out (t)‖ 2 , where g out (t) is the output of NN for a pair of t and u.
Step 12. Calculate the residual matrices for the th principal component, and where E 0 = X and F 0 = Y.
Step 13.Let X = E and Y = F and calculate the next principal component until the rank of X is zero.Then, output the results.
Since the proposed method retains the external model of PLS and uses NN to build the internal model, it has the robust capability of PLS and the adaptive learning capability of NN.The data are mapped by the external characteristics of PLS and then used for training the NN; namely, the multivariate modeling is decomposed.Hence, for the proposed method, the related information from data is removed and the number of network independent weights is reduced.The noisesensitiveness of NN and the local minima problem could be avoided [20].
For the proposed method, two typical neural networks are used for the internal model.They are the back propagation neural network (BPNN) and the adaptive neuro-fuzzy inference system (ANFIS).BPNN always includes three layers, which are the input layer, the hidden layer, and the output layer [21][22][23].The transfer function of the hidden layer is usually a tangent s-type function, and the transfer function of the output layer is purelin function.For BPNN, a neuron is an activation function containing weights and bias parameters.The number of neurons in hidden layer is usually determined by the expert knowledge.Moreover, BPNN adopts a back propagation algorithm to train the parameters of activation functions.ANFIS is one of the most commonly used learning systems for the Tagaki-Sugeno fuzzy rule [24].ANFIS adopts the subtractive clustering algorithm to determine the initial rules and then uses a neural network with fixed five layers for tuning the rule parameters [25,26].In layer 1, each node function is the membership values of each input with respect to its linguistic term, and the standard Gaussian function is used as the membership functions.In layer 2, each node plays the role of a simple multiplier, and the output of each node represents the firing strength of the rule.In layer 3, each node calculates the ratio of the activation level of a rule to the total of all activation levels.In layer 4, each node calculates the contribution of the overall output; namely, it is simply the product of the normalized firing strength and the function of consequent, which is a first order polynomial.Layer 5 computes the overall output as the summation of all input, and it is the final output.In the next section, the experimental results will estimate the effectiveness of the proposed method.

Experimental Results
In the section, we perform BPNN, ANFIS, PLS, the nonlinear PLS with the internal model of BPNN (NPLSB), and the nonlinear PLS with the internal model of ANFIS (NPLSA) to build the prediction models for the atmospheric pressure and the air temperature.The measured values of the atmospheric pressure and the air temperature used in the experiments are obtained from five automated weather stations of Shaanxi Province of China.The five automated weather stations are the Xi'an station (XiA), the Lin Tong station (LinT), the Xian Yang station (XianY), the Lan Tian station (LanT), and the Jing Yang station (JingY).LinT, XianY, LanT, and JingY  1.
For the experiments, the meteorological elements of LinT, XianY, LanT, and JingY are the input variable and that of XiA is the output variable.For BPNN, the three-layer classical structure is adopted.Since the number of the input variables and the output variables is 4 and 1, respectively, the number of nodes of the input layer, the hidden layer, and the output layer of BPNN is 4, 5, and 1, respectively.The learning rate of BPNN is 0.01.For ANFIS, the number of the input variables and the output variables is 4 and 1.The initial step size is 0.01; the step size decrease rate is 0.7 and the step size increase rate 1.3.For the subtractive clustering algorithm in ANFIS, the radii value is 0.3.Moreover, for BPNN and ANFIS, the number of iterations is 100 and the training error goal is zero.For NPLSB, the number of the hidden layer nodes is still 5, and the other parameters are the same as those of BPNN.The related parameters of NPLSA are the same as those of ANFIS.For PLS, NPLSB, and NPLSA, the number of latent variables is determined according to the root-mean-squared error of leave-one-out cross validation (RMSECV), which is where ŷcv and  cv are the prediction values and the measured values of the cross validation set, respectively, and  cv is the number of samples in the cross validation set.
To estimate the effectiveness of the several methods, RMSECV, the root-mean-squared error of prediction (RMSEP), the squared correlation coefficient of cross validation ( 2 cv ), and the squared correlation coefficient of prediction ( 2  ) are adopted in the experiments.RMSEP is defined as where ŷ and   are the predicted values and the measured values of the validation set, respectively, and   is the number of samples in the validation set. 2 cv is defined as where Cov(⋅) is the covariance operation and (⋅) is the variance operation. 2  is defined as In addition, the experiments are implemented in MAT-LAB 8.3.0.532.The running environment is a generalpurpose personal computer with an Intel i5-3570 CPU and The experimental results of the atmospheric pressure are shown in Table 2.Although the RMSECV value of BPNN is the smallest and  2 cv of BPNN is the largest, the RMSEP value of BPNN is larger than those of PLS, NPLSB, and NPLSA.The effectiveness of BPNN is worse.The RMSEP value of ANFIS is the largest, and the effectiveness of ANFIS is the worst.Since the RMSEP value of NPLSB is smaller than that of BPNN, integrating BPNN with PLS could improve the prediction capability of BPNN.However, the RMSECV value and the RMSEP value of PLS are smaller than those of NPLSB, and the internal model of NPLSB would be premature.The RMSEP value of NPLSA is the smallest, and NPLSA has the advantages of learning optimization ability of NN and the humanlike thinking of fuzzy logic technique.The  experimental results verify that the effectiveness of NPLSA is the highest for the atmospheric pressure.Figure 3 shows the scatter diagrams of measured values versus predicted values of BPNN, ANFIS, PLS, NPLSB, and NPLSA for the atmospheric pressure.For BPNN and ANFIS, a few samples are far away from the diagonal line.The points of PLS and NPLSB are mainly distributed on both sides of the diagonal line.Almost all of the points of NPLSA are in the diagonal line, and the prediction capability of NPLSA is better.
The experimental results for the air temperature are summarized in Table 3.For the RMSECV value and the RMSEP value of ANFIS being the largest, the effectiveness of ANFIS is the worst.Although the RMSECV value of BPNN is the smallest, the RMSEP value is still larger.The RMSEP value of NPLSB is smaller than that of BPNN and is larger than that of PLS.Hence, using BPNN to build the internal model of PLS may enhance the prediction capability to a certain extent.The RMSECV value and the RMSEP value of NPLSA are both the smallest.Therefore, the NPLSA model is still more accurate.Figure 4 shows the scatter diagrams of measured values versus predicted values of BPNN, ANFIS, PLS, NPLSB, and NPLSA for the air temperature.Almost all of the points of BPNN and ANFIS are far away from the diagonal line.For NPLSB, a few pieces of data are far away from the diagonal line.The points of PLS and NPLSA are mainly distributed on both sides of the diagonal line.Moreover, the points of NPLSA are closer than those of PLS, and the prediction capability of NPLSA is still better.
In summary, the experimental results verify that NPLSA has higher predictive capability for the atmospheric pressure and the air temperature.Using the method for consistency analysis of meteorological data, if the measured values deviating from the prediction values exceed a preset threshold, the measured values would be labeled as abnormality.Hence, NPLSA could realize the consistency analysis of meteorological data correctly.

Conclusions
In the paper, a nonlinear partial least squares method for consistency analysis of meteorological data is proposed.The proposed model has some advantages as follows.First, the proposed method could realize the consistency analysis of meteorological data.Second, the proposed method integrates the robust capability of PLS and the adaptive learning capability of NN to predict the meteorological data correctly.Third, the multivariate modeling is decomposed with the data being mapped by the external characteristics and used for training the NN.Fourth, the noise-sensitive and the local minima problem of NN could be avoided to a certain extent.BPNN and ANFIS are used for building the internal model of PLS.The experimental results also verify the effectiveness of NPLSA is high.Since the performance of the proposed method may be affected by the training process, in the future research work, some advanced schemes would be adopted to further improve the training process.

F
The next principal componentThe next principal component

Figure 2 :
Figure 2: The locations of the automated weather stations.

Table 1 :
The test set for the atmospheric pressure and the air temperature.

Table 2 :
The experimental results of the atmospheric pressure.

Table 3 :
The experimental results of the air temperature.