The aim of this study is to evaluate the filtering techniques which can remove the noise involved in the time series. For this, Logistic series which is chaotic series and radar rainfall series are used for the evaluation of low-pass filter (LF) and Kalman filter (KF). The noise is added to Logistic series by considering noise level and the noise added series is filtered by LF and KF for the noise reduction. The analysis for the evaluation of LF and KF techniques is performed by the correlation coefficient, standard error, the attractor, and the BDS statistic from chaos theory. The analysis result for Logistic series clearly showed that KF is better tool than LF for removing the noise. Also, we used the radar rainfall series for evaluating the noise reduction capabilities of LF and KF. In this case, it was difficult to distinguish which filtering technique is better way for noise reduction when the typical statistics such as correlation coefficient and standard error were used. However, when the attractor and the BDS statistic were used for evaluating LF and KF, we could clearly identify that KF is better than LF.
Recently, the advances of radar rainfall estimates with high spatial and temporal resolution have demonstrated the prospect of improving the accuracy of rainfall inputs for the accuracy of real time flood forecasting. However, the advantage of the weather radar rainfall estimates has been limited by a variety of sources of uncertainty in the radar reflectivity process, including random and systematic errors. There are a lot of discussions on radar rainfall estimation errors [
There are several ways of filtering a signal in one or two dimensions. An example of one which is often applied is low-pass filtering, an operation which removes all components of the power spectrum whose frequency is higher than a chosen threshold. Having this into account, several approaches have been proposed to reduce radar errors. Panofsky and Brier [
These days, much amount of radar rainfall data is being produced, processed, and used. Also the radar rainfall series is widely applied to hydrologic applications such as flash flood forecasting. However, radar rainfall data include noise from many sources and there are lacks of noise reduction studies on the radar rainfall data itself. Therefore, this study analyzes noise of radar rainfall using chaotic dynamics which has nonlinear and aperiodic nature and filtering techniques for investigating radar rainfall characteristics.
To study the nonlinear characteristics of natural phenomena, many statisticians and scientists have suggested the chaos theory which analyze and forecast the nonlinear phenomena of the natural system. Lorenz [
All hydrological measurements are to some extent contaminated by noise. And the noise limits the performance of many techniques of identification, modeling, prediction, and control of deterministic systems [
This study evaluates the noise cancellation capabilities of filtering techniques of low-pass filter (LF) and Kalman filter (KF). To do this, we regenerate chaotic data series and add noise to the series. And then, we perform the noise reduction analysis for the noise added chaotic series by using two filtering techniques and investigate the noise cancellation capabilities of the techniques by the attractor of the series and by the BDS statistic [
The first step in metric analysis of a chaotic time series is the construction of an
The reconstructed state variables
The BDS statistic is derived from the correlation integral and has its origins in the recent work on deterministic nonlinear dynamics and chaos theory. Grassberger and Procaccia [
And
If the data are generated by a strictly stationary stochastic process which is absolutely regular, then this limit exists. In this case the limit is as follows:
When the process is IID, and since
We can consistently estimate the constants
Under the IID hypothesis, the BDS statistic for
Before applying the BDS statistic, the first addressed issue is which region of “
The Kalman filter (KF) was introduced by Kalman’s famous paper describing a recursive solution to the discrete data linear filtering problem [
The system is then measured at discrete points in time, where the measurement vector
They are assumed to be independent (of each other), white, and with normal probability distributions:
In practice, the process noise covariance
The error covariance matrix
The estimate vector
A posteriori state estimate
The matrix
Kim [
The expression for the computation of the a posteriori estimate
To study the effects of noise in a time series, we added Gaussian noise to the time series. Specifically, it considered the noise added time series
May [
Logistic series with the noise levels of 10%, 50%, and 100% (
The attractor of each Logistic series is reconstructed in phase space and the characteristics of the series can be identified (Figure
The attractors with the noise level.
Noise level = 0%
Noise level = 10%
Noise level = 50%
Noise level = 100%
The original Logistic series which has one variable shows its attractor with a simple quadratic form (Figure
This section studies the noise reduction of the noise added Logistic series using LF and KF. Noise cannot be forecasted but statistically estimated and the parameters of LF and KF are calibrated by trial and error method. The constant
Statistical characteristics of Logistic series after applying the filtering techniques.
Low-pass filter | Kalman filter | |||
---|---|---|---|---|
Coefficient of correlation | Standard error | Coefficient of correlation | Standard error | |
|
0.699 | 0.251 | 0.994 | 0.037 |
|
0.629 | 0.273 | 0.986 | 0.058 |
|
0.501 | 0.303 | 0.925 | 0.134 |
Noise removed data series through Low-pass filter and Kalman filter.
Noise level = 10%
Noise level = 50%
Noise level = 100%
The attractors for noise removed Logistic series by LF and KF are reconstructed in phase space (Figure
The noise removed attractors by LF and KF.
L.F.: noise level = 10%
L.F.: noise level = 50%
L.F.: noise level = 100%
K.F.: noise level = 10%
K.F.: noise level = 50%
K.F.: noise level = 100%
The BDS statistic was applied for testing for nonlinearity of each data series. Not only is it useful in detecting deterministic chaos, but it also serves as a residual diagnostic. If the model (null hypothesis) is correct, then the estimated residuals will pass the test for IID (independently and identically distributed). A failure to pass the test is an indication that the selected model is misspecified. Here the confidence interval (C.L.) of 95% which is a significance level of 5% is used for the randomness test of a time series. The original series, noise added series, and noise removed series of Logistic map are analyzed by the BDS statistic for their randomness and nonlinearity. And the results are shown in Table
The BDS statistic values for data series in each case.
|
|
|
The series with noise | Low-pass filter | Kalman filter | 95% C.I | ||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
|
|
|
|
|
|
| ||||
2 | 0.5 | 487.3 | 535.4 | 33.3 |
|
266.4 | 29.6 |
|
446.0 | 369.0 | 160.2 |
|
1 | 201.9 | 187.6 | 16.1 |
|
151.9 | 16.4 |
|
200.0 | 184.4 | 72.0 | ||
1.5 | 11.1 | 10.6 | 3.4 |
|
12.6 | 5.3 |
|
11.6 | 12.7 | 15.9 | ||
2 | −20.3 | −19.7 | −4.0 |
|
−10.2 |
|
|
−18.1 | −14.9 | 7.2 | ||
|
||||||||||||
3 | 0.5 | 650.9 | 720.8 | 38.6 |
|
361.2 | 34.6 |
|
612.5 | 529.7 | 256.7 |
|
1 | 182.6 | 171.4 | 15.4 |
|
139.6 | 15.3 |
|
187.5 | 177.7 | 77.8 | ||
1.5 |
|
2.0 | 2.9 |
|
3.3 | 4.2 |
|
3.8 | 6.4 | 14.9 | ||
2 | −17.6 | −16.9 | −3.4 |
|
−9.2 |
|
|
−12.9 | −10.0 | 6.6 | ||
|
||||||||||||
4 | 0.5 | 856.5 | 957.6 | 40.4 |
|
477.6 | 37.9 | 2.3 | 807.1 | 717.9 | 380.1 |
|
1 | 174.2 | 164.3 | 14.3 |
|
134.3 | 14.4 | 2.1 | 176.3 | 167.0 | 74.3 | ||
1.5 |
|
|
|
|
|
3.7 |
|
|
2.5 | 11.1 | ||
2 | −15.3 | −14.8 | −3.0 |
|
−8.6 |
|
|
−10.5 | −8.1 | 6.0 | ||
|
||||||||||||
5 | 0.5 | 1173.9 | 1326.4 | 42.3 |
|
656.7 | 40.7 | 2.6 | 1101.8 | 1000.8 | 557.7 |
|
1 | 168.4 | 161.5 | 13.7 |
|
134.5 | 13.7 | 2.1 | 173.6 | 162.8 | 74.3 | ||
1.5 |
|
|
|
|
−0.8 | 3.3 |
|
|
|
8.6 | ||
2 | −13.6 | −13.5 | −2.9 |
|
−8.2 |
|
|
−9.2 | −7.2 | 5.1 |
Radar rainfall is a representative hydrologic data which includes noise from many sources. This study uses the radar rainfall obtained from the radar in Biseul Mountain radar (BSL radar) in Gyeongbuk province, Korea. The radar rainfall series in Gamcheon watershed especially which is produced in BSL radar is used for analyzing the series characteristics according to noise cancellation by LF and KF. BSL radar was constructed in 2009 and it is dual polarization radar. The radar has temporal and spatial resolutions of 2.5 min and 250 m × 250 m. Therefore, BSL radar rainfall series of 2.5 min-time interval is obtained with the data period of 6/24/2011 09:00–6/26/2011 11:00 (about 3000 min; average: 1.73 mm, standard deviation: 1.53 mm).
The ACF of radar rainfall series was exponentially decreased and so the delay time was selected as
The time series (a) of radar rainfall and its attractor (b).
This section applies LF and KF for the noise reduction study of radar rainfall series and the constant
Statistical characteristics of radar rainfall after applying the filtering techniques.
Low-pass filter | Kalman filter | ||
---|---|---|---|
Coefficient of correlation | Standard error | Coefficient of correlation | Standard error |
0.994 | 0.156 | 0.989 | 0.231 |
The raw data and noise removed series of radar rainfall.
The attractors for noise removed radar rainfall series by LF and KF are reconstructed in phase space (Figure
The attractors of noise removed radar rainfall series by LF and KF.
Low-pass filter
Kalman filter
The original radar rainfall series and noise removed series by LF and KF are analyzed by the BDS statistic for their randomness and nonlinearity. And the results are shown in Table
The BDS statistic values for radar rainfall and the noise removed series.
|
|
Radar rainfall | Low-pass filter | Kalman filter | 95% C.I |
---|---|---|---|---|---|
2 | 0.5 | 155.1 | 168.1 | 185.3 |
|
1 | 254.3 | 277.3 | 314.0 | ||
1.5 | 445.9 | 492.5 | 572.4 | ||
2 | 839.8 | 943.4 | 1125.1 | ||
|
|||||
3 | 0.5 | 139.0 | 149.5 | 167.0 |
|
1 | 173.9 | 186.3 | 210.5 | ||
1.5 | 222.5 | 238.8 | 273.5 | ||
2 | 295.2 | 318.6 | 370.3 | ||
|
|||||
4 | 0.5 | 96.8 | 105.3 | 116.1 |
|
1 | 105.6 | 112.9 | 125.2 | ||
1.5 | 115.5 | 122.5 | 136.4 | ||
2 | 128.6 | 135.9 | 152.5 | ||
|
|||||
5 | 0.5 | 77.3 | 82.9 | 89.6 |
|
1 | 77.3 | 81.0 | 87.5 | ||
1.5 | 76.8 | 79.7 | 85.8 | ||
2 | 77.1 | 79.7 | 85.7 |
This study investigated the filtering techniques for removing the noise involved in Logistic series and radar rainfall. The chaotic dynamics and the BDS statistic were used for analyzing the time series which are associated with noise. Logistic series with noise level were used for evaluating the filtering techniques of LF and KF. The analysis for the evaluation of LF and KF was performed by phase space reconstruction and the BDS statistic from chaos theory. As the noise level is increased, the characteristics of Logistic series were becoming random and this phenomenon was also occurred in the attractors and the BDS statistic analysis. The applications of LF and KF to the noise added Logistic series showed that KF reduced noise more clearly involved in the Logistic series than LF.
The noise in radar rainfall series was removed by LF and KF. Then the attractor and the BDS statistic were used for evaluating the filtering techniques. It was difficult to distinguish which filtering technique is better when the correlation coefficient and standard error were used for evaluating LF and KF. However, the attractor and the BDS statistic gave us more clear answers for the determination of the proper filtering technique. In this study, we have shown that KF is better technique than LF and chaos theory can be applied for investigating the characteristics of the time series.
The authors declare that there is no conflict of interests regarding the publication of this paper.
This study was conducted with financial support from the Korean Institute of Civil Engineering and Building Technology’s Strategic Research Project (Operation of Hydrological Radar and Development of Web and Mobile Warning Platform). Also, this work was supported by the National Research Foundation of Korea Grant funded by the Korean Government (NRF-2009-220-D00104).