Error Analysis and Evaluation of the Latest GSMap and IMERG Precipitation Products over Eastern China

1School of Civil Engineering, Hefei University of Technology, Hefei 230009, China 2Interdisciplinary Centre for River Basin Environment, University of Yamanashi, Kofu 400-8511, Japan 3Bureau of Hydrology (Center of Water Resources Information), The Ministry of Water Resources of the People’s Republic of China, Beijing 100053, China 4Groundwater Monitoring Center, The Ministry of Water Resources of the People’s Republic of China, Beijing 100053, China 5Department of Civil and Earth Resources Engineering, Kyoto University, Kyoto 615-8540, Japan


Introduction
An accurate estimation of precipitation is essential to provide useful information for meteorological and climatological research, water resource management, and natural disasters (e.g., drought and flood) management caused by extreme rainfall events [1].Current unevenly and sparsely distributed rain-gauge networks, especially in developing countries, face severe challenges while obtaining reliable spatiotemporal precipitation information.By contrast, satellite meteorology has remarkably contributed to estimating spatially continuous precipitation on a global basis by various visible/thermal infrared (Vis/IR) and passive microwave (PMW) sensors [2][3][4][5][6][7].Till date, numerous sorts of satellite precipitation retrieving algorithms and products (e.g., TRMM Multisatellite Precipitation Analysis product (TMPA), Precipitation Estimation from Remote Sensed Information Using Artificial Neural Networks-Cloud Classification System (PERSIANN-CCS), Global Satellite Mapping of Precipitation product (GSMap), and Climate Prediction Center (CPC) morphing technique product (CMORPH)) have been developed and generated over the past decades [8][9][10][11][12][13].
As TRMM's successor, the Global Precipitation Mission (GPM) which is managed by the American National Aeronautics and Space Administration (NASA) and the Japanese Aerospace Exploration Agency (JAXA) has provided the latest high spatiotemporal resolution global precipitation products since March 2014.Compared to previous TRMM instruments, the GPM Microwave Imager (GMI) and the Dual-Frequency Precipitation Radar (DPR) operated by the overarching goal is to provide basic accuracy information about the GSPPs over eastern China for potential users who wish to use these products in their research, such as water balance analysis, water resource management, and hydrometeorological modeling, and we anticipate that these products will prove useful for subsequent investigations.A two-year data sequence (April 2014 to March 2016) was selected for this study, during which both the satellite data and reference data were available.

Study Area and Data
2.1.Study Area.Previous studies have indicated that most satellite precipitation products show significant differences between the western and eastern parts of China and application of satellite precipitation products over the western China is difficult [21,22].In this study, we mainly focus on the eastern part of China, including eight first level river basins (Figure 1).The study area has a typical humid/semihumid climate and is under control of tropical, subtropical, and temperate monsoon, with annual precipitation more than 400 mm.Concentrating period of precipitation over this region is from May to September.In addition, this region is the most economically active region in China, about 95% of the Chinese population lives in this area, and it comprises 56.1% of the total land area and 73% of the national GDP. Figure 1 and Table 1 provide details about the study region and eight main river basins selected for this study.

Ground Reference Data.
The ground reference data named China daily Precipitation Analysis Product (CPAP) was obtained from the National Meteorological Information Center and China's Meteorological Administration (CMA).This product with a 0.25 ∘ spatial resolution is used as a reference to evaluate the GSPPs.About 2419 rain-gauge observations (as shown in Figure 1) were interpolated with the climatology-based Optimal Interpolation (OI) technique to yield the gridded analysis product.All the gauge data used in CPAP have undergone strict quality control in three levels including the extreme values' check, internal consistency check, and spatial consistency check [19].CPAP has been systematically validated by Shen and Xiong [23] and found that the daily analysis has very good agreements with the observations over different regions of China.It has been    [16].Readers can refer to Huffman et al. [25,26] and Hou et al. [6] for more detailed introduction about IMERG products.At the time when the GPM mission was launched, GSMap project developed a corresponding GPM-era precipitation retrieval algorithm (GPM-GSMap Version 06) by adding information from GPM Core GMI.The GSMap algorithm generates precipitation estimates by following steps: first, calculating the rainfall rate from PWM sensors; then, propagating rainfall affected area using forward and backward morphing technique; finally, refining the estimated data based on infrared brightness temperature by Kalman filter approach [10,[27][28][29].The GSMap algorithm also includes the Japan Meteorological Agency Global Analysis (JMA-GANAL) data and Merged Satellite/in situ Global Daily Sea Surface Temperatures (MGDSST) data as inputs [15].The GSMap-Gauge dataset is one error-corrected GSMap product which is based on GSMap-MVK (a pure satellitebased GSMap product without correction by gauge data) and adjusted by the CPC global daily gauge data analysis [30,31].In this article, we also use "GSMap-gauged" to stand for GSMap-Gauge data.
Both IMERG and GSMap-gauged products with native 0.1 ∘ spatial resolution are aggregated into 0.25 ∘ /daily spatiotemporal resolutions to keep consistent with the ground reference dataset (CPAP).In addition, the gauge analysis used to adjust and correct the satellite estimates, which are the CPC Unified Daily Gauge Analysis for the GSMap-gauged and the GPCC monthly gauge analysis for the IMERG, may contain the same gauge stations which are also used to generate CPAP.Results of the comparison between the GSMap and the IMERG with the ground reference data (CPAP) are therefore affected by the similarities between the two gauge analyses used to adjust the satellite estimates and the reference dataset.We provide the detailed gauge station density distribution for the CPAP, CPC, and GPCC data in the Supplementary File (Figures A1-3) in Supplementary Material available online at https://doi.org/10.1155/2017/1803492.Table A1 shows information of gauge stations used for generating or adjusting the three products and the same stations numbers between CPAP and CPC, GPCC products for eight river basins.There are 348 and 582 stations used in CPC and GPCC data over the study area; among them, 195 and 129 stations are the same with the stations used in CPAP data for CPC and GPCC data, respectively.The grids which cover these same stations are not used for comparison; only the data pairs over a grid where there is reporting stations are used in calculating comparison statistics which are shown in Figures 5,6,7,11,12,and 13, as well as Table 6.

Validation and Extreme Precipitation Metrics.
A set of widely used traditional statistical metrics were selected to quantitatively evaluate the performance of GSPPs in this study.Two continuous verification metrics which measure the accuracy of a continuous variable (i.e., precipitation volume) were used: Correlation Coefficient (CC) and Mean Error (ME).The CC was used to check the degree of linear correlation between GSPPs and observations.The ME describes the average magnitude of the error.In addition, a contingency table was used to evaluate the capacity of GSPPs in monitoring the occurrence of precipitation events (as shown in Table 3).Hit stands for a case that observed precipitation is detected by satellite product correctly.Missed is the case that observed precipitation is not detected by satellite product.False case means when the observation is not precipitation but satellite product is.  ,   , and   are the times of occurrence of the corresponding cases, respectively.Based on Table 3, two categorical verification metrics which measure the correspondence between the estimated and observed occurrences of precipitation events were selected: the Probability of Detection (POD) and False Alarm Ratio (FAR).POD measures the fraction of occurred precipitation events that are correctly detected by satellite product.FAR measures the fraction of events in which satellite detects precipitation but precipitation does not occur [23,32,33].Table 4 lists the validation metrics used in this study.
Besides, six extreme precipitation indices were used to evaluate the performance of the two GSPPs in detecting precipitation extremes over eastern China.They are divided into three categories: percentile (RR99p and RR95p), absolute threshold (R20 and R20TOT), and duration (CWD and CDD).All these indices are defined by the Expert Team on Climate Change Detection and Indices (ETCCDI) which is sponsored by the Commission for Climatology of the World Meteorological Organization's World Climate Research Program [16,[34][35][36].The indices are listed in Table 5.

Error Decomposition.
The error decomposition method used to evaluate the error components of satellite precipitation estimates is first proposed by Tian et al. [33,37].This method breaks down the total precipitation error  (total bias) into three independent parts: hits error (), missed precipitation (−), and false precipitation ().The total bias is calculated by satellite-based precipitation minus observed precipitation.The three decomposed components can be obtained by subtracting observed precipitation from the satellite precipitation estimate for hit, missed, and false alarm events, respectively.The description of hit, missed, and false alarm events have already been presented in Table 3.The relation of these four error type can be described by the following equation:  =  −  + .This approach of decomposing the total bias has been proved that it could improve our understanding and recognizing of the error characteristics of satellite precipitation products.

Error Components Analysis.
In this section, we analyze the error characteristics for both satellite precipitation products at the daily scale and 0.25 ∘ resolution.For each product, we broke down the total bias for each day over the eastern China region and then aggregated the daily error components and total errors to whole study period and seasonal time scale (two summer periods and two winter periods).Figure 2 shows the spatial distributions of error components for the study period.The two products have totally different spatial patterns.For GSMap-gauged, total bias indicates that this product underestimates precipitation over most of the southern China region.This underestimation might have

Statistic metric
Formula Perfect value (range)

CWD (day)
The largest number of consecutive days during which all daily precipitation amount is above 1 mm

CDD (day)
The largest number of consecutive days during which all daily precipitation amount is below 1 mm resulted from the hit bias which shows obvious underestimation over southern China, as shown in Figure 2(b).Missed precipitation is not as obvious as false precipitation but has distinct influence over the south part of Pearl River basin (Figure 2(c)).The slight overestimation at Huaihe, Haihe, middle, and lower reaches of Yellow River basin is mainly caused by false precipitation (Figure 2(e)).For IMERG, total bias illustrates significant overestimations over most of the eastern part of China, except upper reaches of Yangtze and Yellow River basins (Figure 2(e)).This overestimation at eastern part of China is the comprehensive impact of hit bias and false precipitation, although missed precipitation may counteract some of this impact.Besides, missed precipitation is more evident in case of IMERG than GSMap-gauged and underestimations in total bias of IMERG are mostly caused by missed precipitation, especially over upper reaches of Yangtze and Yellow River basins (Figure 2(g)).The spatial patterns of the error components, averaged for the summer (June, July, and August) and winter (December, January, and February), are shown in Figures 3 and 4, respectively, to look into seasonal variations of error components distribution.
In the case of the summer, the two satellite products have similar hit bias and total bias over most part of study area, suggesting that the most significant error contribution is from hit events (Figure 3).Regarding the GSMapgauged, underestimation is dominant and spatial pattern of underestimation in total bias is very similar with that in hit bias, with less contribution from missed precipitation.Slight overestimation in total bias over Huaihe and lower reaches of Yangtze River basins is related to a significant amount of hit bias (Figure 3(b)), with some contribution from false precipitation events (Figure 3(d)).For the IMERG, however, the positive total bias over the eastern China is mostly due to hit bias.This overestimation in hit bias may be caused by the inability to capture warm rain processes or short-lived convective storms at lower latitudes [38].In addition, the obvious underestimation in total bias over upper reaches of Yellow and Yangtze River basins is mainly cause by the hit bias (Figure 3(f)) and missed precipitation (Figure 3(g)), the possible reasons are that, first, the IMERG algorithm may miss warm and orographic precipitation in this mountainous regions; second, the CPCC data used to adjust IMERG may also have significant error in this region because of few gauge stations here used for interpolation and unsatisfactory treatment of orographic effects here.Conversely, missed precipitation is not a significant error source for the GSMapgauged data (Figure 3(c)).Figures 3(d) and 3(h) demonstrate that false precipitation of GSMap-gauged is more pronounced than that of IMERG in summer.
During the winter season, both products showed similar spatial distributions for the total bias and hit bias with the results for the summer season.The significant difference exists in missed precipitation, where comparing with the summer season the GSMap-gauged clearly missed more events in the south part of Pearl River basin (Figure 4(c)); this may be associated with low-level cloudiness and warm rain processes in the winter that may not have the distinct characteristics of ice particles.Meanwhile, IMERG missed more events in the upper reaches of Yangtze and Yellow River basins and southern part of China (Figure 4(g)); this may be attributed to PMW's inability to measure snowfall/rainfall over snow and ice covered land surfaces.Regarding GSMapgauged data, overestimation over the southern China are mainly determined by false precipitation, while for the IMERG, the primary determining error factor is hit bias.It is also noted that false precipitation for both products is more obvious in winter.
Elevation may also be a major impact factor in the error distribution of the satellite precipitation data.Therefore it is useful to identify the total bias and each error component with respect to elevation over eastern China, as shown in Figure 5.We found that for both products, the impact of elevation was totally different.The error components of   IMERG show an obvious topographic dependency, but such feature is not very obvious in GSMap-gauged.The total bias of GSMap-gauged keeps the similar tendency with that of hit bias and is all negative.This indicates that GSMap-gauged tends to underestimate precipitation over most regions of eastern China.For the IMERG, total bias is positive when elevation is less than 750 meters and changes to be negative with elevation larger than 750 meters due to the contribution of hit bias and missed precipitation which increase negatively with the rising of the terrain.In addition, there is more missed precipitation for the IMERG than the GSMap-gauged product, which can also be seen in Figures 2-4(c, g).From the topographic analysis of error components, it can be concluded that the GSMap algorithm and error-correction method are more effective in balancing the errors related to topography.
To better explore the feature of error components of the two products, the intensity distributions of total precipitation as well as precipitation amount of hit, missed, and false precipitation events are shown in Figure 6.These intensity distributions are calculated as the ratio of sum of precipitation amounts in each bin to the total sum of observed precipitation amount over the total study period.Both datasets show different distributions of the total precipitation, as shown in Figure 6(a).GSMap-gauged overestimates light precipitation (<16 mm/day) but underestimates more precipitation with rain rate larger than 32 mm/day, whereas although IMERG gets close to observation for light precipitation (<8 mm/day), it underestimates precipitation at rain rate between 8 and 64 mm/day and significantly overestimates precipitation at rain rate more than 64 mm/day.Distributions of hit precipitation of the two products look similar as these of total precipitation, which proves that hit bias plays the major role in total bias.Figure 6(c) shows that missed precipitation is almost located in range of 1 to 64 mm/day for both products; IMERG has more missed precipitation than GSMap-gauged, which is also reflected by the spatial distribution shown in Figures 2(c) and 2(g).In addition, GSMap-gauged has a narrow distribution of false precipitation, concentrating at precipitation with rain rates less than 64 mm/day, as compared to IMERG.False precipitation for GSMap-gauged is larger than its missed precipitation, while for IMERG the situation is quite the reverse.In summary, the breakdown of errors at spatial better enabled us to identify the error sources and their contributions to the total error.

Extreme Precipitation Events
Comparison.Density-colored scatter diagrams of extreme precipitation indices between GSPPs and observations over eastern China are presented in Figure 7.For two percentile indices (RR99P and RR95P), both satellite products have good correlation with observation with high CC values (all greater than 0.85).It is obvious that IMERG tends to overestimate extreme precipitation with relative bias (RB) values of RR99P more than 17%.But the condition is almost opposite for GSMap-gauged which underestimates RR99P significantly (with RB lower than −10%).This may be explained by probability density function shown in Figure 6(a) that intensity distribution of total precipitation for IMERG (GSMap-gauged) is higher (lower) than that for observation at precipitation rate more than 64 mm/day.For RR95P, both products have high CC values (all more than 0.90) but with negative RB.This may also be explained by probability density function shown in Figure 6(a) that intensity distributions of total precipitation for both products are lower than that for observation at precipitation rate between 16 and 64 mm/day.Regarding absolute threshold indices (R20 and R20TOT), results are similar with percentile indices for both products.In addition, it should be noted that the CC (RMSE) values of six indices for GSMap-gauged are almost higher (lower) than that for IMERG.This implies that the ability of GSMap-gauged in detecting extreme precipitation is relatively more stable than IMERG.For two duration indices (CWD and CDD), IMERG shows poor results with low CC value and high RMSE values.Figure 7(e) shows that GSMap-gauged overestimates CWD significantly (RB more than 20%).This may be attributed to the reason that more no-rain days are identified as rain events when correcting GSMap-gauged data using daily observation data to reduce overall precipitation error.It should be noted by data users, especially while utilizing these products to monitor real-time drought, that IMERG identifies more norain days than reality.
In addition, we calculated these extreme precipitation indices for both products over eight eastern river basins, respectively.The results indicate that at basin scale, both products performed well in cases of the Huaihe, Liaohe, and Yangtze River basins; however, for the Songhua, Haihe, and Southeast River basins, the two products could not observe accurate extreme precipitation, as shown in Table 6.IMERG overestimates RR99P and R20TOT at most of eight basins.GSMap-gauged shows different degrees of underestimation for these two indices at all of eight basins.For duration indices, IMERG shows very bad results at all eight basins.

Performance Analysis at the Basin Scale.
Researchers have been attempting to use satellite precipitation data in water balance analyses, hydrological modeling, and flood forecasting at the basin or subbasin scale [20,39].Considering its usefulness, in this section we consider the basin scale analysis to evaluate the performance of both products over the eight main basins in eastern China.Figures 8 and 9 show the daily and monthly spatial distributions of CC and ME over the 58 subbasins that are located within the eight main river basins.For the GSMap-gauged data on the daily scale, CC values tend to be higher at the middle part of eastern China, with high values (>0.80) located at the Haihe, Huaihe River basins, and the lower reaches of the Yellow and Yangtze River basins.CC values for the IMERG are not as high as those of the GSMap-gauged, but the spatial trend is similar, with the exception of low CC values at upper reaches of Yangtze River.The distribution of ME (Figures 8(c) and 8(d)) shows that IMERG tends to overestimate precipitation over most subbasins that located in the eastern part of the study area and underestimate precipitation over the western part of the study area.But GSMap-gauged shows underestimation over most subbasins of the study area except Huaihe River basin and lower reaches of Yellow and Yangtze River basins.At the monthly scale, CC values of almost all subbasins are higher than 0.8 in the case of both products (Figure 9), and the spatial distributions of these high values are similar for both products.The monthly precipitation accuracy is clearly higher than daily precipitation accuracy over the majority of the subbasins, and this result is even more significant for the IMERG data.However, when comparing the monthly ME, both products have very similar spatial pattern as that on daily scale.
The spatial patterns of POD and FAR at the subbasin scale for both products are shown in Figure 10.It is clearly seen that the GSMap-gauge performed better than the IMERG for POD, with high values (>0.8) locating in most parts of the Yangtze, Haihe, and Huaihe River basins.Similarly, for FAR the GSMap-gauged also performed better than the IMERG.The possible reasons could be partially explained by the results in Section 4.1: missed precipitation for the GSMap-gauged is lower than IMERG over eastern China during the study period (Figures 2(c) and 2(g)).Hence, POD of the GSMap-gauged show higher values than IMERG.
Figures 11 and 12 show the CC and ME values for the eight main basins at the daily and monthly scales.Similar to the results of the previous analysis and based on the CC comparison, the monthly precipitation is more accurate than daily precipitation.The GSMap-gauge seems to perform better than the IMERG at all eight basins, with higher CC values and lower absolute value of ME (except for Yangtze and Southeast River basins, with higher absolute ME values).Figure 13 shows the POD and FAR values for the eight main basins, indicating that GSMap-gauged performs better than IMERG in monitoring the occurrence of precipitation events.Furthermore, the GSMap-gauged data has higher accuracy over the Haihe, Huaihe, Liaohe, and Yellow basins, compared to the rest of the basins.Therefore, for satellite precipitation data application in these regions, we recommend the GSMapgauged over the IMERG.

Conclusions
In this study, first, error sources and characteristics of the two latest GSPPs (i.e., GSMap-gauged and IMERG) were comprehensively looked into by breaking down the total bias of these datasets into three independent components.Then,  the performance of the two products over eastern China including eight main river basins has been systematically evaluated during the period of April 2014 to March 2016 in terms of ability of extreme precipitation detection, precipitation measurement systemic errors (CC and ME), and occurrence of precipitation events (POD and FAR).The main findings are summarized as follows: (1) The two products have totally different spatial distributions of error components.Much of the underestimations over the study area for the GSMapgauged could be traced to significant hit bias, with a secondary contribution from missed precipitation.For IMERG, total bias illustrates significant overestimation over most of the eastern part of China, except upper reaches of Yangtze and Yellow River basins.This overestimation is the comprehensive impact of hit bias and false precipitation, although missed precipitation may counteract some of this impact.GSMap algorithm and errorcorrection method are more effective in balancing the errors related to topography.GSMap-gauged tends (3) Generally, GSMap-gauged performed better than IMERG with higher CC and lower absolute ME values at daily and monthly scale over eight eastern river basins.For the GSMap-gauged data at the daily scale, CC values tend to be higher at the middle part of eastern China, with high values (>0.80) located at the Haihe and Huaihe River basins and the lower reaches of the Yellow and Yangtze River basins.The monthly precipitation accuracy is clearly higher than daily precipitation for both products over most of the subbasins; this phenomenon is more significant for the IMERG data.The GSMap-gauged data shows higher accuracy over the Haihe, Huaihe, Liaohe, and Yellow River basins compared to other basins.
The evaluation of the performance of the IMERG and the GSMap-gauged datasets in this study could provide better understanding on spatial distribution of the error feature and uncertainty of these GSPPs over China.Based on these findings, we recommend that future correction efforts could focus on further reducing underestimation of strong rain for the GSMap-gauged, missed precipitation, and overestimation of heavy rain for IMERG data, respectively, especially over southern China.On the other hand, a better understanding of the advantages and limitations of these products can assist researchers in supporting their use in China.

Figure 1 :
Figure 1: Study area and location of about 2400 gauges used in CPAP.

Figure 2 :Figure 3 :
Figure 2: Total bias and three error components of GSMap-gauged and IMERG precipitation data from April 2014 to March 2016 over eastern China.

Figure 6 :
Figure 6: The intensity distribution of the total precipitation as well as the hit, missed, and false precipitation.

Figure 7 :
Figure 7: Scatter diagrams of extreme precipitation indices between GSPPs and observations over eastern China.The color represents the occurrence possibility and red color region shows that more percent of points are located there.

Figure 9 :
Figure 9: CC and ME distributions over 58 subbasins (included in eight main basins of eastern China) at the monthly scale from April 2014 to March 2016: CC for (a) GSMap-gauged and (b) IMERG; ME for (c) GSMap-gauged and (d) IMERG.

Figure 12 :
Figure 12: Spatially averaged CC and ME at the eight main basins over eastern China at the monthly scale from April 2014 to March 2016.

Table 1 :
Summary of eight first level basins of eastern China.

Table 2 :
Characteristics of the two latest GSPPs.
product (first released to the public sinceMarch 22, 2017)is the latest level-3 GPM precipitation dataset.The IMERG estimate takes advantages of three prior precipitation retrieval algorithms (PERSIANN-CCS, CMORPH, and TMPA).It combines precipitation measurements from various PMW sensors of GPM constellation and infrared estimates on geosynchronous satellites and is finally adjusted by monthly gauge data from GPCC (Global Precipitation Climatology Centre) Monitoring Product (Version 4) using the method applied in TMPA

Table 3 :
Contingency table for comparing observed and satellitebased precipitation.

Table 4 :
List of the continuous and categorical verification metrics used in the evaluation.

Table 5 :
The precipitation indices used in this study.
R20TOT (mm) Total precipitation sum of daily precipitationthat is more than 20 mm

Table 6 :
Statistics of extreme precipitation indices for the eight river basins over eastern China.
Figure 11: Spatially averaged CC and ME at the eight main basins over eastern China at the daily scale from April 2014 to March 2016.
Spatially averaged POD and FAR at the eight main basins over eastern China at the monthly scale from April 2014 to March 2016.to overestimate light precipitation (<16 mm/day) and underestimate more precipitation with rain rate larger than 32 mm/day; however, IMERG underestimates precipitation at rain rate between 8 and 64 mm/day and overestimates precipitation at rain rate more than 64 mm/day.(2) Both products can capture the percentile and absolute threshold extreme precipitation indices well with CC and ME values more than 0.85 over eastern China.While the IMERG overestimates RR99P and R20TOT, with RB values of 17.9% and 11.5%, respectively.GSMap-gauged shows significant underestimation of these indices.For two duration indices (CWD and CDD), IMERG shows poor results with low CC value and high absolute RB and RMSE values.IMERG identifies more no-rain days than reality.At basin scale, both products perform well in cases of the Huaihe, Liaohe, and Yangtze River basins, but for the Songhua, Haihe, and Southeast River basins, the two products could not observe very accurate extreme precipitation.