The performance of six satellite-based and three newly released reanalysis rainfall estimates are evaluated at daily time scale and spatial grid size of 0.25 degrees during the period of 2000 to 2013 over the Upper Blue Nile Basin, Ethiopia, with the view of improving the reliability of precipitation estimates of the wet (June to September) and secondary rainy (March to May) seasons. The study evaluated both adjusted and unadjusted satellite-based products of TMPA, CMORPH, PERSIANN, and ECMWF ERA-Interim reanalysis as well as Multi-Source Weighted-Ensemble Precipitation (MSWEP) estimates. Among the six satellite-based rainfall products, adjusted CMORPH exhibits the best accuracy of the wet season rainfall estimate. In the secondary rainy season, unadjusted CMORPH and 3B42V7 are nearly equivalent in terms of bias, POD, and CSI error metrics. All error metric statistics show that MSWEP outperform both unadjusted and gauge adjusted ERA-Interim estimates. The magnitude of error metrics is linearly increasing with increasing percentile threshold values of gauge rainfall categories. Overall, all precipitation datasets need further improvement in terms of detection during the occurrence of high rainfall intensity. MSWEP detects higher percentiles values better than satellite estimate in the wet and poor in the secondary rainy seasons.
Rainfall is an important parameter for the characterization of water cycle. In Africa, assessment, planning, and management of water resources are often constrained by lack of reliable rainfall data [
In accession to the satellite-based precipitation estimates, a newly released European Center for Medium range Weather Forecast (ECMWF) reanalysis precipitation estimates and Multi-Source, Weighted-Ensemble Precipitation (MSWEP) [
Both satellite-based and reanalysis precipitation estimates exhibit significant bias, which needs to be postprocessed [
Past rainfall studies in the region and elsewhere [
This study attempts to evaluate six commonly available satellite-based precipitation estimates (SPEs) and three newly available reanalysis precipitation estimates at 0.25-degree spatial grid size and daily temporal resolution with the view of improving the reliability of precipitation estimation of the wet season (June to September) and secondary rainy season (March to May) rainfall datasets over the Upper Blue Nile Basin. This comes from the evaluation of state-of-the-art reanalysis that will allow us to understand the current strengths and limitations of these products for water resources evaluations in the region. Results will also contrast the potential benefits/limitations between satellite-based and reanalysis precipitation estimates, information that can be helpful for blending approaches of the two.
The Upper Blue Nile Basin, locally called “Abbay” in Ethiopian, is located within 7.5° to 12° north and 34° to 40° east (Figure
The Upper Blue Nile Basin in the Eastern Africa, overlaying the rain gauge network and the global precipitation estimates grid. In gray highlighted grid pixels, we depict those grid cells that include at least one gauge.
Past studies have demonstrated difficulty to efficiently evaluate water resources of the Blue Nile due to its complex terrain and lack of adequate data, mainly precipitation, at subbasin and short time scales. Existing rain gauge observations are sparse in both time and space within the basin. This could cause a lack of insight into the evaluation of water resource availability for impacts and benefits of major development interventions on the water resources management of the basin.
The rain producing climate systems and rainfall characteristics for the study region has been described in several studies [
The surface rainfall observations are obtained from a network of 153 National Meteorological Agency (NMA) stations within the Upper Blue Nile Basin at daily time scale. The spatial and temporal distribution of these rainfall gauging stations is uneven and exhibits very limited coverage in time and spatial distribution that follows the local road network and major towns (Figure
Evaluation of precipitation estimate is carried out at the 0.25-degree regular grid pixels represented by at least one-gauge observation. A total of 92 satellite-grid boxes that contain gauge observations were considered for the period 2000–2013. The observed gauge rainfall data were interpolated using ordinary kriging (OK) algorithm to produce rainfall fields at 0.05-degree grid size, which were then aggregated to the 0.25-degree grid box, and considered as the reference areal gauge rainfall to evaluate satellite-based and reanalysis precipitation dataset [
The satellite products used in the analysis are commonly used in operational and research activities focusing on water resources planning, design, and decision-making in the basin. This study evaluated six of the main satellite-based precipitation estimates (SPEs) at 0.25-degree spatial grid size and daily time scale for the period 2000 to 2013 (Table
Summary of the satellite-based and reanalysis rainfall products used in this study.
Rainfall product | Abbreviation | Retrieval method | Data used |
---|---|---|---|
TMPA | 3B42V7 | MW + IR + gauge observation | 2000 to 2013 |
3B42RT | MW + IR | 2001 to 2013 | |
|
|||
CMORH | CM | MW + gauge observation | 2000 to 2013 |
CM-unadj | MW | 2000 to 2013 | |
|
|||
PERSSIAN | PN | IR + gauge observation | 2001 to 2010 |
PN-unadj | IR | 2001 to 2013 | |
|
|||
Reanalysis | ERAI-unadj | ECMWF reanalysis | 2000 to 2013 |
ERAI | ECMWF reanalysis + gauge observation | 2000 to 2013 | |
MSWEP | Reanalysis + SPEs + gauge observation | 2000 to 2013 |
The TMPA version-7 precipitation datasets were released in December 2012 [
The TMPA algorithm includes four steps [
The CMORPH product is created by morphing methods that combines MW precipitation estimate with IR sensors observations. The IR-image data are used to propagate the MW-based precipitation estimates forward and propagated backward in time between successive MW sensor observations [
The PERSIANN precipitation datasets are created from IR brightness temperature observations using an artificial neural network method [
In addition to SPEs, we examined unadjusted and gauge adjusted ERA-Interim and Multi-Source Weighted-Ensemble Precipitation (MSWEP) precipitation estimates as indicated in Table
The rainfall events detection capability was examined. The global rainfall products’ skill to detect daily rainfall accumulation greater than 0.1 mm [
A 2 × 2 contingency metric.
Gauge observation | ||
---|---|---|
Rain | No-rain | |
Satellite/reanalysis estimate | ||
Rain | H = hit | F = false detection |
No-rain | M = miss | Correct no-rain |
The quantitative analysis for comparison of satellite rainfall against gauge observation is based on a statistical error metric. We use the following error metrics to evaluate the performance of satellite-based and reanalysis products. The error analysis utilized statistical techniques using bias ratio (bias), Person correlation coefficient (CC), and normalized root-mean-square-error difference (NRMSE) to evaluate performances.
Furthermore, the performance of satellite products is evaluated for different rainfall magnitudes conditional to different reference gauge rainfall threshold values. These threshold values correspond to the 10th, 25th, 50th, 75th, 90th, and 95th percentiles of the reference gauge rainfall.
The Upper Blue Nile Basin has a marked wet season from June to September and a secondary rainy season from March to May. The climatological characteristics and seasonal rainfall driving systems of the region have been discussed in several past studies [
Spatial pattern of mean season rainfall (mm) for the period June to September.
Spatial pattern of mean seasonal rainfall (mm) for the period March to May.
Gauge adjusted SPEs, 3B42TR, CM-unadj, and MSWEP show an equivalent spatial pattern with two regions of peak values (>1000 mm), whereas adjusted ERAI shows a similar spatial pattern without distinct peak rainfall in the basin. The ERAI-unadj estimates mean seasonal rainfall above 1000 mm in most parts of the basin exhibit a stronger overestimation of the wet seasonal rainfall relative to the other products. PNN-unadj does not capture the spatial pattern of rainfall as compared to the other products and underestimate the seasonal rainfall amount in the southern and eastern parts of the study domain. The secondary rain seasonal mean rainfall pattern is illustrated in Figure
We examined the performance of the six SPEs and the three reanalysis products using categorical statistics of POD, FAR, and CSI. The statistics computed from rain/no-rain events of contingency table used to evaluate the skill of products’ that detects rainy events [
Mean value of statistical error metrics for the wet (June to September) seasons
Product type | Bias | CC | NRMSE | POD | FAR | CSI | MRV | FRV |
---|---|---|---|---|---|---|---|---|
3B42V7 | 0.88 | 0.36 | 1.08 | 0.83 | 0.01 | 0.82 | 0.10 | 0.00 |
3B42RT | 0.73 | 0.30 | 1.08 | 0.79 | 0.01 | 0.78 | 0.14 | 0.00 |
CM | 0.87 | 0.39 | 0.98 | 0.92 | 0.01 | 0.91 | 0.04 | 0.00 |
CM-unadj | 0.74 | 0.36 | 0.88 | 0.92 | 0.01 | 0.91 | 0.04 | 0.00 |
PNN | 0.82 | 0.37 | 1.02 | 0.78 | 0.00 | 0.78 | 0.12 | 0.00 |
PNN-unadj | 0.52 | 0.36 | 0.81 | 0.75 | 0.00 | 0.75 | 0.16 | 0.00 |
ERAI-unadj | 1.39 | 0.23 | 1.31 | 0.99 | 0.01 | 0.98 | 0.00 | 0.00 |
ERAI | 0.89 | 0.26 | 0.92 | 0.98 | 0.01 | 0.97 | 0.00 | 0.00 |
MSWEP | 0.92 | 0.37 | 0.80 | 0.99 | 0.02 | 0.98 | 0.00 | 0.00 |
Mean value of statistical error metrics for the secondary rainy (March to May) seasons
Product type | Bias | CC | NRME | POD | FAR | CSI | MRV | FRV |
---|---|---|---|---|---|---|---|---|
3B42V7 | 1.07 | 0.41 | 1.79 | 0.67 | 0.20 | 0.57 | 0.17 | 0.12 |
3B42RT | 1.09 | 0.39 | 1.97 | 0.64 | 0.20 | 0.54 | 0.20 | 0.15 |
CM | 0.93 | 0.42 | 1.70 | 0.65 | 0.16 | 0.56 | 0.18 | 0.22 |
CM-unadj | 1.10 | 0.42 | 1.86 | 0.68 | 0.19 | 0.57 | 0.16 | 0.31 |
PNN | 0.95 | 0.46 | 1.74 | 0.46 | 0.09 | 0.44 | 0.29 | 0.05 |
PNN-unadj | 0.56 | 0.43 | 1.33 | 0.40 | 0.08 | 0.39 | 0.36 | 0.02 |
ERAI-unadj | 1.05 | 0.30 | 1.63 | 0.82 | 0.22 | 0.66 | 0.08 | 0.08 |
ERAI | 0.93 | 0.32 | 1.49 | 0.82 | 0.21 | 0.67 | 0.08 | 0.08 |
MSWEP | 0.99 | 0.44 | 1.31 | 0.92 | 0.29 | 0.66 | 0.03 | 0.03 |
Categorical statistics for (a) POD for June to September, (b) POD for March to May, (c) FAR for June to September, and (d) FAR for March to May.
Results from SPEs showed that CMORPH products scored higher mean POD (92%) which have better skill in detecting rainfall events in the wet season while 3B42V7 is about 83%. 3B42RT and gauge adjusted PERSIAN are equivalent with mean value of POD about 78%. The precipitation-estimating algorithms are different among the SPEs (explained in the methodology section). The MW-based products provide better estimate of precipitation event detection than the IR-based estimate. Both products of CMORPH are a result from propagation and morphing techniques of MW-based estimate which are superior in rainfall event detection performance among the six SPEs in this study (Figure
From March to May of rainy season, both products of TMPA and CMORPH performance is nearly equivalent with mean POD (62–67%) and higher values over some locations. FAR is the relatively higher mean value of about 20% for TMPA and CM-unadj while CM has 16%. PERSIAAN products exhibit a lower value of mean FAR (<10%). Figure
MRV and FRV for June to September (a and c) and March to May (b and d).
In addition to SPEs, results from the reanalysis products of POD and FAR are shown in Figures
The results from CSI showed that, during the wet season, CMORPH products exhibiting 91% of rainfall events were correctly detected followed by 3B42V7 (82%). The three newly created reanalysis products outperform (CSI > 97%) the satellite estimates in terms of correctly detected rainfall event in both seasons. During the secondary rainy season CMORPH and 3B42V7 are nearly equivalent (Figures
CSI for June to September (a) and March to May (b).
The error analysis is done on a grid cell by grid cell basis for the unconditional case where the reference gauge rainfall threshold ≥ 0.1 mm and the averages over the study domain are described. The spectrum of the error metrics CC, bias ratio, and NRMSE collected from each grid box is illustrated in Figure
Statistical error metrics of bias ratio (a and b), CC (c and d), and NRMSE (e and f) for the rainfall seasons.
The results from CC statistics showed all products are nearly equivalent both in wet (CC ~0.3) and in secondary rainy (CC ~0.4) seasons (Figures
Recently, Abera et al. [
The gauge unadjusted ERA-Interim overestimated, which is the most biased product in the wet season (Figure
The performance of the SPEs and reanalysis was evaluated with different percentile threshold values of gauge rainfall categories for the two rainy seasons (Figure
Conditional error metrics.
ERAI-unadj overestimated below the median by 30% at 10th percentile and exhibits lower underestimation for higher threshold values in the wet season. The other eight products show the increased magnitude of MRE with an increasing percentile of threshold values. ERAI, MSWEP, CMORPH, and 3B42V7 nearly are equivalent having a lower magnitude of MRE (10–20%) below the 25th, whereas CMORPH and 3B42V7 outperform better in capturing relatively higher percentile threshold values.
In the secondary rainy season, TMPA and CM-unadj have slight overestimation (~<5%) below the 25th and outperform with a lower magnitude of underestimation for higher thresholds. The reanalysis estimates sharply increase the magnitude of underestimation for higher quantile. The PNN-unadj exhibits highest underestimation in both seasons. The CC relatively lower values decrease with gauge rainfall threshold categories. The ERAI and ERA-unadj have a lower CC for the two rainfall seasons.
The results of this study provide an examination of the performance of satellite-based and newly released reanalysis rainfall products at a daily time scale for the wet and the second rainy seasons during the period 2000 to 2013 over the Blue Nile Basin. The study utilized six adjusted and unadjusted satellite precipitation products (TMPA, CMORPH, and PERSIANN), two ERA-Interim reanalysis products, and MSWEP, which is a blended product that combines satellite, reanalysis, and gauge precipitation. The evaluation of rainfall estimate was carried out at 0.25-degree regular grid and at least one-gauge observation within the grid box. Comparison analysis using a point observation or averaged rainfall of the unevenly distributed station networks at daily time step could cause a substantial consequence on the value of error metrics. We used the unbiased linear estimators, ordinary kriging (OK) algorithm, to produce areal representation of gauge rainfall at 0.25-degree grid size for comparison at daily time scale. Based on the categorical and quantitative error metrics used in our analysis, we summarize our findings as follows: The categorical error metrics for event detection showed that CMORPH products have higher POD, which are better in detecting rainfall events in the wet season while TMPA and CMORPH products are nearly equivalent during the secondary rainy season. FAR is below 1% of all products during the wet season and 8 to 20% during the small rainy season. MSWEP product outperformed the satellite-based and reanalysis with the highest POD in both rainfall seasons; both exhibit a slightly higher mean FAR than the satellite-based on the wet and small rainy seasons. In terms of volume of missed and falsely detected rainfall, CMORPH has lower MRV than the TMPA and PERSIANN products. All products are nearly equivalent in detecting percentage of FRV. The ECMWF reanalysis and MSWEP products are relatively better than the satellite products in avoiding missed rainfall volume in both seasons. The mean value of FRV indicates that the reanalysis products detected slightly higher volume of false rainfall. The results from CSI showed that CMORPH products outperform the rainfall events correctly detected in the wet season. During the secondary rainy season, CMORPH and 3B42V7 are nearly equivalent. The three newly released reanalysis products outperform the satellite estimates in terms of correctly rainfall event detection in both seasons. The bias ratio results showed that satellite-based rainfall products underestimated the wet and slightly overestimated the small rainy season’s gauge precipitation. The CC statistics showed that all products are nearly equivalent in both seasons. The spread of random error components was shown to be slightly higher for TMPA products. Among the reanalysis products, the error metric statistics show that MSWEP outperform ERAI-unadj and ERAI estimates. The magnitude of error metrics is linearly increasing with increasing percentile threshold values of gauge rainfall categories. 3B42V7 and CM are relatively better in capturing higher percentile in the wet season while CM-unadj better in capturing higher percentile in the secondary rainy seasons. MSWEP detects higher percentiles values better than 3B42RT and CM-unadj detect higher percentiles values in the wet season. This indicates that the detection ability of both satellite and reanalysis products needs further improvement during heavy rainfall intensity in the retrieval algorithms. Gauge adjusted SPEs, 3B42TR, CM-unadj, and MSWEP are equivalent in capturing the spatial pattern of wet seasonal rainfall amount with two regions of peak values. During the second small rainy season, all products have similar spatial distribution performance of the seasonal rainfall pattern with PNN-unadj having relatively lower seasonal rainfall amount.
We observed that, among the six satellite-based rainfall products, CM has relatively better estimation performance in the wet season, while, in the small rainy season, CM and 3B42V7 are nearly equivalent over the Upper Blue Nile Basin for the period we examined. The MSWEP performed better in rainfall event detection. A unique nature of MSWEP product is the availability of data since 1979. To take full advantage of the best performing satellite and reanalysis products with the view of improving the analysis of water resources in the basin, our next step is to create blended rainfall products that combine the different satellite and reanalysis datasets accounting for each product’s respective uncertainty.
The authors declare that they have no conflicts of interest.
This work was supported by the EU-funded Earth2Observe Project (ENVE.2013.6.3-3) of AAU and in part by a small support from the Ethiopian Minister of Water, Irrigation and Electricity.