Performance of Multimodel Schemes for Seasonal Precipitation over Indian Region

This study uses downscaled rainfall datasets from 16 coupled climate models at high resolution of 25 km from 1987 to 2001. The multimodel superensemble scheme is widely tested for rainfall forecast over mid-latitude, subtropical, and, especially, various regions of the monsoonal belt. A well-known statistical estimation theoretic approach, namely, Best Linear Unbiased Estimator (BLUE), is examined on 16 member models. The results are compared with superensemble methodology based on various skill scores. Results show that BLUE is providing promising forecasts. As far as comparative studies are concerned BLUE and superensemble schemes compete and show their importance from normal years to extreme rainfall years. BLUE methodology is capable of predicting draughts very well comparedwith othermultimodel schemes. One basic advantage of BLUE is computationally less expensive than superensemble scheme.These statistical schemes like downscaling, BLUE, and superensemble can improve rainfall forecasts further, if a dense rain gauge data is provided.


Introduction
Several multimodel schemes are being listed in the literature of the climate and weather for rainfall prediction.These commonly used multimodel rainfall forecast schemes are ensemble scheme, biased removed ensemble mean, clustering techniques, and superensemble method.The superensemble scheme from Florida State University (FSU) is being tested since 1999 by various researchers [1][2][3][4].Nowadays a good quality of rainfall dataset is available from satellites (e.g., Global Precipitation Measurement, GPM/Tropical Rainfall Measuring Mission, TRMM), reanalysis (e.g., MERRA), and rain gauge (e.g., APHRODITE) at high resolution.Somehow climate global models still have a coarse resolution of 100 km.Such gap of resolution calls for downscaling of the climate global models.All the acronyms are mentioned in Acronyms for Models, Institutes, or Other Names.
Statistical downscaling procedures have been used to improve the horizontal resolution of the member models [4][5][6][7][8][9].By doing so, the regional details of the dry and wet patches of rainfall bulge out.There are limitations of dynamical models and postprocessing statistical techniques in predicting seasonal rainfall [1,10].The successes of statistical methods depend on the long-time series of data for training period to calculate better-quality coefficients.If the training datasets consist in many new information pieces on flood and drought events, then their obtained coefficients do better in forecast period.On the other hand, dynamical models have problems with their parametrization schemes and some simplification of various schemes used in them.In these models, systematic error grows with time.Worldwide more than 20 climate prediction centers are engaged in the monthly to seasonal predation with their home grown global models.Multimodel schemes were suggested to bring consensus forecast for a season [11][12][13].
India Meteorological Department has used statistical models and modified them over a period to provide an improved Indian summer monsoon rainfall prediction [14][15][16][17].Some of the limitations of statistical and dynamical models used for Indian rainfall prediction are noted by Nanjundiah [18] and Gadgil et al. [10].Various new multimodel 2 Advances in Meteorology schemes were tested on the Indian region for rainfall forecast [19][20][21].On examination of 5-multimodel schemes they realized that the accuracy of the rainfall forecasts can be increased over Indian region.Furthermore, they worked upon probabilistic prediction of the Indian region and found the probabilistic forecasts are superior to multimodel ensemble mean.This group has numerous research works on the prediction of rainfall using various techniques.In midlatitude, the sea level pressure, wind, and rainfall have a strong tie and thus can be used in multiple regression method to downscale the rainfall.In a recent study, canonical correlation analysis is used to downscale rainfall over Indian region and other [22,23].They found some improvements in the forecasts skills over some parts of northeast and peninsular India.There is no strong relationship between rainfall and other variables like sea surface temperature, winds, and outgoing long wave radiation over tropical region.In another study using stepwise regression, Salvi et al. [24] showed that their method could capture the rainfall over mountainous regions of India.They evaluated the future projection of rainfall over Indian region.The group is engaged in the various kinds of downscaling methods for rainfall over Indian region.
In this study, we used liner regression method to downscale the rainfall over the Indian region.It is known that, even in hindcast mode, none of the models provide correct forecast for a range of years.Perhaps, that was one of the necessities of the multimodel based prediction techniques.In a better way postprocessing datasets and statistical techniques can work together to refine the forecast further.Answers on various issues, for example, minimum number of member models to construct superensemble, length of datasets, and other sensitivity issues can be found in Kumar and Krishnamurti [25].The rainfall product is being improved first by downscaled methodology and then by superensemble method.In some of the studies, the prediction of Indian summer monsoon rainfall is being improved by superensemble and downscale method [25][26][27][28].In the present study, we worked with rainfall anomalies and the skills were compared among the best models (ECMWF model comes out best among 16 suites of models for Indian region, Kumar and Krishnamurti [25]), ensemble mean (EM), and two multimodel schemes.One of the important aspects here we tried to bring out is, how, accurately, can we forecast the extreme events?A new multimodel scheme, based on estimation theory, namely, Best Linear Unbiased Estimator (BLUE), has been examined [29].Furthermore, this study compares two operational schemes which have been used in hurricane prediction in the Atlantic basin.
The present study illustrates performance of the best model, ensemble mean, synthetic superensemble (SSE) technique, and BLUE scheme on 16 state-of-the-art coupled climate models for 15 summer seasons for the Indian region.This paper deals with the application of multimodel statistical methods.The skill scores used in this work are spatial correlation coefficient, RMSE, chi-square values for measure of association, ETS, BIAS, Heidke Skill Score [30], and ROC (Relative Operating Characteristic [30,31]).ROC is the plot between true positive rates (here hits score ratio or probability of detection) and false positive rate (here FAR ratio or False Alarm Ratio) for the forecasts.A curve closer towards the -axis indicates more accurate test.Thus, the area under the curve is the measure of the ROC score.

Dataset Used
Downscaled rainfall datasets (for 15 years, 1987-2001) from sixteen coupled models [33] are included in this study.All the models were integrated from May 1 to September 30 for the summer season (JJAS).Here we analyzed only summer season of monsoon (June to September) datasets in this study.Table 1 contains some details for atmospheric and oceanic components of each model, namely, model name, model resolution, initial conditions for simulation, and numbers of ensemble predictions.The ensemble mean forecasts from a single model's several runs are also included in this study.These model forecasts are cast at a common horizontal resolution of 2.5-degree latitude by 2.5-degree longitude for the construction of multimodel ensembles.APHRODITE Rainfall [34] dataset was used as observed rainfall.This data is based on thousands of rain gauges over a large region of monsoon Asia.The spatial resolution of the datasets is 0.25 × 0.25 lat-lon grid while the time interval of data is daily to monthly.To interpolate model's data from coarse resolution to fine resolution of observational dataset, we used 4-point Bessel interpolation method.

Downscaling and Multimodel Schemes
Liner regression scheme is applied for downscaling and to construct downscaled datasets from each member model against APHRODITE Rainfall datasets.
Chakraborty and Krishnamurti [26] have shown the improved rainfall forecasts with downscaling and without downscaling from member models, ensemble mean, and superensemble scheme.They illustrated that the downscaled superensemble scheme shows higher correlation and reduced RMSE over Indian summer monsoon rainfall.During multimodel ensemble, we considered entire duration of datasets of 15 years (15 years × 4 months = 60 values) of monthly rainfall.Next, we constructed multimodel schemes based on downscaled datasets.It is shown that the data of 15 years were sufficient to carry out the downscaling as the coefficients stabilize after 10 years of datasets [25].
We believe that a data processing method improves the model datasets and adds some error as well.However, this can be reduced in some situations.There is a major difference between the mathematical strategy for downscaling and for the construction of the multimodel superensemble scheme.The former downscales each model separately with respect to the observed estimates, whereas the multimodel superensemble calculates a single forecast considering forecasts from the member models all together.It performs a multiple liner regression to remove the collective bias of the suite of models.The two methods are mutually independent.Over all, first downscaling helps in sprouting the regional features in the rainfall forecasts from each member model and then superensemble scheme is improving the forecast based on multimodel.Cross validation method is used during superensemble and BLUE methodology.In this method, a year, which was forecasted, was not taken, while calculating the downscaling or superensemble weights.[34] is used to downscale the rainfall forecast from member models over the Indian region.

Downscaled Methodology. APHRODITE Rainfall
where  obs and  mdl are the observed and interpolated model forecasts of rainfall (at the same resolution), respectively;  and  are regression coefficients known as the slope and intercept of the least square fitting; and  is the error term.
where  dscl is the downscaled rainfall forecast of the model; here  and  are calculated using (2) at each grid point and separately for every month of the year.We left out the year to be downscaled from the calculation to calculate  and  following the method of cross validation.There are many more downscaling methods, for example, canonical analysis and stepwise pattern projection.The linear downscaling methods perform well as compared to other methods [35].
We choose a linear downscaling method here.

Synthetic Superensemble Technique.
The superensemble methodology [1,36] produces a single forecast based on multimodel forecasts.Multimodel superensemble forecasts based on downscaled datasets from member models were constructed as well [37].We expressed that as follows: where  is the superensemble prediction,  is the observed time mean (climatology),   are the weights for the individual models ,   and   are the forecast and forecast mean for a model  for training period, and  mdl is the number of models.Here weights are obtained by minimizing error using least square method.The sum of the weights needs not be one and they vary from negative values to positive values.

BLUE Technique.
In this study, we introduce another multimodel construction technique based on estimation theory.Individual model is downscaled to sprout the regional features of the rainfall.Next, superensemble scheme and BLUE acted on multimodels to remove the model biases.In case of BLUE the coefficients are inversely proportional to the errors of the models and the sum of coefficients is one.The methodology is described in the Appendix.2 and 3 show the year by year spatial correlation and RMSE numbers for all the member models, EM, superensemble scheme, and BLUE.From Tables 2 and 3, we found that the correlation varies from −0.31 to 0.59 for all the models.The ranges of correlation coefficients are varying from negative to positive values which is why we cannot talk about significance of the correlations.For some of the years (e.g., 1991, 1995) the correlation has significance of 0.02 (two-tailed probabilities).

The Spatial Variability of Rainfall
It may be noted that the highest correlation for a year varies from model to model, yet multimodel schemes (BLUE and superensemble) perform better than any member model and EM.We observe that for the year 1999 none of the models and schemes has a positive correlation except CERF, KORAM, MAXP, and NCEP.Table 2 has the RMSE range from 1.34 to 3.82.Here multimodel schemes tried to minimize the RMSE but the margin between them and member models are not so much.It may be mentioned that rainfall variability over Sri Lanka was very well captured by superensemble (correlation coefficient (CC) = 0.44).The skills of rainfall variability from year to year are explained in Figures 2(a) and 2(b) in terms of spatial correlation coefficient and RMSE.BLUE and EM keep their spatial correlation coefficient positive for most of the time except for 1999.In Figure 2, we considered the target region slightly smaller than the bigger region displayed in Figure 1, because many of the northern regions especially north of 30N are rain gauges sparse.Chakraborty and Krishnamurti [38] found the negative anomaly correlation for year 1999 for a bigger monsoon region.In case of ECMWF and superensemble scheme spatial correlation is not higher for all years.It is varying from positive to negative from 0.5 to −0.24.It is interesting to note that BLUE is almost following the EM.Furthermore, in case of year 1995 all the models have lowest RMSE and highest correlation, while for the year 1999 RMSE reached the highest value and correlation became lowest.
In this study, we also used measure of association of attributes to judge the extent of closeness between a model forecast and observed rainfall.Qualitative variables whose outcomes are expressed as "yes" or "no" or by some categories, namely, "Good" or "Bad," are referred to as attributes in the statistics literature.whether two attributes are associated or not is tested using the chi-square test of independence.
Figure 3 displays the chi-square values for comparing association between observed and different model forecasts for domain under study.It is based on every grid point over India.The target region is India (69-92E, 8-30N).BLUE is showing highest categorical association for 1996, 1997, 2000, and 2001, while superensemble scheme shows highest categorical association for 1987, 1990, 1998, and 1999.Next ECMWF shows highest categorical association for 1988, 1989, 1992, and 1994, while EM comes out with highest association for 1991, 1993, and 1995.Both BLUE and superensemble scheme are doing well with respect to this measure.One contingency table, Table 3, is provided for year 1987 for the threshold of −0.1 rainfall anomalies for BLUE.This table has a significance to calculate the skill scores for the categorical rainfall.ETS and BIAS were calculated for rainfall anomalies for the threshold of −3 to 3 mm/day (Figures 4(a) and 4(b)) for the Indian region (lon = 69.0,lon = 92.0E;lat = 8.0, lat = 30.0N).Interestingly, BLUE is commanding for negative threshold while superensemble scheme is commanding for all positive rainfall thresholds for 15 years of period.BLUE had ETS range between 0 and 0.28, while superensemble scheme had ETS range from 0.04 to 0.18.That indicates BLUE has remarkable potential to predict droughts.An ETS of 0.3 is considered a good one in case of rainfall [38].For the various categories of rainfall (light, moderate, and heavy rains) Dash et al. [39] get ETS values of 0.24 to 0.03 over the Indian region.Next in case of BIAS (Figure 4(b)) BLUE shows least BIAS for positive thresholds while superensemble scheme shows least BIAS for negative thresholds.BLUE has the range of BIAS from 0.1 to 1.67 while superensemble scheme has 0.1 to 0.4.ECMWF and EM are not performing well with BIAS as compared to multimodel schemes.Some of the incompatibility has been discussed regarding ETS while scoring about extreme events [40].Heidke Skill Score (HSS) has been presented for various years (Figure 5) including flood and drought years of Indian summer monsoon.BLUE is doing better for 1995 and 1998, while superensemble scheme does better for 1988 and 1989.Still, their response becomes mixed if we pin down their superiorities for all the thresholds.For example, in 1995, BLUE does well with ETS for the threshold range of −3 to 1.5 but skill degraded for 1.5 to 3. In case of 1988 superensemble scheme does good for −0.5 to 3 but stumbled for −3 to −1 thresholds.Overall, these two multimodel schemes come out finer and doing better for all the threshold and years except for few.Hogan et al. [40] recommended HSS over ETS to express skill from multimodels.In Table 4, we explained the numbers of hits, misses, false alarm, and correct negatives for year 1987.It is same as Table 5, but for a year with values.
Rainfall over Indian region shows high rainfall variability, due to large variations in orographic lands, vegetation cover, and soil texture.Probabilistic forecasts are based on the yes/no proposition.Over a grid, for a threshold, this yes/no proposition decides the hits (both observation and model show nonzero rainfall values), misses (where observation shows nonzero while model shows zero rainfall values), and false alarm (where observation shows zero while model shows nonzero rainfall values) as basic variables for the probabilistic forecasts.Figure 6 shows the ROC plots between hit ratio and false ratio for JJAS seasonal rainfall anomalies, for four years.If the ROC curve for a model is far away from the 45-degree line that model performs better than others.The pink dotted line indicates exact matches of the observed and forecast cases, that is, ideal forecast cases.For 1998 BLUE comes out as the best one while for 2000 superensemble comes out as the best.For the two remaining years 1987 and 1998 their responses are mixed.Acharya et al. [41] showed results for three categories of rainfall from multimodel schemes over Indian region.They found better skills of ROC for the wet and dry years as compared to normal monsoon years.

Conclusions and Discussion
All the results based on commonly used skill metrics are performed on downscaled datasets for 15 years (1987-2001), 16 member models, and observed rainfall datasets from APHRODTE.Year by year, the superiority of four models has been cited based on chi-square which indicates the dependency of model on skill matrix.The maximum values of correlation coefficient obtained from the multimodel schemes EM, SSE, and BLUE are −0.28 to 0.51, −0.16 to

Figure 2 (
b) shows RMSE, which is lowest in case of BLUE.Here superensemble scheme comes out distinct in many years with lowest RMSE. 1987

Figure 4 :
Figure 4: (a) ETS and (b) BIAS for ECMWF, EM, superensemble scheme, and BLUE for JJAS seasonal rainfall anomaly predictions for Indian region.

Figure 5 :
Figure 5: Heidke Skill Score plots for ECMWF, EM, superensemble scheme, and BLUE for JJAS seasonal rainfall anomaly predictions for many years over Indian region.

Table 1 :
Details of sixteen global coupled models used in this study.
in the first, second, third, fourth, and fifth rows, respectively.The rainfall anomalies from APHRODITE, ECMWF model, EM, superensemble scheme, and BLUE captured a range of variability from drought to flood year rainfall over Indian region.Year 1987 was considered as one of the worst droughts in the history of Indian summer monsoonal rainfall variability, which, remotely, had an influence from El-Niño event in Eastern Pacific Ocean.The central Indian region was badly affected by very low rainfall while eastern India received a good rainfall.Rainfall deficient over central India was simulated by most of the models, while the patches of extreme rainfall were not captured by any one.Year 1991 was affected by low rainfall over northern and northeastern India.Interestingly ECMWF captured it fully, as well as superensemble scheme, but EM and BLUE failed here.Year 1995 was witnessed with drought over southcentral India while flood kinds of situations prevailed over northern India.ECMWF model was best to simulate the rainfall variability over the Indian region, but it failed to simulate the rainfall over the eastern parts of India.SSE tried to simulate the rainfall variability but missed deficient rainfall patches over central India.Some of the patches of dry region over Odisha (20.95N, 85.05E) were remarkably captured.It is to be noted that BLUE did better than other models in case of year 2000, which was almost a monsoon drought (rainfall was −9% of the climatological normal) over Indian region.Tables Spatial patterns of rainfall anomalies for 1987, 1991, 1995, and 2000 are being shown in Figure 1.Rainfall anomalies from APHRODITE Rainfall datasets, coupled model from ECMWF, ensemble mean, superensemble scheme, and BLUE are shown

Table 2 :
CC for all the member models, EM, superensemble scheme, and BLUE for JJAS seasonal rainfall anomaly predictions.CC more than 0.5, statistically significant at 95% level, is

Table 4 :
Contingency table for year 1987 corresponding to the rainfall anomaly threshold of −0.1 mm/day for BLUE.
a Hits.b Misses.c False alarm.d Correct negatives.
41C plots for ECMWF, EM, superensemble scheme, and BLUE for JJAS seasonal rainfall anomaly predictions over Indian region.0.59, and −0.25 to 0.52.On the other hand, ECMWF has the range from −0.35 to 0.49 only.Clearly the improvement is not much, but for rainfall anomalies, surely it is appreciable.While, in case of RMSE, EM has a range from 1.41to 2.78, SSE has the range from 1.48 to 2.79 and BLUE has the range from 1.36 to 2.75.Categorical association attained by ECMWF is 0.39, EM is 0.42, SSE is 0.48, and BLUE is 0.45.In case of ETS, ECMWF attained 0.27.The ETS for EM is 0.28, SSE is 0.18, and BLUE is 0.29.BIAS is reduced to 0.35 in case of ECMWF, 0.15 for EM, and 0.05 for SSE and BLUE.