Comment on “School-Based Obesity Prevention Intervention in Chilean Children: Effective in Controlling, but not Reducing Obesity”

Objective. To evaluate the effectiveness of a 12-month multicomponent obesity prevention intervention. Setting. 9 elementary schools in Santiago, Chile. Subjects. 6–8 y old low-income children (). Design. Randomized controlled study; 5 intervention/4 control schools. We trained teachers to deliver nutrition contents and improve the quality of PE classes. We determined % healthy snacks brought from home, children’s nutrition knowledge, nutritional status, duration of PE classes, and % time in moderate/vigorous activity (MVA). Effectiveness was determined by comparing BMI between intervention and control children using PROCMIXED. Results. % obesity increased in boys from both types of schools and in girls from control schools, while decreasing in girls from intervention schools (all nonsignificant). % class time in MVA declined (24.5–16.2) while remaining unchanged (24.8–23.7%) in classes conducted by untrained and trained teachers, respectively. In boys, BMI declined (1.33–1.24) and increased (1.22–1.35) in intervention and control schools, respectively. In girls, BMI remained unchanged in intervention schools, while increasing significantly in control schools (0.91–1.06, ). Interaction group time was significant for boys () and girls (). Conclusions. This intervention was effective in controlling obesity, but not preventing it. Even though impact was small, results showed that when no intervention is implemented, obesity increases.

We read with interest the article "School-Based Obesity Prevention Intervention in Chilean Children: Effective in Controlling, but not Reducing Obesity" [1], hereafter "the article." In the article [1], nine schools were randomized into intervention and control groups according to the socioeconomic conditions of the children (a stratified randomization design at the cluster level), resulting in five schools in the intervention condition and four schools in the control.The intervention consisted of training teachers to deliver content on healthy eating and to improve the quality of physical education classes.The primary outcome was change in BMI  score between baseline and follow-up.This is a typical cluster randomized controlled trial (cRCT) in which the inferences are intended to apply at the individual (student) level while randomization is at the cluster (school) level [2,3].In cRCTs, the potential lack of independence among individuals in the same cluster, that is, intracluster correlation (ICC), creates special methodological issues in both design and analysis.Any individual level analysis without considering the clustering is invalid [3].Unfortunately, the article [1] ignored the clustering in its sample size estimation and final data analysis, which potentially increased type I and type II error rates and put their conclusions in doubt.
The article [1] claims the study as a cluster randomized trial; however, the power and sample size estimation in Section 2.1 completely ignores the facts that (1) the sample size of cRCTs consists of cluster number () and cluster size (), (2) the power is more dependent on the cluster number than cluster size [2], and (3) the "design effect" is caused by the similarity of individuals in the same cluster [2,3].The presentation of study sample sizes described in Section 2.1 is therefore misleading and might confuse readers.
More severely, ignoring the clustering in the final data analyses of cRCTs (as done in the article [1]) will cause inflated type I error rates by (1) underestimating the variance of intervention effects and (2) using the extremely magnified degrees of freedom (df) in the hypothesis testing.The fact that clusters are nested within intervention conditions makes the df available to estimate the intervention effects much smaller than the df without nested clusters.For a hypothetical cRCT with  persons nested within  clusters across  experimental conditions, there will be  =  *  *  total persons.Because of the impact of clustering, the df for estimating the between cluster variance is the number of conditions multiplied by the number of clusters minus one, or df =  * ( − 1), which is far smaller than  * ( *  − 1), used in the article [1].Considering only nine schools that are involved in the trial, the Kenward-Roger small sample df approximation [4] should also be recommended, which has been implemented in some commercial statistical packages including SAS and R.
Furthermore, in order to improve the transparency and utility of cRCTs, the CONSORT 2010: extension to cluster randomized trials [5], states that the ICC and an indication of its uncertainty are to be reported in describing (1) how the sample size is determined and (2) how clustering is taken into account in the statistical analysis.In addition, the reported ICCs are also helpful for those who may subsequently perform similar (replication) studies.
To evaluate the validity of this study, we conducted a permutation/randomization test for the difference in change of BMI  scores, using the data in Table 2.Although the major conclusion of the study-in which the school-based treatment prevented increase in BMI  scores compared to control schools-does have statistical merit (permutation  < 0.03), we suggest that the authors redo their analyses taking the clustering into account and report the unconditional ICC (or constituent variance components) and its confidence interval for better practice.

Disclaimer
The opinions expressed are those of the authors and do not necessarily represent those of the NIH or any other organization.