Editorial: Evaluation of the Project P.A.T.H.S. in Hong Kong: Are the Findings Replicable Across Different Populations?

Existing youth enhancement programs commonly deal with isolated problems and issues in adolescent development and they are relatively short-term in nature. To promote holistic development among adolescents in Hong Kong, we launched Positive Adolescent Training through Holistic Social Programmes (P.A.T.H.S.). One unique characteristic of the Project P.A.T.H.S. is systematic evaluation of the program using various evaluation strategies. In this special issue, comprising 8 research articles, we have applied the principle of replication to re-evaluate the effectiveness of the Programs, including subjective outcome evaluation based on program participants and implementers, evaluation based on secondary data evaluation, and objective outcome evaluation. Generally speaking, the findings are consistent with those reported previously, thus providing evidence for the replication of related research findings. These replicated findings generally suggest that different stakeholders have positive perceptions of the program, workers and benefits of the program and there is evidence supporting the effectiveness of the program.


Objective Outcome Evaluation: A randomized group trial with 24 experimental schools and 24
control schools initially has been carried out. 2. Subjective Outcome Evaluation (Tier 1 Program): Both students and program implementers are invited to complete subjective outcome evaluation forms (Form A and Form B, respectively) after completion of the program. 3. Process Evaluation: Systematic observations are carried out in randomly selected schools to understand the program implementation details.

Interim Evaluation:
To understand the process of implementation, interim evaluation is conducted by randomly selecting roughly half of the participating schools in the Experimental and Full Implementation Phases. 5. Qualitative Evaluation (Focus Groups Based on Students): Focus groups involving students based on schools randomly selected from the participating schools are carried out. 6. Qualitative Evaluation (Focus Groups Based on Program Implementers): Focus groups involving instructors based on schools randomly selected from the participating schools are carried out. 7. Qualitative Evaluation (In-Depth Interviews with Program Implementers): Prolonged indepth interviews with teachers are carried out. repertory grid tests that assess their self-identity systems before and after joining the program and perceived changes across years.
Generally speaking, triangulation of the available evaluation findings shows that different stakeholders had positive views about the Tier 1 Program and they perceived the program to be beneficial to the development of the program participants. Most importantly, the findings suggest that the project is effective in promoting positive youth development among Chinese adolescents in Hong Kong. For the Tier 2 Program, subjective outcome evaluation was evaluated by subjective outcome evaluation method. The program participants are invited to complete a subjective outcome evaluation form (Form C) after completion of the program.
To date, evaluation studies of the Project P.A.T.H.S. with reference to the above evaluation strategies have been carried out. These include objective outcome evaluation utilizing a randomized group trial [4]; subjective outcome evaluation based on quantitative and qualitative data collected from the program participants and instructors [5,6]; qualitative evaluation based on focus groups involving students and instructors [7]; in-depth interviews with program implementers, student logs, and student products [8]; process evaluation involving systematic observations of delivery of the program [9]; and interim evaluation [10].
Although the aforementioned mechanisms consistently provide strong evidence that the Project P.A.T.H.S. has a beneficial influence on students [11], there is still the question of whether the evaluation findings can be replicated across populations. Actually, in the realm of science, replication has been regarded as an important strategy for confirming the validity of scientific investigations. Fahs et al. [12] pointed out that "replication of research is essential to the building and continued development of the scientific basis of any discipline.' (p. 67). Reese [13] similarly reminded that "although replication research is often disvalued as 'derivative', it can be an invaluable aid to scientific progress" (p. 1). King [14] also stated that "the most common and scientifically productive method of building on existing research is to replicate an existing finding -to follow the precise path taken by a previous researcher, and then improve the data or methodology in one way or another" (p. 445).
The role of replication in guarding against uncritical acceptance of research findings is also proposed by different researchers. Singh et al. [15] asserted that "replication serves the fundamental role of 180 protecting against the uncritical acceptance of empirical results. It is thus as important as the core academic practices of peer review and publication of research, and is necessary for any stream of scientific inquiry to develop the requisite rigor of a science" (pp. 533-534). Hubbard and Vetter [16] similarly argued that replication "protects against the uncritical assimilation of specious empirical results into the literature. Replications with extensions serve to determine the scope and limits of empirical findings by seeing if they can be generalized to other populations, contexts, time periods, geographical areas, and so on" (p. 153).
Adopting a critical realist position, Tsang and Kwan [17] argued that "although replication cannot yield certain conclusions, it does not follow that it is not desirable to conduct replicated studies or that replication has no epistemic significance. When a replication successfully confirms the findings of the original study, it provides at least some support for the theory concerned. If the two studies are far apart in time, the replication supports the temporal validity of the theory as well. When numerous replications in diverse circumstances are repeatedly successful, it is highly likely that the theory has hit upon some real structure or mechanism in the social world, barring an alternative nonrealist explanation of this success" (p. 759). In short, replication serves an important role in social science research, particularly in the area of program evaluation.
In this special issue, several evaluation papers on the effectiveness of the Tier 1 and Tier 2 Programs, including subjective outcome evaluation based on program participants and implementers, evaluation based on secondary data evaluation, and objective outcome evaluation, are included [18,19,20,21,22,23,24,25]. Generally speaking, the findings arising from these papers are consistent with those reported previously, thus providing evidence for the replication of related research findings. These replicated findings generally suggest that different stakeholders have positive perceptions of the program, workers, and benefits of the program and there is evidence supporting the effectiveness of the program.