Considering District and School Factors and Their Relationship to ACT Performance in North Carolina: An Examination of the ACT Pilot Results

Since 2001 several states have adopted the requirement that high school students either take the SAT or ACT to assess high school programs or assist students in accessing post-secondary-educational opportunities. In 2012 the state of North Carolina adopted a new accountability program that included the ACT as a measure of college readiness. Previous research on the relationship between school districts and school level performance found that district size had a role in school achievement. This study looked at how district factors influenced the ACT performance of students across North Carolina in an effort to better understand if there were district factors other than size that may be influencing student performance and how high school reforms, given the influence of district factors is meeting the goal of increasing student college readiness. The results of this study are as follows. (1) District factors are related to school level performance, where student race and parental education levels were found to be significant predictors of achievement, (2) the traditional school level factors of race and student socioeconomic status did significantly predict ACT scores, and (3) as a high school reform model, students attending early college high schools did score higher on the ACT as compared to traditional high schools. (c) an accelerated track to after secondary education is realized; (d) there is an outreach for middle grade students to develop interest and enhance transition into the program; (e) students benefit from integrated academic and social supports helping to create a cohesive learning environment; (e) small learning communities are used; and (f) the transition from high school to college is eliminated as students take college classes while being in high school.


District and School Factors Affecting Student Performance
The association of school district composition or characteristics and student performance is not well documented and when research is available, district size appears to be the factor most often controlled for. For example, Trani [1] found that as district size increased the performance of students in general declined and when coupled with lowwealth students, the performance of these students declined further. These results run counter to other studies that found increases in school district size was associated with higher levels of student performance [2][3][4][5]. It has been suggested that some of the inconsistency in results seems to be contributable to the methods employed to study the hypothesized relationship (see [1]). Including other variables to enhance the understanding how district factors are associated with student performance are wanting and the application of methodological approaches that reflects the organization or schools and districts that may lead to more consistent findings and findings that can be readily applied to district and school reform efforts. In contrast to the lack of research understanding district factors and student performance, there has been a great deal of research on the relationship between school related factors and student achievement. The research has pointed to the influence of student wealth, race, school size, curriculum, teacher preparation, and other variables that were found to predict a reasonable degree of student outcomes [6][7][8][9][10]. For example, Ma and Klinger, in their study of New Brunswick Schools, found that how student and school factors influenced achievement varied over the type of academics examined such that race, gender, and wealth of students is seen as significant predictors of performance. They also found that the school level factor of school size was limited in its relationship to student performance. The Rainey and Murova study was interesting as it found that parental education level was a strong predictor of student performance a finding consistent with other research. Additionally Sirin [11] found that student wealth is a critical factor in predicting performance. These relationships can be traced back to Coleman [12] when he posited that schools can do little to overcome such student related factors. Therefore, while individual student factors have been associated with student performance, the relationship between school characteristics and a student performance is less well understood. That is, how the overall composition of the school is related to individual student achievement is less clear (see [13]). And as stated above, how the organizational nature of schools and school districts affect student performance is even less well documented.

High School Reform: One Consideration
Increasing access to higher education for traditionally underserved students began in earnest in North Carolina in 2002 when then governor Mike Easley relying on a grant from the Bill and Melinda Gates Foundation opened the first early college high schools under the umbrella of the Early College High School Initiative [14,15] by improving the achievement of students. The emphasis on improving college access for students has now become a statewide initiative including all high schools with the recent redesign of the North Carolina high school accountability model to include scores from the ACT for college bound students and the WorkKeys for those students contemplating attending community colleges. This new redesign calls for a focus on career and college readiness [16] and includes new pathways students can select that are intended to better prepare students for after secondary education. The ACT was chosen as a measure to include in the new model since it has been presented as a measure of college readiness and the ACT has developed standards that have been shown to correlate well with initial college success [17].
The emphasis on increasing college access has also been addressed at the federal level when the United States Department of Education [18] released the report A Test of Leadership: Charting the Future of U.S. Higher Education in 2006 which called for significant changes in higher education. Of the several areas suggested for improvement were access, affordability, quality of instruction, accountability, and innovation. Aside from calling on colleges to change, the report also cited that college professors reported that substantial portions of college freshman were ill prepared for the rigors of higher education. The report in addition to calling for reforms in higher education "strongly encourages early assessment initiatives that determine whether students are on track for college" (p. 17). The report even endorsed the expansion of early college and dual enrollment programs. The combined focus on the report of simultaneously improving higher education and high school preparation was reported to have had limited success [19] in improving higher education access and accountability. Field reported that at the time of publication 14 states preformed early assessments to determine if students were ready for college, an increase of 8 since 2006. As proof to the impact of these policies, Klasik [20] also found that this approach had increased college access in several states.

High School Reform and Accountability
3.1. North Carolina School Accountability Model. The North Carolina school accountability model has been revised several times since its inception in 1996 [21]. In its current version the new accountability model evaluates schools on the degree to which students are ready for the future and in the case of high schools whether students are either career or college ready. The model for high schools includes assessing math performance in higher-level courses, ACT and WorkKeys achievement, graduations rates, and a graduation project. Key to this model is the incorporation of ACT performance benchmarks [22]. Schools will be evaluated according to how many students achieve the benchmarks that ACT has developed as measures for college readiness. According to the ACT, students who meet the ACT Test benchmark have a high probability of success; for those students meeting this standard means there is a 50% chance that they will receive a B of better in the corresponding college course. The NCDPI does state that parents and students should expect scores to be lower than the national average as all grade 11 students will take the test not just those students planning to attend college. The utilization of the ACT to gauge college readiness does have support [23,24] to predict student success especially when coupled with high school grades [25,26], which the current accountability model substitutes for end-of-grade test scores. On face, the North Carolina model appears to be aligned with the available data on how to reasonably predict success beyond high school using standardized assessments and given the research present by Klasik [27] a model that if successful, could contribute to better prepare students and increase college participation.

The North Carolina Early College
Model. The early college model in North Carolina is designed to provide students who are traditionally underrepresented in post-secondarysettings access to rigorous educational opportunities by supporting academic attainment in high school and beyond. The model is proposed to target students who are on predominately low income and minority and who generally will be the first members of their immediate family to attend a college or university. The importance of this is clear as minority students are less likely to attend college when compared to nonminority students according to the American Council on Education (ACE) [28]. Compounding this is the nagging reality that in addition to minorities being less likely to attend college than other students, far fewer minority males access after secondary education compared to minority females. Further complicating the issue, entrance to college is not only a function of academic performance but one of accessibility in terms of affordability Commission on the Future of Higher Education [18]. So for low-income minority students, attending college and/or graduating is problematic. The North Carolina model attempts to address these issues by providing post-secondary-educational opportunities for High school innovation projects (HSIP) are characterized by the pervasive, transparent, and consistent understanding that the school exists for the purpose of preparing all students for college and work. They maintain a common set of high standards for every student to overcome the harmful consequences of tracking and sorting.
Require powerful teaching and learning HSIP are characterized by the presence of commonly held standards for high quality instructional practice. Teachers in these schools design rigorous instruction that ensures the development of critical thinking, application, and problem solving skills often neglected in traditional settings.

Personalization
Staffs in high school innovation projects understand that knowing students well is an essential condition of helping them achieve academically. These high schools ensure adults leverage knowledge of students in order to improve student learning.

Redefine professionalism
The responsibility to the shared vision of the HSIP is evident in the collaborative, creative, and leadership roles of all adult staff in the school. The staffs of these schools take responsibility for the success of every student, hold themselves accountable to their colleagues, and are reflective about their roles.

Purposeful design
High school innovation projects are designed to create the conditions that ensure the other four design principles: ready for college, powerful teaching and learning, personalization, and redefined professionalism. The organization of time, space, and the allocation of resources ensures that these best practices become common practice.
early college high school (ECHS) students by giving tuitionfree access to community college and university level courses. These schools are located on the campuses of two-year and four-year colleges and universities to alleviate transportation barriers and provide access to higher education courses. The early college high schools are required to adhere to a well-defined list of design principles NCNSP [29]. These design principles (presented in Table 1) are seen as central to the success of these schools, are therefore non-negotiable, and must be complied with by all High School Innovation Projects (HSIP), including early college high schools. In addition to the above, all early college high schools share the following. (a) Students have the opportunity to earn an associate's degree or two years of college transfer credits; (b) students earn college credit without additional costs; (c) an accelerated track to after secondary education is realized; (d) there is an outreach for middle grade students to develop interest and enhance transition into the program; (e) students benefit from integrated academic and social supports helping to create a cohesive learning environment; (e) small learning communities are used; and (f) the transition from high school to college is eliminated as students take college classes while being in high school.
Many traditional high school students can access these educational options; however, taken as a whole they are unique to early colleges. For example, across the United States high school students can take college classes while being in the traditional high school; however, due to the manner in which these opportunities are provided, students are usually not well integrated into the college environment resulting in a disjointed learning experience (see [30]). A key curricular design element in early college high schools is the alignment of high school and college classes in a manner that accelerates four years of high school into two and coordinates the instructional delivery in the high school and college environments in an effort to have high school students that are not only successful in high school but also as college students. To assist in achieving this goal high school and college instructors attend professional development as a team, discuss students, and work collaboratively to support students (see [30]). Given at least in design, North Carolina seems well positioned as a state to increase college access and increase the representation in higher education of minorities and child of poverty. In North Carolina, not all districts have early college high schools and therefore investigating how district factors are related to the new college readiness measures may lead to a reconsideration of this by those districts not presently offering this option to students.
To operationalize the district school relationship and college readiness, this study was designed to determine if the early college model has an effect on a measure of college readiness, as defined by performance on the ACT; an answer to the following question was sought. In North Carolina do early college high schools have more students considered college ready as compared to traditional high schools? To reflect the nested nature of schools within districts a multilevel model was used to explore the possible mediating effects of district and school factors such as wealth, size, demographic composition, and level of parent education.

Methods
This study used a quasi-experimental design which incorporated a multilevel regression modeling. The use of multilevel or hierarchical methods has been well documented and its purpose is clear [31]. In this study utilizing hierarchical modeling provided the opportunity to examine both school and district effects in a nested model. The nested model reflected the theoretical posit that schools performance is connected to district characteristics.

Research Questions/Objectives
Purpose. This study was designed to seek answers to the following two questions.
(1) What if any relationships exist between school district factors and school achievement as measured across select ACT performance scores?
(2) Is there a performance difference between traditional and early college high schools when district and school level factors are considered; that is, isECHS more college ready? Since the data was aggregated at the school level, neither individual student scores nor data were available such as special education status, growth, or other factors that could be included with student level data. The demographic data for each school was taken from various files that the NCDPI uses and makes available to the public, mainly from the Business and Financial section. Likewise, district data was taken from these sources as well as U.S. census data that was used to report school district educational attainment.

Data Analysis.
Descriptive statistics were computed from the data to report the means, standard deviation, percentages, and counts for each of the parameters used in the analysis. Data were analyzed using a two-level linear mixed model with a random intercept and random slope for high school type. The performance outcomes of English, mathematics, science, reading, writing, and an overall composite score were the dependent variables and the predictors were the school and district variables listed in Table 2. Racial characteristics of both the district and school are accounted for in the majority variable, which is a composite of Asian and white students. An unconditional model was run first, one that does not have any estimation parameters to establish basic statistics that were used to calculate the intraclass district correlation and later to assess model fit using the deviance statistics to test the hypothesis that the addition of the level 2 predictors did not result in significant improvement in model fit.
Below are two equations that represent, in the general case, the models used in the analysis. Originally, the full model had two more level 2 variables, school district wealth and overall composite score of high school performance. When these variables were included, the coefficients were found at no time to be significant for any ACT performance outcome; therefore these variables were not included in the final model. Model 1 (unconditional ( = th school, = th district)). One has the following: Model 2 (full). One has the following: ACT Performance = 00 + 01 * ADM D + 02 * Majority D + 03 * Education + 10 * ADM S + 20 * Majority The full model has a random intercept and the random slope for the school type (ECHS). The random intercept was selected to reflect the variability in each school districts' mean scores. The random slope for school type was selected to allow for the early college/not-early college gaps in performance to vary across the two school types again consistent with the observed differences revealed in the graphical and descriptive analyses.

Results
The results from the descriptive analysis for the dependent and independent variables are shown in Table 3. As can be seen there are differences in the means and standard deviations of the variables across school type and that there is significant variation at the district level. There appear to be  substantial differences in the ADM and performance means at level 1 while the racial composition and wealth variables vary to a lesser degree. In Table 4 below the descriptive statistics for the sixpredicator variables are presented. There are substantial differences in the ADM and performance means at level 1 while the racial composition and wealth variables are somewhat similar. As seen in the table, the ECHS has higher mean scores in all six ACT performance areas with larger standard deviations in all except writing. This may indicate that there is greater variability between ECHS as compared to traditional high schools. For both groups of schools there are outliers, with the traditional schools having many more, these outliers were inspected and determined to be valid and were ultimately included in the data.

Mixed Model
Results. The first question sought to answer if there were school district factors that were related to ACT performance across schools. An unconditional model was run for each outcome and the results were reported in Tables 5 and 6. The estimates for the means of the dependent variables in the unconditional models are similar to the means presented in Table 4. The intraclass correlation (ICC) for these models was calculated to show the amount of variance in ACT scores that can be accounted for by group membership or in this case the school district. As seen in the table, the ICCs ranged from a low of 0.11 for English to 0.16 for writing. The conditional ICCs were calculated from the full models and in all cases except for writing; there were considerable reductions in the amount of the variance of ACT explained by district variables of average daily membership, racial composition, and parental education. In the case of writing, the conditional ICC was larger indicating that using the full model, more variance, was explained by district membership (between groups) than by the schools (within group).
The results show that of the three district level factors, school district size (ADM D) the estimated parameter was not significant and its influence is quite small. While the coefficient for ADM D is small it is important to remember that the scale measurement is a single student, and therefore it is quite plausible to understand how adding one student would not be associated with large increases in performance. For the variable, level of education, the results predicted a small but positive influence such that all else being equal as the percentage of adults in a distinct with a baccalaureate by age 25 increased, and student performance was predicted to increase as well. The final district variable was that of racial composition. The slope for this coefficient was quite large indicating that as a school district had greater percentages of white and Asian students, a large performance increase was predicted to occur. For example, in math a 10 percent increase in majority students and ACT scores are predicted to increase by 2.96. Of all the district variables, the racial composition of the district was found to have the largest influence on student performance.
The second research question addressed school type; specifically the question sought to determine if there was an association between the type of school a student attended and the student's performance on the ACT. The variable ECHS, a dummy variable with non-early college high schools set as the reference, has positive and highly significant estimates for all six ACT tests. These results suggest that early college high school students are predicted to have higher ACT scores than students who attended a traditional school. The influence of previous school achievement as measured by the estimator for the variable performance, although highly significant, is small which implies that as the performance on state tests increases for the school, ACT performance is predicated to increase, although small in magnitude. Viewing Table 3, early college high schools have much higher mean composite scores and as the results report, this higher mean state test score composite is associated with significantly higher ACT scores; however, the influence of this is controlled for in the model and may explain this small coefficient. School size (ADM S) was found to have a positive and significant coefficient for all outcomes. This implies that all else being equal, larger schools should expect to have higher student performance. It is interesting to note that on average all the early college high schools have a much smaller student enrollment as compared to traditional schools, but yet the early college high schools are predicted to have higher student performance. Therefore, while small school student enrollment maybe a defining characteristic of an early college, the degree to which the size of a school's student population predicts student performance is much smaller than other factors. The remaining two school level predictor variables in all but one case have highly significant coefficients. Consistent with the estimated coefficients for district racial composition, as a school's percentage of majority students increases, student performance is predicted to increase. In fact, this variable in 5 of the 6 cases has a much larger estimated coefficient that any of the other variables. The final variable, school wealth, the percentage of students classified as economically disadvantaged, has powerful negative influence on student performance, ranking only second to race as the most prominent school level factor. In every case, the models predicted that as a school served more economically disadvantaged students, student performance is expected to decline by a large degree.

Discussion
In summary, this study presented findings that showed some district factors can have an influence on school achievement and confirmed that school factors such as student wealth, race, and previous achievement are associated with student performance and the school design can positively influence student performance.

District Level.
The results of the study are somewhat consistent with previous research on how school district factors are associated with school performance; namely, that district size and racial composition do influence how schools perform (see [1,32]). Similar to what Rainey and Murova [9] found for rural districts, that parental educational level of attainment influenced student achievement, this study found (in the case of ACT scores) that parental education is an important factor. As stated earlier several other district level factors were originally considered and were found to play no significant role in student achievement, namely, per pupil spending, overall district achievement, and the percentage of economically disadvantaged students. The lack of influence of the process factor of spending was no surprise as earlier research Caldas, [33]; Hanushek [34] found this also to be evident in his work. The result that overall district achievement had no significant influence on achievement was somewhat surprising at least from a conceptual perspective in that as high schools represent an overall composite of the district's performance, how students perform across the district was conceived to influence how the students performed in high school. This outcome may be accounted for applying Weick's [35] idea of educational organizations as being viewed as loosely coupled to the larger school system. He proposed that in systems there are strong and weak associations or degrees of coupling and therefore it could be argued that high schools are separated from the other schools in the district which may result in loose or weak linkages between the overall district context with regard to achievement and specific school performance. It would be interesting and useful to investigate how loosely coupled high school performance is to overall district performance to see if this result can be explained.
The finding that wealth as measured at the district level being found not to be a significant factor in predicting school level performance is perplexing. Trani [1] and Abbott et al. [36] all found that there are relationships between district size and negative effects on the performance of students of poverty. Specifically in Abbott, it was found that large school districts have lower achievement, that poverty effects are stronger in large school districts, and that small schools are especially sensitive to the effects of student poverty on achievement. In sum, the results herein contradict or call into question at least some of previous findings when school district size was found to be significantly related to student performance. However, even though the estimated coefficients were positive, not being significant calls into question their usefulness in contradicting the previous studies.

School Level-College Readiness.
For this group of schools, the results suggest that students attending early college high schools exhibit greater "college readiness" than do their traditional high school peers. Across all six ACT performance measures, students attending early college students are estimated to have from a 9% to 16% mean ACT score boost. If in fact the ACT can be a useful determinant of college readiness [17] and a view held by the State of North Carolina [17], then early college high schools are predicated to have a clear advantage in meeting the revised accountability standards and their students will have a marked advantage in being successful in after secondary environments. This finding is encouraging for the targeted population of traditionally underserved early college high school students. The ACT performance advantage is consistent with previous research on the North Carolina Early College program where it has been found that early college high schools have a more narrow achievement gap [37,38] and stronger performance in several academic areas including enhancing college readiness [39]. Combining these findings points to a more promising future for children attending early college high schools as these school are predicted to attenuate several barriers facing minority and poor students in terms of college attendance. Researchers have found that minority and poor students fail to complete the necessary steps to apply to college and at a much lower rate than their white or nonpoverty peers [20,40,41]. While the revised state accountability measures will have all students to take the ACT, students attending early college high schools appear to have the additional benefit of predicted higher performance.
The other school level variables were significantly related to student performance in a manner consistent with previous research (see [10]). It was found that wealth and race had significant and profound estimated effects on student performance such that race was the largest (magnitude) influence on performance. The small but positive significance of school size runs counter to what others have found with regard to school size. Results from the research on small high schools have not been consistently positive as [42] presented in their evaluation of the Chicago High School Redesign Initiative. They found the following: (a) small schools did not offer a stronger instructional climate, (b) nor did these small schools improve student achievement, and (c) students also reported very different experiences and degrees of success as a result of the school they attended. In another study, Iacovou [8] investigated the effects of class size and she found that small classes are not related to attainment. However, the result that school size is related to ACT may not be so unusual as larger high schools can offer more advanced placement courses and generally a more challenging curriculum than smaller schools, although this advantage has been attenuated some by online programs. The fact that early college high schools, which are much smaller than traditional high schools, can offer advanced courses and a rich curriculum since they all are located either on a community college or a four-year college/university may account for some of the observed performance differences.

Suggestions for Future Research and Conclusion.
One limitation of this study is the type of data that was available; data aggregated at the school level fails to adequately represent the individual students who attend the schools. If student level data could be accessed, this study could be extended to the 3-level model where students would be nested in schools and schools grouped in districts. This would allow for a more comprehensive, complex evaluation being possible by linking specific course performance to ACT scores (English, math, and science) and other type of student variables, such as exceptional children/academically gifted, and provide an opportunity to apply propensity score matching across students.
Second, not all early college programs share the same goal. Some prepare students to obtain an associate's degree and prepare them for work after high school graduation while others seek to prepare students to matriculate into a four-year college or university. Stratifying the early colleges across, these may lead to different results and provide a more comprehensive understanding of the success of the varying types of programs.
It is clear from this study that school district factors in North Carolina influence student performance and that different school programs yield contrasting achievement results. If in fact the results presented in this study can be replicated and generalized beyond this sample of schools, implications for school reform policy are at hand. The fact that developing an early college high school is a district decision emphasizes the need to better understand what prompts or support a district's decision to create such a high school. The analysis did not attempt to discover why a district opted to have an early college or not but rather attempted to discern what district characteristics can be associated with school level performance. At the school level the results in this study and others seem to support the notion that as a high school reform, the early college model is quite effective in increasing student achievement and as defined in the context of North Carolina, it produces more college ready students. While the cost of maintaining the early college model is shared by the state and district, the bulk of the cost is borne by the state making this reform fiscally appealing. If postsecondary-educational opportunities are to be improved for all high school students, then further understanding of how districts influence achievement is needed, and second if the early college model is as effective as shown by this study, either continued expansion of the program is warranted or determining those salient programmatic processes and features that yield these outcomes needs to be better understood and scaleup conducted to provide these opportunities to more students.