Exploring the Relationship between the Collegiate Learning Assessment , Student Learning Activities , and Study Behaviors : Implications for Colleges and Universities

Globally, institutions of higher learning have attempted to utilize various methods to assess student learning outcomes and simultaneously determine what factors influence targeted performancemeasures. eNational Survey of Student Engagement is a popular instrument many colleges and universities employ to gain an understanding of what student behaviors are linked to such desired outcomes as graduation and persistence. Internationally, other universities are exploring the concept of student engagement as a means to assess college environments. Recently, calls from stakeholders have prompted institutions to use assessments to demonstrate so-called twenty-first century skills such as critical thinking; the Collegiate Learning Assessment is one such tool. is study reports preliminary efforts to link the NSSE engagement indicators to CLA performance at a medium-sized Historically Black College or University. Results indicate that the NSSE indicators are (1) poor predictors of student GPA, (2) of the ten indicators of the NSSE, only one was found to be a significant predictor of CLA performance, and (3) of the items comprising this indicator, only half are associated with CLA outcomes.


Introduction
Recent events in defining the role and purpose of higher education from the federal and local levels have been the impetus for many institutions of higher learning to explore ways to demonstrate that a postsecondary degree matters [1].
ese pressures are present worldwide as the role of a college education has shifted from that of the private to public well [2,3], such that colleges and universities have aligned their focus to service business and industry in order to remain economically viable and receive the support of both the individual and the society [4,5].As a result of these pressures, institutions of higher learning have been tasked with teaching and assessing the degree to which a college degree can improve attributes such as the ability to think logically and solve problem [6,7].In an effort to demonstrate the valued-added nature of the college degree, these institutions have expanded accountability and assessment initiatives to demonstrate to stakeholders that college matters.
ere have been multiple accountability initiatives in higher education [8] that have called for the direct assessment of student learning in ways that provide comparable information across institutions and states [9,10].Using regression analysis and archival data, this article examines to what degree measures of student learning and engagement experiences are related in an effort to identify the link between learning activities and performance to support postcollege career readiness is possible [11].
ese researchers state that, despite the numerous examples of institutions using the National Survey of Student Engagement [12] to monitor programs that are linked to enhancing student engagement, there is a paucity of studies that attempt to link student engagement to desired student outcomes.is study does so by utilizing regression analysis to explore if there are links between student performance on the Collegiate Learning Assessment [13], student learning activities, and study skills as reported by the National Study of Student Engagement in an effort to determine if such a process could be useful in providing insight into how colleges and universities may better align the engagement work they do with the aims and outcome-associated measures of student performance.

Literature Review
In the twenty-first century, the skills essential for today's students are far different from those required in the previous decades.A greater emphasis is being given to college readiness and college outcomes because employers now more than ever expect college graduates to possess writing, critical thinking, and problem-solving skills [14] in response to the changing demands of available jobs [15,16].Teachers and students can no longer rely solely on the accumulation of disciplinary knowledge and skills as employers evaluate potential employees not purely on what they know, but also on what they can do and how they do it.
ere is now an emphasis and precedent on focusing on the twenty-first century skills [17,18] (SBAC, 2012) in addition to knowledge in specific content domains [19][20][21][22] in hopes of fostering the development of critical thinking, problem-solving, communication, collaboration, creativity, and innovation skills [19], skills coveted by employers.e impetus to change instructional programs is multifaceted such that it includes various stakeholder groups including prospective students, the very customer colleges, and universities trying to attract.In a recent annual survey conducted by UCLA, Wyer [23] reports that, for the first time, freshman state that getting a better job is most important to attend college.

2.1.
Assessing Learning and Engagement.Assessment in higher education is a complex issue not only with regard to its purpose but also includes social justice perspectives, equality, and overall public policy considerations of what is assessed in higher education [24].Messick suggests that assessment for learning in higher education must serve two groups: (1) the faculty must get feedback to support changes in instruction to enhance student outcomes and (2) students must learn from assessment so they can reflect on their own learning practices to better support their knowledge and skill development.Furthermore, it is posited that administrators as well as key policy-makers must also be aware of how the types of learning activities instructors utilize are linked to these same immediate pedagogical outcomes, which includes the long-term implications the college experience has for students and society [25][26][27][28][29]. Assessment is a multifaceted activity often intended to serve many related and divergent purposes.An institution must be clear on what assessment could be added not only in terms of measuring student outcomes but also how these outcomes can be influenced by the actions of faculty and administrators.

Collegiate Learning Assessment.
e Collegiate Learning Assessment (CLA) is designed to assess the skills most critical for success beyond college [30,31].CLA provides a reliable measure of students' ability to solve complex problems by analyzing and synthesizing disparate pieces of information, while presenting cohesive, persuasive, and grammatically correct solutions [32,33], and its values to do so have received international attention and application [34].e research on the CLA+ as a predictive measure of multiple postcollege outcomes was presented in [33,35].e CLA+ was found to gauge and predict factors such as employment, salary, and graduate school attendance.ese findings support the utilization of such assessments by employers and institutions of higher learning as a method to determine if applicants and graduates are suitable for employment and further education [36].It has been suggested that the CLA+ can among other things provide information to potential employers, graduate schools, and communicate achievement standards that can serve to certify the rigor of a school's educational programs [33,35,37].

Student Engagement.
Assessing student engagement has become a critical activity for institutions of higher learning globally.Recent efforts in the United Kingdom [38] illustrate the growing attention student engagement is acquiring.In the United States, the National Study of Student Engagement (NSSE) has been utilized by institutions of higher learning to better understand how to measure student engagement and then to assess the degree to which these schools provide their students with value-added learning environments [39,40].It also has been argued that the survey can support the actions for faculty to better align their behaviors to better achieve certain desired outcomes for students [41].Kinzie et al. [42] comment that the NSSE can be a useful tool in monitoring institutional practices and student behaviors that have been linked to certain outcomes.However, according to Gordon et al. [11], schools that use the NSSE should not be concerned on how well they do on the NSSE, but rather should focus on the application of the results to develop a better understanding of how they can improve student outcomes by linking the NSSE to actual instructional activities, and these pedagogical actions support school success [43].e use of the NSSE data for institutional purposes, specifically for improvement student outcomes, has been criticized with regard to the reliability and validity of the results [44].However, as these criticisms are not without merit, as Pike [45] previously stated that these shortcomings necessarily need to be evaluated within context and that it is important to apply any set of data carefully and understand the intended purpose of the data.Interestingly, Pike [44] posits that the results from the NSSE are (1) dependable measures of student engagement in educational practices when at least 50 students are used and (2) the NSSE institutional benchmark scores were significantly related to graduation and retention.He further provides that it is possible the use these scores to gauge the engagement of students and evaluate the effectiveness of institutional actions and programs to improve student engagement and academic success.
Despite the robust research on student engagement and the relationship to student learning [46][47][48], supporting student retention [49,50], and impacting student persistence 2 Education Research International [51], faculty and administrators still struggle to effectively assess student engagement at lower levels within the college or university.Considering the preceding arguments, it is argued that analyzing the relationship between the NSSE and the CLA is consistent with the recent emphasis on predicting the postcollege activities of graduates as being important for both college officials and potential students.Furthermore, the potential results from this study may yield powerful policy tools to institutional leaders, faculty, and students as to select which institutions of higher learning offer the most value [52].

Research Questions.
e following questions were used to guide the study by examining the possible relationships between student engagement and critical thinking as defined above: (1) Are the NSSE engagement indicators associated with students' GPA? (2) Are specific items in the NSSE related to the performance scores on the CLA?
Finding that benchmark scores of the NSSE are related to institutional retention and graduation rates [11,44] may suggest that the scores from the benchmarks can serve as measures of the influence of institutional programs and practices that are intended to enhance student success.While previous research has demonstrated weak relationships between NSSE benchmark scores and certain student outcomes [53], additional exploration is warranted.

Research Question 2.
is question explores the possible relationships of the student engagement measures used in the NSSE with the critical thinking outcomes of the CLA.e research of Niu et al. [54] showed that there are limited effects of the interventions colleges and universities have implemented in an effort to affect the critical thinking skills of students.Examining how these lower level measures are related to measures of student performance has shown that, although the macromeasures conceptualized and the benchmarks are useful, it is not without limitations, and looking at the microlevel measures may yield relationships not revealed by using aggregated methods [11].

Methods
is study uses multivariate linear regression using maximum likelihood estimation with missing values utilizing robust standard errors.To accomplish this, extant data from several past administrations of both the NSSE and CLA were retrieved via institutional research.e participants were freshmen and senior students from 2013 to 2015 that took both the NSSE and CLA.Students are not required to complete the NSSE; therefore, the engagement indicators are weighted to improve the representativeness of the sample to the larger student population.In contrast, the CLA is given to all students as a freshman and again as a senior to measure the value-added experiences of attending the university; therefore, no weighting is required.Table 1 reports the demographic data on the sample.
In Table 2 the descriptive statistics for the aggregated level of the regression variables are reported.To investigate the relationship between measures associated with student engagement and problem-solving skills, this study replicated to some degree the work of Gordon et al. [11] such that both the aggregate measures of student engagement as defined on the NSSE were regressed onto CLA performance and then the individual questions which are grouped by these identifiers were also regressed on the same outcome measures.To establish parsimonious models the demographic variables in Table 1 were originally included in the regression analysis, and it was found that these factors did not significantly account for differences in either the engagement indicators of CLA scores to the extent that including them in the final models reduced the power and failed to improve the overall precision.

Results
e first analysis was to regress the NSSE engagement indicators onto the student cumulative GPA at the time of taking the NSSE to address research question 1. e results shown in Table 3 indicate that there is little predictive power of the engagement indicators and institution researchreported GPA as of the ten engagement indicators, only four were found to have coefficients different from zero.
ey were as follows: Student-Faculty Interaction, Supportive Environment, Quality Interaction, and Effective Teaching for all students.When student class was used to create groups, two engagement indicators were found to have some predictive power for freshman status of students-Student-Faculty Interactions and Effective Teaching.
is is in stark contrast for senior class students as the predictive ability of the NSSE engagement indicators of student GPA revealed that not one of the indicators was found to have an estimated coefficient different from zero.
For the second question, a regression was run to determine if the NSSE engagement indicators are predictive of CLA performance.Prior to running this regression, a simple pairwise correlation was run between GPA and overall performance on the CLA, and it was found there was a significant positive relationship (r � 0.4, p < 0.001, r 2 � 0.16); a small correlation exists between the two.erefore, it was expected that if a relationship between NSSE and CLA was to be discovered, it would be limited.As shown in Table 4, using all students, of the ten indicators, four were found to have coefficients different from zero, Student-Faculty Interactions, Diverse Discussion (DD), Collaborative Learning (CL), and Reflective and Integrated Learning (RI).e predicted coefficients had both negative and positive values and ranged in size with Collaborative Learning having the largest negative coefficient implying that a one-unit change in this value was predicted to lower a CLA score by approximately 2.3 points or less than 1%.Both Diversity and Reflective and Integrated Learning had positive coefficients Education Research International with RI having been found to increase CLA scores by 2.09 points for every one-unit change, a very small percentage increase.When examining these relationships by student class status, there is a simple difference on how each class of students perceives engagement being a factor on CLA performance sharing only one indicator.For freshman, RI, CL, and Learning Strategies (LS), all have estimated coefficients different from zero.While for senior students, only DD and CL have nonzero estimated coefficients.A test was run to determine if the difference in estimators was significant using with z � 0.19, indicating that the difference in estimators was not significant.e sign for CL was negative for both groups indicating that as either group reported more frequent occasions of being involved in collaborative settings, overall CLA scores were predicted to decrease.For freshman, RI and LS appear to predict CLA scores positively, such that for freshmen who reported high frequencies of each behavior, their CLA scores were predicted to increase albeit in small amounts.For example, a one-unit increase of RI was estimated to increase CLA scores by 1.48 or well less than a 1% change.Seniors reported that working with and being exposed to others different from themselves tend to increase CLA scores, again small in magnitude.e small magnitude of change in CLA scores for seniors was similar to what was found with freshmen.e next phase of analysis was driven by the results above, whereas only the items measuring those engagement indicators (EI) that were found to have coefficients different from zero would be included in the subsequent analysis.is lower level analysis examined the relationships of the individual items that compromised each of the four engagement indicators: Discussion with Diverse Others, Reflective and Integrated Learning, Student-Faculty Interactions, and Collaborative Learning.
e descriptive statistics for the lower level items are reported in Table 5.For a complete definition of the items listed in Table 5 see http:// nsse.indiana.edu/html/summary_tables.cfm.
It was decided to continue to use maximum likelihood estimation here as with many survey-derived data sets; missing values occur randomly and therefore limit sample sizes when convention OLS regression is employed.
e first EI to be examined at the individual item level was that of Collaborative Learning-or the frequency of how often student collaborated with others as measured by four different behaviors.As seen in Table 6, the results indicate that not all behaviors contribute to the prediction of the CLA score.When considering both groups of students, three of the four items that comprise this EI had coefficients significantly different from zero.It appears that, as students ask others more frequently for help, CLA scores are predicted to decline.However, when looking across groups, studying with others fails to retain a significant coefficient.Asking for help and explaining to others retain significant estimators for both, with senior students having larger estimates.To determine whether the estimators were significantly different from each other, using equation ( 1) again, we found that z � 0.84 and −0.91, revealing that the differences reported were not significant for either item across student class.It is suggested that these be interpreted as measures or indicators of student preparedness or understanding of the material.at is, as students need more help from peers, it indicates lower confidence or perceived ability, yielding the negative value for the predictors.Explaining to others conversely maybe an indicator of mastery of material or that explaining to others reinforces knowledge, skills, of confidence in the student's perception of themselves as learners.In relation to each other, asking for help appears to exert the greatest influence on the CLA in terms of the items for this indicator for all students, and this relationship appears to change when groups are considered, albeit slightly.A oneunit change in explaining to others was predicted to increase CLA scores about 1.5% and 2.4% for freshmen and seniors, respectively, small but potentially influential.
e second engagement indicator analyzed was that of Reflective and Integrative Learning (RI) reported in Table 7.When all students were included, two of the seven items were found to have significant predictors.e group analysis is quite revealing as not one of the items for freshmen has significant estimated coefficients, while for senior students, four items that range from moderate to highly significant predictors. is may reflect the value added from the educational experiences accumulated by the senior students.Conversely, freshmen had yet to benefit from the educational environment to the university and therefore may have been reporting based on prior experiences.
Senior class students were found to have the only significant results for any of the items defined as Reflective and Integrated Learning.
e connect item asks students how often they connect ideas from their courses to prior e reported positive relationship predicts that, for students who reported more opportunities to connect the learning to prior knowledge and experiences, their scores on the CLA were predicted to be higher.erefore, as students believe they are more reflective and integrate prior learning with new material and courses, students score higher on the CLA and they engage more in such thinking.In fact, this was the strongest relationship found both in terms of frequency and percent of items.Also, the coefficient for connect with others was the largest found where a one-unit change was predicted to improve CLA scores by 3.8%.
e next item with significant estimated coefficients is diverse perspectives, where an increase in discussions       8.For all students of the four items for this indicator, only discussions with others as defined by race was found to have a coefficient different from zero. is may not be surprising for students attending this campus where the student population is fairly homogeneous across wealth, religion and political perspectives.For example, 63% of the student body is black, while economically 97% of students are classified as in need of financial assistance.While no data are available for political and religious diversity, all indicators appear to support the claim of homogeneity.
erefore, the most frequent opportunity for interaction with diverse others is by race, which as reported below may provide students opportunities to encounter opinions and beliefs defined across racial boundaries given that there are differences between how the two classes of students have this indicator, predicting CLA performance.
e results are different when reported across the two student groups with freshmen having two items with significant predictors and seniors none.e signs for the two coefficients appear to convey very different perceptions of how economic and political diversity are translated into CLA scores.Discussion with others from different economic statuses was estimated to improve CLA scores about 2.3% for freshmen.
It is seen as freshmen more frequently reported discussion of educational experiences with those of different economic means, CLA scores are predicted to increase while the converse was reported when discussions with those with different political views are part of the students' discussions.Freshmen may be reflecting in their scores previous life experiences, and given the nature of those experiences, how they are reflected in CLA scores may indicate the need to engage in such activities.
e final group of items to be analyzed were those from the EI of Student-Faculty Interactions.e results seen in Table 9 show that, for all students in the sample, the items addressing students working with faculty other than course work (committees, student groups, etc.) was found to have coefficients different from zero.
is was also true for senior students but not freshmen.e negative predicted coefficient may point to the fact that, as students engage in activities not directly associated academic classwork, CLA scores are estimated to decline. is decline may result from the additional time away from studies as a result of the commitment these activities require.Or that the nature of these activities is such that they do not provide experiences that improve the type of thinking and problem-solving as measured by the CLA.Or stated differently may reveal that as students were more involved in work outside the classroom, the demands these activities presented as far a time and effort, the return to critical thinking as measured on the CLA was negative for about a 1.2% decrease in CLA scores.
ere were nineteen items explored in this part of the analysis and on seven occasions when both groups of students were included significant predictors were estimated.For freshmen, this happened four times (21%) and seven (37%) for senior students.Only twice (5%) did both senior and freshmen have significant predictors (asking for help (−) and explaining to others (+)).Clearly, two patterns were revealed from both analyses (1) that for freshmen and seniors, the relationship between these NSSE items and CLA performance is different; (2) there was little agreement between the two groups of students; and (3) overall the relationship between NSSE and CLA is tenuous, complex, and may be of limited value.

Discussion
While the literature is saturated with studies reporting the linkage between engagement and beneficial student experiences, it seems plausible that associations should exist between the items surveyed on the NSSE and student performance measures.While such a linkage has been demonstrated on broad measures such as retention and graduation rates, linking the results from the NSSE to other measures of student learning remains elusive [11,44].
is current study of the relationship between NSSE engagement indicators and CLA scores yielded outcomes that were consistent with those of the work of Gordon et al. [11] when they found that the NSSE indicators were not reliable predictors of student GPA, and this study extends this outcome to another measure of student performance.
Given this, the results presented herein call into question the acceptance of the hypothesis presented by Carini et al. [46].eir hypothesis stated students with higher ability, as measured by tests such as the SAT, may benefit less from student engagement elements.Gordon et al. mentioned this in their study as students in their sample had an average SAT score of 1340, implying that the relationship between NSSE measures and GPA would be weak as a result of the high mean SAT scores.However, students in this study had a mean score SAT of approximately 861.If students with lower ability are to benefit more from engagement, then the weak relationship between these two measures is curious.More completely, if the above were present, it is suggested that lower ability students would show stronger relationships to the NSSE indicators and that these indicators would be better predictors of GPA.
e authors in the Carini et al.'s study proposed a term called "educational efficiency" that suggests institutions may experience a variation in how the engagement indicators associate with various learning outcomes.As a reference, the Gordon et al.'s study reported of the twelve opportunities for the NSSE benchmarks to correlate with GPA; it occurred 4 times, and in this present study, this did not occur for any of the ten indicators.
erefore, the proposed relationship between lower ability students benefiting more from engagement may not be present.In fact, it seems that, for this group of students with SAT scores some 450 points below of those in the Gordon et al.'s study, there are is a much weaker overall relationship between these measures.e measures were all consistent for all freshmen and senior class students.
Results of previous research attempting to link the NSSE benchmarks and measures of student learning have shown weak associations as presented in the work of Carini et al.Education Research International [46,53,55,56]; therefore, it is not surprising in the current study that the reported relationships between the current NSSE engagement indicators and CLA performance was limited.Only four of the ten engagement indicators were seen as a reliable predictor of performance on the CLA.While this is better than the relationship between the NSSE and GPA, the association between engagement and achievement appears to be infrequent.Examining the item level findings reveal that, although there were four indicators found to be significant predictors of CLA performance, the connections are limited.
e engagement indicator of Collaborative Learning having two of four items showing significance is consistent with what Fiorini, et al. found albeit again in their study that associations were found on the more global measures of student success such as graduation.
Research on the effect of diversity experiences on student self-reported gains in various measures of cognition is welldocumented [57,58].In their study, Pascarella extended this work to use a standardized measure of critical thinking and found results supporting the original work and revealing any such relationships were complex.erefore, it was not surprising to find that the indicator Discussions with Diverse Others was a significant predictor of CLA performance.While not mirroring the complexity developed by Pascarella, the results presented herein show that this relationship is limited in scope as few of the item-specific measures yielded significant relationships with performance.Specifically, for all students, only discussions with diverse others defined by race proved to be a reliable predictor of CLA performance, and this was different for freshmen and senior students.Although two different approaches were used to investigate the idea of diversity and critical thinking, a degree of commonality was found. is lends support that students do benefit from interacting with others different from themselves.However, in this case, it was limited to freshmen, therefore implying that the college experience has a possible moderating effect on this interaction as a result of this university being more homogeneous than many high schools from which the freshmen graduated from.
e point being the paucity of relationships found in this study between the NSSE and the CLA illustrates the difficulty and complexity of finding measures of student learning and student behaviors that can be measured in such a way to provide beneficial information to administrators, policymakers, and other stakeholders.is is problematic as intuitions continue to rely on student self-reports as measures of behaviors, learning, and perspectives of the quality of education they receive exam-based [59].

Implications.
e results are somewhat consistent with those of the larger studies that developed associations between ways to measure institutional effectiveness in terms of student learning as this study exclusively examined an HBCU with high percentages of Black students as compared to predominately white institutions (PWI).e results appear to offer that institutions serving minority students face similar issues as they strive to construct approaches to determine what institutional and faculty-based practices contribute to improvements in student learning.Taking this further, race or even gender may not be the student characteristics associated with attempting to understand how engagement should be considered as a tool to refine learning environments.Developing this, if students of diverse racial backgrounds experience the effects of engagement in a similar manner, then possibly, it is the school context that matters the most and not the individual student which aligns with the arguments posited by Kuh et al. [60].It has been presented that [61] using data derived from student self-reported approaches may yield little in terms of information upon which to base improvement efforts. is may account for some of the perplexing results reported earlier.e potential for students to interpret the intent of a question fail to accurately remember personal and academic experiences; therefore, basing perceptions on  Aside from the issues associated with self-reported data, what policy implications does this study suggest?If institutions are interested in using data to drive improvements, a well-designed plan needs to be implemented that (a) either uses a probabilistic random sampling process to improve the potential to generalize or require key surveys or assessment activities to include all students, (b) invests in institutional research efforts focused on improving student outcomes, and (c) supports faculty involvement to engage in assessment activities as faculty are directly responsible for the types of engagement and learning activities that may have the potential in effecting student outcomes.is study did match the individual student survey and performance data; however, limitations in generalizability may be attributed to the sampling processes used by the university.While the analysis did show that including student demographic characteristics failed to improve the power of the models, this may have been more a result of sample size and not model misspecification as the researcher did face sample size limitations such as attempts made to develop alternative models.

Recommendations.
Researchers and policy-makers are plagued with a multitude of technical and practical issues especially when examining the utility of attempting to associate student learning and institutional behaviors at the higher education level [38].As Porter [59] recommends, assessing student learning with instruments such as the NSSE is problematic at best, arguably inappropriate, and should be discontinued.While possibly well-suited to link to gross measures of institutional performance such as retention and graduation rates [44], the NSSE benchmarks appear to be not well-suited in associating with specific measures of student learning; specifically, the CLA.e root cause of this can be argued, regardless the conclusion appears to be well-supported by this study and others.
Where does this leave researchers and policy-makers interested in understanding those actions an institution can take to improve learning (quality) not just graduation (quantity), as many private and public sector members are questioning and demanding that institutions clearly demonstrate the worth of a college education [62,63].erefore, searching for alternatives to assessing what defines a successful the college is complex and varied therefore utilizing such metrics as returns on investment [64] remain relevant.Taking this into consideration, measuring the quality of an education and the potential worth is critical for students and society.It is important to continue to research how institutions can not only demonstrate that they do produce quality graduates but also must understand how they did it.

Table 1 :
Select demographic variables for freshmen and senior students.

Table 2 :
Descriptive statistics for aggregate regression variables.

Table 3 :
Regression results for GPA and NSSE engagement indicators with robust standard errors.

Table 4 :
Regression results for CLA total score and NSSE engagement indicators with robust standard errors.(how a being exposed to a different view caused you to form a new one) are curious.It would seem that either of these would prompt different and deeper thinking in students; yet, this is not reflected in improved CLA scores.It just may be that the CLA offers problems and measures thinking that fails to utilize such skills.
Education Research Internationalrepresenting diverse views was found to predict higher CLA scores.Intuitively, this appears correct as being exposed to different views should cause you to be more reflective and consider more information before forming opinions or solving problems.e negative predicted coefficients for societal (connecting your learning to societal problems or issues) and new view

Table 5 :
Descriptive statistics for engagement item level regression variables across all, freshmen, and senior class students.
Note: highest score is 4.0.

Table 6 :
Regression results for CLA total score and collaborative learning items with robust standard errors.

Table 7 :
Regression results for CLA total score and reflective and integrative learning items with robust standard errors.

Table 9 :
Regression results for CLA total score and student-faculty interaction items with robust standard errors.

Table 8 :
Regression results for CLA total score and discussion with diverse others items with robust standard errors.p<0.05; * * p < 0.01; * * * p < 0.001; all estimated constants are significant at p < 0.001.8EducationResearch International faulty or incomplete information all can contribute to biased and inconsistent results and large numbers are needed to support efforts to comprehend how engagement is manifested in student outcomes.