The aim of the present study was to explore students’ learning-related cognitions prior to an in-class achievement test, with a focus on metacognitive strategy use. A sample of 70 students in grade 11 (58.6% female,
Metacognitive competencies have over the past three decades developed from a largely neglected issue to one of the most elaborated areas of theory and research in the educational sciences [
Despite considerable research on metacognitive strategies, several questions warrant further investigation. More specifically, how often and when are metacognitive strategies applied in a learning process (e.g., when is it best to start to prepare for a test)? Which metacognitive learning strategies are most commonly employed in actual learning situations and most effective with respect to academic performance? Whereas previous research has consistently evaluated the relative utility of metacognitive strategies, it is also important to investigate the frequency and effectiveness of metacognitive strategy use as evaluated during the actual learning process [
The objective of the present study was to evaluate the relation between students’ use of metacognitive strategies as assessed during the actual learning process preceding an achievement test. To this end, the experience sampling method (ESM; [
Put simply, the term
Although a number of metacognitive regulatory strategies has been examined [
Given the above findings concerning the co-occurrence of these three strategies, particularly with respect to monitoring, this metacognitive strategy has been a focus of particular empirical interest in the context of self-regulated learning. For example, incorporated into the theoretical model of Winne and Hadwin [
According to Winne and Hadwin [
Regulating one’s own learning through the use of metacognitive strategies is widely considered to be the most elaborated form of learning (e.g., [
Some correlation studies show the use of metacognitive strategies to positively correspond with academic achievement (e.g., [
Concerning the moderating effect of academic environments on self-regulated learning, it is assumed that certain academic settings afford more limited opportunities for engaging in self-regulation than do others [
At present, there exist two major approaches toward the assessment of metacognitive strategy use involving either externally observed measures of behavioral or physiological indicators, or self-report methods. Whereas behavioral or physiological indicators are highly reliable and less subject to participant bias (e.g., [
Among the most popular methods for assessing students’ use of metacognitive strategies are questionnaires (e.g., Motivated Strategies for Learning Questionnaire, MSLQ; [
In contrast to the above methods, think-aloud protocols (e.g., [
Taken together, the limitations of previous assessment methods suggest that more ecologically and empirically valid methods are required to better evaluate metacognitive strategy use as it naturally occurs in real-life learning situations, thereby contributing to a more dynamic, differentiated, and ecologically valid understanding of the nature of self-regulated learning [
In adapting ESM procedures to the study of metacognitive strategy use, the present study aimed to provide ecologically as well as empirically valid data in support of the effectiveness of metacognitive strategies used by students during the weeks prior to an achievement test. As such, this research aims to contribute to the self-regulation literature by addressing significant limitations of both self-report (questionnaire, interview, think-aloud) and more objective assessment methods (behavioral, physiological). More specifically, the use of ESM protocols allowed for research questions concerning the frequency, timing, as well as achievement benefits of metacognitive strategy use to be evaluated as assessed during the weeks prior to a class test. Consistent with the assertion that students’ use of self-regulated learning strategies be evaluated as an ensemble comprising more than just the sum of its parts [
The present study aimed to explore students’ real-life test-related cognitions (more specifically if and when they thought about the test) and use of metacognitive strategies prior to an achievement test. In so doing, the present study focused on the learning process with respect to changes over time in the frequency of test-related cognitions in general, the involvement of metacognitive strategies, as well as predictive relations between test-related thoughts or specific strategies and performance. Through the use of the experience sampling method [
Research Hypotheses.
Hypothesis 1 | Hypothesis 2 | |
---|---|---|
Frequency and effects of test-related cognitions | Frequency and effects of metacognitive strategies | |
a | Test-related cognitions are linked to learning-related situations and are observed more often as the test date approaches. | Metacognitive strategies are interrelated and used more often as the test date approaches. |
b | Test-related cognitions correspond to performance improvement. | Metacognitive strategies correspond to performance improvement. |
How often and when do students occupy themselves with thoughts concerning an upcoming test in mathematics during the 14 days preceding the test, and is this cognitive engagement related to their test performance?
Based on the assumption that optimal self-regulated learning involves the ability to disengage from achievement-related cognitions in situations not consistent with learning and achievement, we anticipate that students will more frequently report test-related cognitions during learning-related situations (e.g., mathematics or learning situations) than during their leisure time. Furthermore, considering the goal-oriented nature of learning behaviors, we expect students to think more often about the test as the test date approaches.
We expect that the frequency of global test-related cognitions will correspond to performance improvements on the subsequent test based on the assumption that such cognitions are associated with the use of metacognitive strategies.
When occupied with thoughts about the upcoming test, which metacognitive strategies do students report, how does the use of these metacognitive strategies develop as the test date approaches, and to what extent are specific strategies related with test performance?
We anticipate that students’ global test-related cognitions will positively correspond with each of the metacognitive strategies of planning, monitoring, and regulation. Furthermore, strong positive correlations are expected between the three strategies, and each metacognitive strategy is expected to demonstrate a increasing growth curve over time consistent with the anticipated curve for global test-related cognitions.
We hypothesize that students’ use of each metacognitive strategy will be related to subsequent improvements on the next class test. Monitoring is expected to be most positively related to test improvement as suggested by the aforementioned research in which this specific strategy is examined.
Data was collected through the use of the experience sampling method [
Participation in all parts of the study was voluntary and all responses were anonymous. In addition to the ESM data collection, achievement data was collected consisting of students’ self-reported grades on their most recent test as well as actual grades on the upcoming test for which they were currently preparing as obtained from the mathematics teachers.
Students’ self-reported grades on their most recent mathematics test, as well as their actual grades on the subsequent test, were assessed as measures of academic achievement. In the German school system, grades range from 1 (very good) to 6 (failed). Students’ grades were inverted prior to analysis allowing for higher scores to reflect better academic achievement. The first test in mathematics (
It is important to note that by utilizing residual as opposed to difference or gain scores, it is not the improvement in terms of a difference in average change between the two tests that is measured, but rather the extent of improvement on the second test relative to the first test controlling for initial levels on the first test. In other words, residual scores allow for analysis of levels of the second test over and above what can be predicted by first test. As such, the use of residual scores allowed for a more sensitive analysis of changes in grades relative to prior achievement, as opposed to changes in raw achievement scores over time.
To avoid having students complete overly long state-based questionnaires, the test-related cognition and metacognition constructs were assessed using single-item measures. This practice is consistent with similar ESM research on academic emotions [
The contingent assessment of the three strategy questions following the initial test-related cognition item was important in order to more innocuously assess their use among students who were already thinking about the test, as opposed to unintentionally highlighting these strategies and encouraging their use by having students first consider the specific strategies and then reflect on their relevance to their upcoming test. Nonetheless, this two-step process still allowed for a differentiated assessment of metacognitive strategies in that by having highly similar wordings in the second set of questions in which only the strategy type differed, students who were already reflecting on their upcoming test were assumed to be better able to recognize and report on these more specific elements of test-related cognitions. In light of global test-related cognitions being a requisite for the subsequent reporting of metacognitive strategies, students who indicated no level of thought about the upcoming exam were provided a score of 0 (
The study data obtained reflects a two-level structure consisting of points of assessment (Level 1;
To evaluate the extent to which the experience sampling method interfered with the learning process, the participants’ achievement scores on both tests in mathematics were compared to the scores of other students from the same classes (
Some level of thought concerning the upcoming mathematics test over the previous hour was reported by students during 23% of the assessment periods. Results also supported our hypothesis that test-related cognitions would most frequently occur in learning situations, with students found to report thinking about the test, at least once, 56% of the time in mathematic classes, 15% of the time in other classes, 24% of the time while completing homework for all subjects, and 40% of the time spent learning for all subjects (e.g., reviewing class materials, preparing for a test at home), as compared to during 14% of leisure time and 15% of the time spent on additional activities. Figure
Average intensity of test-related cognitions.
Concerning the development of students’ occupation with thoughts about the upcoming test, we calculated the average frequency of students’ test-related cognitions across all assessments administered during a given day. As the questionnaires were completed at randomized intervals, we assumed that the average of all measures completed during each of 14 days prior to the mathematics test should provide a good estimate of the relative intensity of students’ test-related thoughts during that day. To assess change over time in test-related cognitions, each student’s development was represented by an individual polynomial growth trajectory based on a unique set of parameters, as reflected by a polynomial Level 1. By subsequently adding polynomial parameters of a higher order (linear, quadratic, cubic) until the beta weight for the fixed parameter of the highest polynomial predictor was not significant, the most accurate shape for the polynomial growth curve was identified. If the beta weight of the highest polynomial order predictor was not significant, this indicated that no additional within-person variance could be explained by increasing the polynomial degree of the growth function and this parameter was excluded (cf., [
With respect to the manner in which time was coded [
Analysis of change in test-related cognitions prior to test completion.
Basic model | Linear model | Quadratic model | MG + quadratic model | |||||
---|---|---|---|---|---|---|---|---|
|
SE |
|
SE |
|
SE |
|
SE | |
Fixed effects | ||||||||
Intercept (I) | .52 | .05 | .87** | .10 | 1.19** | .15 | 1.19** | .15 |
|
.19* | .08 | ||||||
Linear Slope (LS) | .05** | .01 | .19** | .04 | .19** | .04 | ||
|
.02* | .01 | ||||||
Quadratic Slope (QS) | .01** | .002 | .01** | .002 | ||||
| ||||||||
Var | Var | Var | Var | |||||
| ||||||||
Random effects | ||||||||
Between: I | .100** | .405** | .776** | .695** | ||||
Between: LS | .003** | .033* | .031* | |||||
Between: QS | .000 | .000 | ||||||
Within | .577a | .487a | .443a | .443a | ||||
| ||||||||
Model statistics | ||||||||
Deviance | 1503.04 | 1420.01 | 1381.23 | 1377.15 | ||||
No. parameters | 3 | 6 | 10 | 12 |
+
Note: Intercept represents the values one day before the test. State measures across one day were aggregated.
Results showed the majority of the variability to reside within individuals
Based on the manner in which time was coded, the growth parameters are interpretable in the following manner (Table
Average intensity of test-related cognitions.
The random effects assessed in the model provide information as to whether the shape and position of the parabola varied between participants. Results showed significant variability in the intercept (variance of .415 one day before the test) and in the linear component of the trajectory (variance of .034 one day before the test). In contrast, the variability of the quadratic component of the trajectory was not significant, implying substantial variability in the relative position of the growth curve across students, and relatively little variability in its curvature.
Having identified the polynomial function that best represents change over time in test-related cognitions, test performance improvement from the first to the second test was entered as a predictor of the different growth coefficients, namely, the individual intercept and the linear slope on Level 2 (Table
Based on the manner in which time was coded [
A significant effect of .19 for test improvement on the intercept was found suggesting that the more students thought about test, especially one day prior to the test date, the more they improved on this test in relation to the preceding test. A significant effect of .02 for test improvement was also found on the linear slope suggesting, in combination with a visual inspection, that increased frequency over time in test-related cognitions, particularly on the final day before the test, contributed to improved test performance. As the magnitude of this effect is contingent upon the coding of time (
Average intensity of test-related cognitions by improvement classification.
Results showed that students’ thoughts about the upcoming mathematics test over the previous hour were accompanied by reports of engaging in at least one of the metacognitive strategies 86% of the time. More specifically, time spent thinking about the test was accompanied 75% of the time by students reminding themselves of what they had to learn for the test (planning), 63% of the time by students monitoring their learning, and 41% of the time by students thinking at least once about whether to change their learning process (evaluation) over the previous hour. The average intensity of metacognitive strategies when test-related thoughts were reported indicate that planning (
Correlations between test-related cognitions and metacognitive strategies.
Planning | Monitoring | Evaluation | |
---|---|---|---|
Monitoring | .57*** | — | |
Evaluation | .37*** | .45*** | — |
Test-related cognitions | .59*** | .50*** | .25*** |
Note:
To further analyze change over time in these metacognitive strategies over the 14 days prior to the mathematics test (Tables
Analysis of change in planning strategies prior to test completion.
Basic model | Linear model | Quadratic model | MG + quadratic model | |||||
---|---|---|---|---|---|---|---|---|
|
SE |
|
SE |
|
SE |
|
SE | |
Fixed effects | ||||||||
Intercept (I) | .41** | .04 | .78** | .09 | 1.06** | .13 | 1.06** | .13 |
|
.09 | .07 | ||||||
Linear Slope (LS) | .06** | .01 | .18** | .03 | .18** | .03 | ||
|
.01 | .01 | ||||||
Quadratic Slope (QS) | .01** | .001 | .01*** | .002 | ||||
| ||||||||
Var | Var | Var | Var | |||||
| ||||||||
Random effects | ||||||||
Between: I | .068** | .296** | .595** | .578** | ||||
Between: LS | .002** | .023+ | .023+ | |||||
Between: QS | .000 | .000 | ||||||
Within | .497a | .420a | .386a | .386a | ||||
| ||||||||
Model statistics | ||||||||
Deviance | 1400.07 | 1311.45 | 1276.15 | 1274.64 | ||||
No. parameters | 3 | 6 | 10 | 12 |
+
Note: Intercept represents the values one day before the test. State measures across one day were aggregated.
Analysis of change in monitoring strategies prior to test completion.
Basic model | Linear model | Quadratic model | MG + quadratic model | |||||
---|---|---|---|---|---|---|---|---|
|
SE |
|
SE |
|
SE |
|
SE | |
Fixed effects | ||||||||
Intercept (I) | .34** | .04 | .64** | .08 | .90** | .12 | .90** | .12 |
|
.14* | .07 | ||||||
Linear Slope (LS) | .05** | .01 | .16** | .03 | .16** | .03 | ||
|
.01* | .001 | ||||||
Quadratic Slope (QS) | .01** | .002 | .01** | .001 | ||||
| ||||||||
Var | Var | Var | Var | |||||
| ||||||||
Random effects | ||||||||
Between: I | .052** | .252** | .524** | .463** | ||||
Between: LS | .002** | .021+ | .019+ | |||||
Between: QS | .000 | .000 | ||||||
Within | .381a | .322a | .293a | .292a | ||||
| ||||||||
Model statistics | ||||||||
Deviance | 1232.34 | 1152.43 | 1111.16 | 1107.86 | ||||
No. parameters | 3 | 6 | 10 | 12 |
+
Note: Intercept represents the values one day before the test. State measures across one day were aggregated.
Analysis of change in evaluation strategies prior to test completion.
Basic model | Linear model | Quadratic model | MG + quadratic model | |||||
---|---|---|---|---|---|---|---|---|
|
SE |
|
SE |
|
SE |
|
SE | |
Fixed effects | ||||||||
Intercept (I) | .17** | .03 | .27** | .05 | .35** | .08 | .35** | .08 |
|
.07 | .04 | ||||||
Linear Slope (LS) | .02** | .005 | .05* | .02 | .05* | .02 | ||
|
||||||||
Quadratic Slope (QS) | .003* | .001 | .003* | .004 | ||||
| ||||||||
Var | Var | Var | Var | |||||
| ||||||||
Random effects | ||||||||
Between: I | .029** | .081** | .200** | .188** | ||||
Between: LS | .000* | .010* | .009* | |||||
Between: QS | .000 | .000 | ||||||
Within | .142a | .133a | .125a | .353a | ||||
| ||||||||
Model statistics | ||||||||
Deviance | 628.66 | 604.49 | 582.21 | 579.14 | ||||
No. parameters | 3 | 6 | 10 | 12 |
+
Note: Intercept represents the values one day before the test. State measures across one day were aggregated.
Change in frequency of planning, monitoring, and evaluation strategy use.
Consistent with the achievement analysis for test-related cognitions,
Change in frequency of monitoring strategy use by improvement classification.
The present findings suggest that students did indeed think about their upcoming test in mathematics during nearly a quarter of the experience sampling assessments obtained during the 14 days prior to test completion. Further, this result provides evidence in support of evaluating domain-specific test completion, in the present case with respect to mathematics class, as a specific achievement goal that students are acutely aware of and explicitly think about on a regular basis [
Nonetheless, students were also found to report not thinking as often about the test during their leisure time, indicating that students are also able to disengage from test-related thoughts in situations not related to academic achievement. According to Boekaerts [
Concerning changes in students test-related thoughts as the test date approached, the development of test-related cognitions was best reflected by a quadratic curve implying that not only did students think more about the test over time, this growth in the frequency of test-related thoughts also further increased as the test date neared. This pattern of change in test-related thinking is perhaps not surprising as the incremental cognitive investment toward achievement goals allowing for goals with more immediate deadlines to be most fully pursued (i.e., tests in other classes) is assumed to be a critical component of self-regulated learning [
The intercept as well as slope for the frequency of test-related thinking over time were found to positively correspond with improvements in test performance. This finding suggests that despite the highly generalized nature of the present test-related cognition measure, it nonetheless appears to be consistently associated with more specific cognitions that more directly contribute to improvements in test performance, namely, metacognitive learning strategies (cf., [
To more explicitly address the above assumption that test-related cognitions imply a potential ensemble of learning strategies, analyses further revealed that at least one of the three metacognitive strategies assessed in this study was typically reported if test-related thinking was indicated. More specifically, the strategy of planning, defined as thinking about what to learn for the test, was most frequently reported, followed by the strategy of monitoring, operationalized as reviewing existing knowledge as it applies to the upcoming test. In contrast, evaluation was applied far less often and also reported to occur less frequently over the course of the previous hour than were the other two strategies. Correlations further showed planning and monitoring to have a strong positive relationship, with evaluation also showing positive albeit notably weaker relations with these two strategies.
These findings are in line with previous findings showing these metacognitive strategies to be highly interdependent [
The observed changes in the use of these three metacognitive strategies parallel those observed for test-related cognitions in that all three growth trajectories were best reflected by a quadratic growth curve. However, whereas the linear and quadratic slope parameters for the strategies of planning and monitoring were very similar to each other, the development of evaluation strategies over time was far more flat and constant in nature. This finding is further consistent with previous findings showing these metacognitive strategies to not be related to each other in a hierarchical or sequential manner, but rather to occur simultaneously toward the completion of an achievement goal [
In contrast, the test improvement measure was found to positively correspond with the intercept as well as linear slope for the strategy of monitoring. This finding suggests that the more monitoring was evidenced by students especially during the final days before the test, and the more monitoring increased during the final days before the test, the better they performed on this test relative to their previous test in mathematics class. Similar relations were not found for the strategies of planning and evaluation.
These significant results underscore the importance of monitoring as a critical metacognitive strategy that, according to Winne and Hadwin [
The present study findings thus demonstrate a significant relationship between students’ engagement in metacognitive strategy use with respect to an upcoming test in mathematics and their actual performance on this test. Further, these results highlight the importance of evaluating not only the overall frequency with which such strategies are employed, but also change over time of the use of metacognitive strategies during the learning process prior to test completion. Findings revealed the intercept as well as growth in the frequency of monitoring the learning process to be significantly related with improvements in test grades relative to prior test performance. Taken together, the results of the present study provide empirical support for the temporal relationship between metacognitive strategy use and achievement over and above the findings of correlational questionnaire studies (cf. [
As suggested by the present findings showing the experience sampling method to be a worthwhile instrument for evaluating self-regulated learning processes, future studies in which such elaborated assessment methods are employed are recommended to further explore students’ learning behaviors in real-life achievement settings. Similarly, greater methodological research in which better and additional measures of validity and reliability for data obtained from experience sampling methods is required. It is anticipated that future research in which additional learning strategies are explored using such data collection and statistical methods can provide considerable insight into students’ use of such strategies in actual learning situations. For example, these studies could evaluate metacognitive as well as more specific cognitive learning strategies used by students when presented with learning material [
Future ESM research in which objective evaluations of specific elements of the students’ learning environment are also assessed should serve significantly to complete our understanding of how and why metacognitive and learning strategies are employed by students in specific learning situations. By assessing not only how students regulate their learning process but also the extent to which classroom teachers as well as goal structures promote mastery of learning material, as opposed to performance on achievement tests (see [
Similarly, it is anticipated that such findings from studies in which situational factors are more fully explored should serve to inform efforts to improve classroom environments as well as develop effective intervention programs for struggling students. Upon identifying aspects of the classroom environment that facilitate or prevent the use and effectiveness of self-regulated learning strategies, such findings can be readily incorporated into teacher education programs. As such, it should be possible to inform preservice and practicing teachers of the importance of creating optimal classroom settings for fostering students’ self-regulatory competences and increase their awareness of the necessity and utility of students’ self-regulatory efforts (see [
Finally, further research on the moderating effect of situational factors on the use and achievement benefits of metacognitive and cognitive learning strategies should also serve to enhance the effectiveness of related intervention programs. For example, the lack of change in the use of evaluation as a metacognitive strategy needs further research with respect to the impact of classroom features on students’ use of this strategy such as test format, lesson structure, or explicit guidance concerning the effective use of self-regulated learning strategies by the instructor. By better delineating the effects of classroom and instructional dynamics, as compared to dispositional factors (e.g., individual differences in motivational strategies) on students’ metacognitive strategy use, we can design better intervention programs that target the critical sources of maladaptive learning approaches (e.g., having a focus on teacher training versus improving student motivation). In summary, the present study illustrates that students do indeed have the ability to effectively use metacognitive strategies, particularly the strategy of monitoring, and in so doing, can positive impact their subsequent achievement. Nevertheless, this research also suggests further avenues of research into the intentionality of changes in students’ self-regulatory strategy use over time, possible changes in related self-regulatory strategies over time (e.g., to regulate one’s motivation, emotions), exploring and improving the use and effectiveness of specific learning strategies (e.g., evaluation, elaboration), as well as the potential moderating effects of instructional methods and classroom features.
This project is based on data from the doctoral dissertation of the first author.
It should be noted that the present sample affords a three-level data structure consisting of measurement points within individuals within classes. The intraclass correlations (
Achievement on the second test in mathematics is largely predicted by previous test performance, and this relationship may, in turn, influence the regression weights of the growth parameters. As such, what we assessed were the residuals of performance on the second test obtained after variance explained by the first test was removed. This score is thus referred to as test improvement and describes the relative improvement over time from the first test to the second, with students who performed better than expected considering their previous test score obtaining a score above zero, and those who performed worse than expected considering their previous test score obtaining a negative score. Integrating performance on the first test in the model as a predictor follows from the assumption that the prior achievement may influence learning behavior. Although including test improvement as a predictor of prior learning behavior is counterintuitive, it was nonetheless included as such due to the significance of the relationship between the individual growth parameters and test improvement being accurately reflected by the regression results despite an inverse direction of causality being assumed.
Applying the HLM notation, the resulting equations were as follows: