Education Research International
Volume 2019 (2019), Article ID 3402035, 19 pages
https://doi.org/10.1155/2019/3402035

Review Article

Learning Mathematics in Metacognitively Oriented ICT-Based Learning Environments: A Systematic Review of the Literature

Lieven Verschaffel,¹ Fien Depaepe,¹ and Zemira Mevarech²

¹Center for Instructional Psychology and Technology, KU Leuven, Leuven, Belgium
²David Yellin Academic College of Education, Jerusalem, Israel

Correspondence should be addressed to Lieven Verschaffel; lieven.verschaffel@kuleuven.be

Received 4 March 2019; Revised 30 June 2019; Accepted 21 July 2019; Published 16 September 2019

Academic Editor: Christos Troussas

Copyright © 2019 Lieven Verschaffel et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract.

This article encompasses a systematic review of the research on ICT-based learning environments for metacognitively oriented K-12 mathematics education. This review begins with a brief overview of the research on metacognition and mathematics education and on ICT and mathematics education. Based on a systematic screening of the databases Web of Science and ERIC wherein three elements—ICT-based learning environments, metacognitive pedagogies, and mathematics—are combined, 22 articles/studies were retrieved, situated at various educational levels (kindergarten, elementary school, and secondary school). This review revealed a variety of studies, particularly intervention studies, situated in elementary and secondary schools. Most studies involved drill-and-practice software, intelligent tutoring systems, serious games, multimedia environments, and computer-supported collaborative learning environments, with metacognitive pedagogies either integrated into the ICT software itself or provided externally by the teacher, mainly for arithmetic or algebraic word problem-solving but also related to other mathematical topics. All studies reported positive effects on mathematical and/or metacognitive learning outcomes. This review ends with a discussion of issues for further theoretical reflection and empirical research.

1. Introduction

In How People Learn, Bransford et al. [1] propose the support and exploitation of metacognition as one of the three core instructional design principles. This proposal is based on the vast amount of research showing that (a) learners with better metacognitive knowledge and skills will acquire domain-specific knowledge and skills more effectively and efficiently and will transfer their knowledge and skills more easily to other domains and contexts and (b) well-designed metacognitively oriented instruction embedded in a specific content domain has a positive impact on learners’ metacognitive behavior as well as their learning within that domain [2–7].

This article encompasses a systematic review of the research on learning mathematics in metacognitively oriented ICT-based learning environments: research describing and evaluating attempts to increase the power of ICT-based learning environments for K-12 mathematics education by enriching them with a metacognitive pedagogy, or the reverse, that is, attempts to increase the power of metacognitive pedagogies for K-12 mathematics education by embedding them in ICT-based learning environments. We begin with a brief overview, first, of the research on metacognition and mathematics education. Afterwards, we briefly review the research on ICT and mathematics education. Finally, we explicate the aims and scope of our literature review wherein these three elements—metacognition, ICT, and mathematics education—are combined.

1.1. Metacognition and Mathematics Education

Since the 1980s, it is commonly accepted that the cognitive system depends on higher-order processes that enable it to work efficiently. Examples of metacognitive components are planning, monitoring, control, and reflection (e.g., [8]). From the very beginning, researchers ([8]; see also [9, 10]) distinguished between two closely interrelated components of metacognition: (1) knowledge of cognition (e.g., knowledge about the task, strategies appropriate for solving the task, and personal characteristics relevant to the task) and (2) regulation of cognition (e.g., monitoring, control, and reflection). (The term “metacognition” is closely related to the term “self-regulation.” According to some researchers, self-regulation refers to the second component of the metacognitive system, namely, one’s ability to control, evaluate, and modify his or her cognitive processes in the pursuit of a particular goal [11]. Others conceive self-regulation more broadly, referring not only to the ways that learners systematically activate and sustain their cognitions towards the attainment of their goals but also to their behaviors, motivations, and affects [6]. In this article, we take the former perspective, implying that we conceive metacognition as the most general concept and self-regulation as the second, regulatory component of metacognition.)

While the early research on metacognition assumed that children younger than 10 years have little or no metacognition, from the late nineties on, researchers started to report evidence demonstrating that kindergarten children can already spontaneously plan, monitor, control, and reflect on their mathematics activities [12–15].

The relationship between metacognition and mathematical reasoning and problem-solving is well documented in the research literature (e.g., [16, 17]). Research has indicated that learners at all ages, from kindergartners to adult learners, who plan, monitor, evaluate, and reflect on their problem-solving processes, perform better on mathematics reasoning tasks and solve mathematical problems better than those who do not use these activities, or use them less frequently (e.g., [18]). This phenomenon has been documented by applying a large variety of off- and online measurements, including questionnaires, observations, think-aloud and retrospective techniques, (semi)structured interviews, various types of brain coding techniques, and analysis of peers explaining the solutions to one another or working in small groups. In general, these studies reported high positive correlations between metacognition and mathematical reasoning or problem-solving, even after controlling for IQ [13, 19].

Given the well-documented positive association between metacognition and mathematical reasoning or problem-solving [1], researchers started to look for ways of improving learners’ mathematical reasoning and problem-solving, based on the teaching of metacognitive knowledge and/or skills [6, 16]. Over the years, several such “metacognitive pedagogies” have been designed for the area of mathematics learning, for various age levels and embedded in different parts of the mathematics curriculum.

Most of these methods were routed in the seminal work of Schoenfeld [5, 20]. In his classic studies from the 1980s, Schoenfeld [20] worked with undergraduates on improving their problem-solving by regularly prompting them to consider metacognitive questions such as “What are you doing?” “How are you doing it?” and “How does it help you?” He found that, by asking students these questions throughout a one-semester course, the students themselves began asking and answering these questions, which in turn yielded greater metacognitive awareness of their problem-solving as well as more efficient problem-solving.

These results about the possibility of improving mathematical competence through metacognitively oriented instruction were confirmed in a number of subsequent studies throughout the nineties and into the 2000s not only with university students but also with learners as young as kindergartners (e.g., [16, 21–28]).

In particular, Kramarski, Mevarech, and colleagues engaged in a robust program of research related to metacognition instruction in mathematics for more than 20 years (e.g., [16, 25, 26]). They designed and investigated an instructional method that they refer to as IMPROVE, an acronym representing the series of following teaching steps:

(i)Introducing the new materials, concepts, problems, or procedures using metacognitive scaffolding(ii)Metacognitive self-directed questioning in small groups or individually(iii)Practicing by employing the metacognitive questioning(iv)Reviewing the new materials by teacher and learners, using the metacognitive questioning(v)Obtaining mastery on higher-order and lower-order cognitive processes(vi)Verifying the acquisition of cognitive and metacognitive skills based on feedback-corrective processes(vii)Enrichment and remedial activities

The core component of the IMPROVE program consists of training the learners to use four kinds of metacognitive self-directed questions:

(i)Comprehension: What is the problem all about?(ii)Connection: How is the problem at hand similar to or different from problems you have solved in the past?(iii)Strategies: What strategies are appropriate for solving the problem and why?(iv)Review: Does the solution make sense? Can you solve the problem differently, how? Are you stuck, why?

However, in many cases, the positive effects of all these interventions were moderate rather than strong, implying that improving learners’ competence in mathematical reasoning and problem-solving by means of metacognitively oriented interventions is a complex and demanding matter, which comes with a cost, in terms of effort, time consumption, and reduction of the curriculum in the traditional sense [13].

1.2. Metacognition and ICT

Similar to the research literature on metacognition and mathematics education, the literature on metacognition and ICT is immense. In the early days of ICT, a computer’s potential was mainly seen as a machine for drill-and-practice in basic arithmetic and algebraic skills [11]. But as a result of emergence of new theoretical approaches to learning and instruction (e.g., constructivism), on the one hand, and new technological advancements, on the other hand, researchers started to explore its potential for developing learners’ higher-order cognitive and metacognitive competencies by means of more advanced learning technologies such as intelligent tutoring systems, simulations, programming, serious games, hypermedia, computer-supported collaborative learning environments, and virtual reality [11, 29–31].

So, already for several decades, ICT is being considered a powerful tool for teaching higher-order processes such as metacognition and self-regulation, in various domains including mathematics. As argued by Mayer (2010, in [16]; p. 148), “ICT is particularly useful for approaching such complex and unfamiliar problem-solving tasks because it enables the learner to search for information on the web, look for similar problems and sub-problems on-line, and use various computerized tools that can carry out the tedious work that is sometimes associated with solving mathematics problems (such as plotting graphs or doing computations) and hence release cognitive energy for higher-order cognitive processes [32]. Furthermore, search tools and online information sources (…) may lead users to reflect on the given information, decide which piece of information is most applicable to the given problem (…). Computer supported collaborative learning (e.g., asynchronous learning networks, forums, even emails) can become a powerful reflection tool, enabling students to be aware of how and why a solution path was chosen (…).”

This claim has been reiterated by scholars working in research projects that are aimed at realizing learning outcomes in various curricular domains, including mathematics, through advanced learning technologies such as simulations, serious games, hypermedia, computer-supported collaborative learning environments, and virtual reality. According to these researchers, several design characteristics of these advanced learning technologies (such as learners’ access to nonlinear information, to multiple representations, or to input or ideas from other learners) may play a critical role in the development of their metacognitive or self-regulatory skills and, consequently, lead to improved learning outcomes in the curricular domain at stake [30, 33–35].

However, the above-mentioned design features of these advanced learning technologies also require learners to actively, deliberately, and critically consider and monitor the logical relations between various pieces of information, different kinds of representation, or the varied input from co-learners, in relation to the progression in their own learning process. Clearly, all these processes can be characterized as metacognitive or self-regulatory [6]. So, learners’ ability to monitor, control, and reflect on their learning in these advanced technological learning environments will also have a significant impact on their learning outcomes [16, 33]. Indeed, successfully navigating nonlinear information, integrating information from various representations of the same concept, or integrating input from a peer learner requires learners to actively regulate their own learning, and learners’ inability to do so will undermine their learning success with these technologies. In response to this challenge, researchers have developed various kinds of scaffolds or prompts designed to stimulate, guide, and support learners’ metacognition or self-regulation while working with these advanced learning technologies [30, 33]. While initially these metacognitive scaffolds or prompts embedded in available ICT environments were typically general and static in nature, researchers are increasingly striving for more individually tailored pathways involving adaptive scaffolding, prompts, assessments, and navigation, by making use of techniques of artificial intelligence, learning analytics, etc. [33, 36].

Learners’ inability to monitor, control, and reflect on their learning with advanced learning technologies is not the only reason why the potential of advanced technologies in (mathematics) teaching and learning may not be fulfilled. Other reasons for that failure given in the literature are (1) the very nature of ICT, which calls for “trial-and-error” and does not encourage learners to reflect on their problem-solving processes, and (2) the cognitive overload in learners that these ICT-enhanced learning environments might be induced by the multitude of information and extra tools they offer [16].

1.3. Aims and Scope of This Review

This article reviews research on enriching ICT-enhanced environments by metacognitive scaffolds or prompts, or the reverse, that is, attempts to increase the power of metacognitive pedagogies by embedding them in ICT-based learning environments in the domain of K-12 mathematics education. So, our review covers research on how to embed metacognitive pedagogies in available ICT environments to increase the quality and efficiency of mathematics learning, as well as research on how to apply ICT to increase the quality and efficiency of metacognitively oriented mathematics learning environments.

With this systematic review, we hope to answer the following questions concerning the available research on mathematics learning in metacognitively oriented ICT-based learning environments:

(i)What is the educational level of the learners (ranging from kindergarten to secondary education)?(ii)What are the characteristics of the learners (e.g., regular learners, low- or high-achieving learners, or learners with special needs)?(iii)What kind of ICT environment is being used (e.g., computer-supported practice, educational e-books, intelligent tutoring systems, serious games, multimedia, simulations, hypermedia, computer-supported collaborative learning environments, and virtual environments)?(iv)What is the nature of the mathematical subdomain being addressed (e.g., early number sense, arithmetic, geometry, algebra, graphing, and word problem-solving)?(v)What research methodology is being used (e.g., observational study, case study, correlational study, (quasi-)experimental intervention study, and design research)? and in case of intervention studies, What is the nature of its conditions?(vi)What kind of metacognitive training, guidance, or support is provided (e.g., utilizing software features that help the learner to navigate within a math learning environment, providing assistance in solving problems or processing information by offering metacognitive scaffolds, and increasing the awareness of the problem-solving or learning process with mirroring tools [13])? And how is the metacognitive support delivered? Is it embedded in the ICT environment itself, provided outside by the teacher or other instructional means (e.g., cue cards), or both?(vii)What are the targeted learning outcomes of the learning environment (i.e., cognitive knowledge and skills in a given content domain, metacognitive knowledge and self-regulatory skills, improved performance in the specific mathematical subdomain being addressed, or affective or motivational learning outcomes)?(viii)What are the main findings?

2. Method

We conducted a systematic review of the research literature regarding technology-mediated K-12 mathematical learning environments implementing a metacognitive pedagogy, making use of the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses [37]) guidelines. These guidelines consist of a checklist and a flow diagram and help improve the reporting of this review. A summary of the search and selection process, which consists of four consecutive phases, is presented in Figure 1 and is based on the PRISMA statement.

Figure 1: Overview of the search procedure.

First, a systematic search was done through the consultation of the databases ERIC and Web of Science. The databases were browsed for peer-reviewed English articles, based on four combinations—accounting for synonyms in search terms—that is, “metacognition AND mathematics AND online learning” (n = 33), “metacognition AND mathematics AND computer-based learning” (n = 12), “self-regulation AND mathematics AND online learning” (n = 36), and “self-regulation AND mathematics AND computer-based learning” (n = 28). The total number of articles found by these search terms was 109 (78 in ERIC and 31 in Web of Science).

Second, the literature findings were narrowed down by excluding duplicates and conference proceedings, resulting in 46 articles.

Third, the dataset was further reduced—by reading the abstracts, and if necessary the full texts—based on four inclusion criteria that were implied by the focus of this article. More particularly, the study has to involve (1) an online learning environment, (2) K-12 learners in regular education, (3) relation to math content, and (4) implementation of a metacognitive pedagogy. After application of these four criteria, 10 articles remained in the dataset.

Fourth, backward and forward reference search was applied to these 10 articles. In case of backward reference searching, reference lists of these 10 articles were systematically screened to identify relevant studies for the scope of this article. Forward reference search was applied to identify more recent articles that cited one of the initial 10 articles in our dataset. This backward and forward reference search resulted in another 12 articles that also met the above-mentioned inclusion criteria. This means that a total number of 22 articles formed the basis of this review study; these articles are marked with an asterisk in the reference list.

To answer the above-mentioned research questions, each article was summarized in terms of the following aspects: (a) the educational level of the learners, (b) other characteristics of the learners, (c) the kind of ICT-based environment being used, (d) the mathematical subdomain being addressed, (e) the metacognitive pedagogy being included, (f) the overall research method and the nature of its conditions, (g) the dependent variables being targeted, and (h) the obtained results.

3. Overview of the Available Studies

An overview of the retrieved studies on metacognitively oriented ICT-based learning in K-12 mathematics education is provided in Table 1. In this overview, we use the learners’ educational levels—namely, kindergarten, elementary education, or secondary education—as the main ordering principle. In addition, each article was summarized in terms of the following aspects: other characteristics of the learners, the kind of ICT-based environment being used, the mathematical subdomain being addressed, the metacognitive pedagogy being included, the overall research method and its conditions, the learning outcomes being targeted, and the main result(s).

Table 1: Overview of the retrieved articles on a metacognitively oriented ICT-based learning environment for K-12 mathematics.

No.
Article
School level
Participants
ICT-based environment
Math subdomain
Research method and conditions
Metacognitive training/guidance/support
Targeted learning outcomes
Main findings

1
Baeten et al. [12]
K
167 last-year kindergarten children
Computer-supported practice
Early number sense
Intervention study with 5 computer-based practice conditions: (1) counting; (2) comparison; (3) counting + comparison; (4) counting + comparison with MC support; (5) gaming (control condition)
Training in monitoring and self-regulation of simple memorization skills (computer-based)
Cognitive: early math skills
Metacognitive condition was most effective, even after controlling for intelligence. In all computer-based conditions, lower performing children made higher learning gains, possibly because of ceiling effects.

2
Elliott and Hall [38]
K
54 at-risk children
Computer-supported practice (number farm)
Early number sense
Intervention study with 3 conditions: (1) computer-based practice + MC support; (2) computer-based practice; (3) no computer-based practice (control condition)
Various instructional strategies (i.e., goal identification, active monitoring, modeling, scaffolding, questioning, reflecting, etc.) for the MC skills to be learned (teacher-based)
Cognitive: math achievement
Computer-based practice + MC support group outperformed other groups on math achievement.

3
Shamir and Lifshitz [39]
K
77 at-risk children
Educational e-book
Early number sense
Intervention study with 2 conditions: (1) e-book with MC guidance; (2) e-book without MC guidance
Presentation of MC questions on each screen page, before (questions with respect to planning) and after (questions with respect to monitoring understanding) each activity (computer-based)
Cognitive: emergent literacy (rhyming) and emergent mathematics skills (addition and ordinal numbers)
Both conditions improved on emergent literacy and math skills.
MC condition made a larger improvement for literacy, but not for math (probably because of cognitive overload).

4
Chen and Chiu [40]
E
80 fifth graders (from 4 classes)
Computer-supported collaborative learning
Geometry
Intervention study with 2 conditions:
(1) CSCL environment with MC scripts; (2) CSCL environment without MC scripts
Provision of a fixed sequence of prompts for individual MC processes and for structuring pupils’ interactions about MC (based on the “think-pair-share method”) (computer-based)
Cognitive: lower levels of math literacy (cognitive level of understanding and applying) and higher levels of math literacy (cognitive level of analyzing and evaluation)
Metacognitive: MSLQ questionnaire: scales of controlling and planning
MC scripted group outperformed MC nonscripted group on high-level math literacy, but no difference on low-level math literacy.
MC scripted group outperformed MC nonscripted group on reported controlling skills, but no difference on reported planning skills.

5
de Kock and Harskamp [41]
E
390 fifth graders (from 18 classes)
Computer-supported practice
Word problem-solving
Intervention study, in the real-school setting, with 2 conditions: (1) computer-based practice with MC support (experimental); (2) practice without computer and without MC support (control)
Provision of a sequence of general MC prompts/questions (i.e., read and analyze, explore, plan, verify, and reflect) and content-specific MC prompts that pupils are free to consult (computer-based)
Cognitive: word problem-solving
Metacognitive: self-monitoring behavior
Experimental group outperformed control group on solving word problems and on self-monitoring behavior.

6
Dresel and Haugwitz [42]
E
151 sixth graders
Computer-supported practice (MatheWarp)
Unspecified
Intervention study with 3 conditions: (1) MatheWarp with attributional feedback;
(2) MatheWarp with attributional feedback + MC training; (3) MatheWarp without attributional feedback and without MC training
Provision of worksheets made up of questions addressing MC processes (e.g., planning and self-evaluation) (teacher-based)
Cognitive: achieved math knowledge
Metacognitive: questionnaire on goal setting and planning before learning activities and on monitoring processes during learning activities
Motivation: questionnaire on attribution (internal success + variable failure, self-concept, and experienced helplessness)
Conditions 1 and 2 outperformed condition 3 on motivation questionnaire.
Condition 2 outperformed conditions 1 and 3 on math achievement and on reported MC strategies.
Both were immediate and delayed effects.

7
Jacobse and Harskamp [43]
E
49 fifth graders (from 2 classes)
Computer-supported practice
Word problem-solving
Intervention study with 2 conditions: (1) computer-supported practice with MC support (experimental); (2) traditional classroom teaching (control)
Provision of a sequence of general MC prompts/questions (i.e., read and analyze, explore, plan, verify, and reflect) and content-specific MC prompts that pupils are free to consult (computer-based)
Cognitive: word problem-solving
Metacognitive: use of metacognitive skills while solving word problems (think-aloud protocols of some experimental pupils + qualitative analysis while using the program)
Condition 1 outperformed condition 2 on word problem-solving and on MC measures.
Pupils who asked more for MC prompts benefitted more from the intervention.
Condition 1 improved in metacognitive skillfulness from pre- to posttest.
Using the MC hints during the intervention resulted in better word problem-solving behavior.

8
Lazakidou and Retalis [44]
E
24 fourth graders
Computer-supported collaborative learning
Word problem-solving
Case study
Sequence of learning activities for augmenting MC skills: (1) observing an expert; (2) collaboration in small groups; (3) individually solving problems with coaching; (4) individually solving problems without coaching (computer- and teacher-based)
Cognitive: problem-solving performance
Metacognitive: use of metacognitive strategies
There were a significant increase from pre- to posttest in problem-solving performance, a
decrease in duration of solving a problem over time, and a significant increase in metacognitive strategies as a result of the intervention.

9
Okita [45]
E
Study 1: 62 children (9–11 yrs) to 40 children
Study 2: 22 children
Computer-supported practice
Arithmetic
2 intervention studies with 2 computer-supported practice conditions each: (1) self-training: pupils solved all problems on their own; (2) self-other training: pupils worked with computer character
In the self-other condition, pupils had to monitor a computer character displaying correct or erroneous reasoning processes (computer-based)
Cognitive: calculation time and accuracy
Metacognitive: through log files of intervention, looking at self-monitoring behavior and correcting own mistakes
There were longer calculation times but higher accuracies in the self-other group.
The self-other group also outperformed the self-training group on monitoring and self-correcting their own behavior.
Extended training (in study 2) provided a longer-lasting effect.

10
Teong [46]
E
40 low achievers (11-12 yrs)
Computer-supported practice (WordMath)
Word problem-solving
(a) Intervention study with 2 computer-supported practice conditions: (1) practice without MC guidance; (2) practice with MC guidance (b) Case study
Provision of a card with MC cues and questions related to different phases of the word problem-solving process (i.e., careful reading, recalling possible strategies, implementing possible strategies, monitoring, and evaluation) (teacher-based)
Cognitive: word problem-solving (experimental study)
Metacognitive: knowing when and how to use MC strategies (case study: qualitative problem-solving behavior of some students of the two conditions)
Practice with MC guidance condition outperformed condition without MC guidance on cognitive and metacognitive outcomes.

11
Kapa [47]
S
441 eighth graders (high and low achievers)
Computer-supported practice
Word problem-solving
Intervention study with 4 computer-supported practice conditions: MC support (1) during and after problem-solving process, (2) only during problem-solving process, and (3) only after problem-solving process; (4) no MC support
Provision of MC questions during different phases of the word problem-solving process (e.g., What are you asked to find?) and/or MC questions after problem-solving (e.g., Is there any other way to solve the problem you have already solved?) (computer-based)
Cognitive: performance on word problems
The two conditions providing MC support during the solution process were more effective than the learning environments providing MC support after the solution process or no MC support. Low-achieving students were more influenced by MC support than the high-achieving ones.

12
Kapa [48]
S
231 eighth graders (from 14 classes)
Computer-supported practice
Word problem-solving
See Kapa [47]
See Kapa [47]
Cognitive: performance on near-transfer tasks (problems similar to the intervention) and far-transfer tasks (unfamiliar open-ended problems)
Metacognitive: qualitative analyses of the protocols to look for MC functions
MC support was effective in realizing near and far transfer.
Intervention was effective for both high- and low-achieving students.
There was almost no improvement of metacognitive functions throughout different conditions of the intervention.

13
Kramarski and Dudai [49]
S
100 ninth graders (from 3 classes)
Computer-supported collaborative learning environment
Word problem-solving and math inquiry
Intervention study with 3 conditions: (1) online learning + group feedback guidance (GFG); (2) online learning + self-explanation guidance (SEG); (3) control group (same learning tasks, but not online and no MC guidance)
GFG and SEG: instruction and practice in 4 self-addressed MC questions (related to comprehension, connections, strategies, and reflection) based on the IMPROVE instructional approach
GFG: students providing elaborated feedback to each during online forum discussions
SEG: students providing an explanation for their own thinking when working online in forum discussions (computer- and teacher-based)
Cognitive: students’ mathematical inquiry ability (problem-solving, explanations, mathematical feedback in online forum discussions, and transfer ability)
Metacognitive: SRL questionnaire and metacognitive feedback in online forum discussions
GFG students outperformed SEG students in most cognitive and metacognitive measures and control students in all measures.
SEG students outperformed control students in mathematical problem-solving but not on mathematical transfer ability or SRL.

14
Kramarski and Friedman [50]
S
90 eighth graders
Multimedia environment
Graphing
Intervention study with 3 multimedia conditions: (1) unsolicited, i.e., with automatic MC prompts (system control); (2) solicited, i.e., free access to MC prompts (learner control); (3) control condition: multimedia environment without any MC prompts
Unsolicited or solicited: instruction and practice in 4 self-addressed MC questions (related to comprehension, connections, strategies, and reflection) based on the IMPROVE instructional approach (computer-based)
Cognitive: immediate problem-solving comprehension (think-aloud protocol); delayed transfer problem-solving test
Metacognitive: SRL questionnaire: student cognition, metacognition, and problem-solving motivation; peer discourse analysis and help-seeking behavior (metacognitive behavior)
Unsolicited condition was better than solicited condition and solicited condition was better than control condition on immediate comprehension tasks, delayed transfer, and level of metacognitive discourse, particularly in the planning phase. Unsolicited condition was worse than solicited condition and solicited condition was worse than control condition on perceived mental effort. Unsolicited group consulted more aids.

15
Kramarski and Gutman [51]
S
65 ninth graders (from 2 classes)
Computer-supported practice
Algebra
Intervention study with 2 computer-based practice conditions: (1) practice without MC support; (2) practice with MC support
Instruction and practice in 4 self-addressed MC questions (related to comprehension, connections, strategies, and reflection) based on the IMPROVE instructional approach (teacher-based)
Reflection on “What is a good math explanation?”) (teacher-based)
MC feedback (through questions during and after the solution process) (computer-based)
Cognitive: math test on linear functions: (1) standard, procedural tasks; (2) more complex transfer tasks; (3) providing math explanations
Metacognitive: SRL questionnaire: (1) use of problem-solving strategies; (2) use of self-monitoring strategies
Both conditions improved in cognitive measures after the intervention. MC condition outperformed non-MC condition on standard procedural tasks, transfer tasks, and providing math explanations.
Both conditions improved on reported SRL strategies after the intervention. MC condition outperformed non-MC condition on use of self-monitoring strategies, but no difference on problem-solving strategies.

16
Kramarski and Hirsch [52]
S
43 students (13 yrs; from 2 classes)
Computer-supported practice
Algebra
Intervention study with 2 computer-based practice conditions: (1) with MC support; (2) without MC support
Instruction and practice in 4 self-addressed MC questions (related to comprehension, connections, strategies, and reflection) based on the IMPROVE instructional approach (computer- and teacher-based)
Cognitive: algebraic thinking
Metacognitive: qualitative analyses of self-regulation behavior during the paired problem-solving sessions
Condition with MC students outperformed condition without MC on algebraic thinking and effective regulation of their learning.

17
Kramarski and Mizrachi [53]
S
43 seventh graders (from 2 classes)
Computer-supported collaborative learning environment
Word problem-solving
Intervention study with 2 CSCL conditions: (1) with MC guidance; (2) without MC guidance
Instruction and practice in 4 self-addressed MC questions (related to comprehension, connections, strategies, and reflection) based on the IMPROVE instructional approach (computer- and teacher-based)
Cognitive + metacognitive: students’ problem-solving of real-life tasks: understanding the task; using math strategies; processing information; math reasoning
Metacognitive: SRL questionnaire about learning online
Task performance in the online environment was significantly higher in MC guidance condition than in condition without MC guidance on understanding the task, using math strategies, processing information, and math reasoning.
Both conditions improved on the SRL questionnaire.
MC guidance condition showed more reported SRL after intervention than non-MC guidance condition.

18
Kramarski and Mizrachi [54]
S
87 seventh graders (from 4 classes)
Computer-supported collaborative learning environment
Word problem-solving
Intervention study with 4 conditions: (1) computer-based environment with MC guidance; (2) computer-based environment without MC guidance; (3) face-to-face environment with MC guidance; (4) face-to-face environment without MC guidance
See Kramarski and Mizrachi [53]
Cognitive: standard test on algebraic knowledge
Cognitive + metacognitive: see Kramarski and Mizrachi [53]
Metacognitive: SRL questionnaire: (1) general part to all conditions: use of problem-solving strategies; (2) specific part only to online conditions: questionnaire about online learning
Online + MC > F2F + MC > online = F2F for all outcome measures (cognitive and metacognitive). Online > F2F for part of the algebraic knowledge test and solving real-life problems, but no difference for SRL.

19
Maras et al. [55]
S
40 students with autism (13 yrs); 95 typically developing students (13 yrs)
Computer-supported practice (Math Challenge)
Unspecified
Intervention study with 2 computer-based practice conditions: (1) with MC feedback; (2) without MC feedback
Engage in MC monitoring activities before and after solving each mathematical task (e.g., intentions and judgements of accuracy) and negotiate with the system on the level of difficulty (computer-based)
Feedback condition with MC monitoring support regarding accuracy of answers, goal reminders, and strategy support (computer-based)
Cognitive: performance on math test
Metacognitive: questionnaires on judgement of confidence, intention monitoring, and metacognition
No infect of additional feedback on judgement of confidence and intention monitoring. Feedback condition outperformed nonfeedback condition on math test performance for both types of students.

20
Roll et al. [56]
S
Study 1: Aleven et al. [57]: pilot study
Study 2: Roll et al. [58]: 60 students
Study 3: Roll et al. [59]: 80 students
Intelligent tutoring system
Algebra and geometry
Intervention study with 2 ITS-based conditions: (1) ITS with MC help tutor; (2) conventional ITS without MC help tutor
Provision of intelligent and adaptive help with respect to students’ MC behavior while working through the math problems. The nature of the MC help is adapted to the level of MC development (computer-based)
Cognitive: math performance
Metacognitive: help-seeking behavior in the learning environment and help-seeking skills in the posttest
Help tutor improved students’ help-seeking behavior when working with the cognitive tutor.
Improved help-seeking behavior did not transfer to a paper-and-pencil evaluation of students’ help-seeking strategies.
There is no improved math learning due to the help tutor.

21
Roll et al. [60]
S
Study 1: 58 tenth/eleventh graders
Study 2: 67 tenth and eleventh graders
Intelligent tutoring system (Geometry Cognitive Tutor)
Geometry
Intervention study with 2 ITS-based conditions: (1) Geometry Tutor with MC help tutor; (2) Geometry Tutor without MC help tutor
Study 1: see Roll et al. [56]
Study 2: idem as study 1 + additional MC help-seeking instruction and support for self-assessment (computer-based)
See Roll et al. [56]
Study 1 and study 2: help tutor improved students’ help-seeking behavior while learning with the tutor.
Study 2: improved help-seeking skills also transferred to delayed learning of new domain-level content while the help-seeking support was no longer in effect.

22
Schwonke et al. [61]
S
60 eighth graders (high and low achievers)
Intelligent tutoring system (Geometry Cognitive Tutor)
Geometry
Intervention study with 2 ITS-based conditions: (1) with MC support; (2) without MC support
Provision of MC cue cards encouraging and helping students to use instructional resources in the learning environment (i.e., 3 hints on “How do I solve the problem?” and 3 hints on “What do I do when I get stuck?”) (teacher-based)
Cognitive: effects on procedural and conceptual knowledge; time needed for learning in the environment (learning efficiency)
Metacognitive: learners’ interactions with the learning environment, i.e., use of the hints (based on log files and gaze data)
Metacognitive condition outperformed only the control condition for low achievers in view of conceptual knowledge. No differences for high achievers or for procedural knowledge. Metacognitive condition improved learning efficiency by reducing the total learning time.
Metacognitive condition outperformed control condition in view of MC skills.

Note. In the column “School level,” the abbreviations K, E, and S, respectively, mean kindergarten, elementary, and secondary school levels. In the column “Metacognitive training/guidance/support,” we specify, between brackets, whether the MC training/guidance/support was embedded in the ICT environment itself (computer-based) or not (teacher-based) or divided between both (computer- and teacher-based). Other abbreviations: MC = metacognition/metacognitive; SRL = self-regulation/self-regulatory.

3.1. Kindergarten

We retrieved three studies on metacognitively oriented ICT-based learning environments for mathematics education in kindergarten [12, 38, 39]. The main features of these three studies are, respectively, summarized in nos. 1, 2, and 3 in Table 1.

Children in these studies had the mean age between 4 and 6 years. Two of the three studies (2 and 3) involved children at risk for (mathematical) learning disabilities.

In two studies, the ICT-based environment involved drill-and-practice (1 and 2), while the other one was built around an educational e-book (3).

In all three cases (1, 2, and 3), the mathematical subdomain was early numerical skills such as counting, numerical magnitude comparison, ordinal number relations, and simple addition.

All three studies were intervention studies with (quasi-)experimental designs comparing an ICT-based environment with and without a metacognitive pedagogy, and in two studies (1 and 2), a non-ICT control condition was added.

The metacognitive training was embedded twice in the ICT environment (1 and 3) and once provided by the teacher as a supplement to the existing ICT environment (2). The metacognitive strategies taught to the children were basic forms of planning, monitoring, and reflection, which were provided by means of instructional techniques such as providing cues and questions, demonstrations, and explanations.

As far as targeted learning outcomes are concerned, all three studies assessed the impact on the early mathematics skills that were addressed in the learning environment, and in one study (3), transfer to the emergent literacy skill of rhyming was assessed too. None of the studies involved a metacognitive measure.

In all three studies, the extra metacognitive pedagogy resulted in a significant positive learning effect compared to the other conditions. Remarkably, in the study of Shamir and Lifshitz [39] (i.e., study 3 in Table 1), this positive learning effect only holds for the literacy and not for the mathematics measure. Furthermore, taking into account the results of Shamir and Lifshitz [39], it seems important that the metacognitive support does not overload these young children’s working memory, especially when these children are at risk for learning disabilities.

In sum, even though only few studies have explored the potential of metacognitively oriented ICT-supported mathematics education to preschoolers, the available evidence suggests that it is possible to pay instructional attention to basic metacognitive skills in ICT-based environments from a very early age on, before the start of elementary school. Not surprisingly, at-risk children seem to be seen as a particularly relevant group for this kind of learning environments, and in line with the educational technology for early mathematical education in general, the privileged application domain is drill-and-practice software for basic numerical skills, although one study involved e-book reading. While these kindergarten studies suggest that this instructional attention to metacognitive pedagogy in ICT-based environments may lead to better learning outcomes in emergent mathematical and/or literacy skills, absence of metacognitive pre- and posttest measures and/or process data about the development of children’s metacognitive behavior during the intervention forces to be cautious by making causal claims about the role of metacognition in children’s improved early mathematical and/or literacy skills. Furthermore, the results indicate that these metacognitive pedagogies may either be provided by the ICT itself or be supplemented by the teacher. Finally, reflection on the disappointing results for early mathematics from one study (3) led to the important warning that the added metacognitive support runs the risk of overloading young children’s working memory, especially when these children are at risk for learning difficulties.

3.2. Elementary School

We retrieved seven studies on metacognitively oriented ICT-based learning environments for mathematics education in elementary school [40–46]. Their main features are, respectively, summarized in nos. 4–10 in Table 1.

These seven studies involving elementary school children showed a strong tendency to focus on regular children from the upper grades of elementary school. Remarkably, none of the studies involved children in the age group between 6 and 9, and in only one study (10), participants were low achievers in mathematics.

The studies applied a variety of ICT uses. In five studies (5, 6, 7, 9, and 10), the ICT-based learning environment essentially involved individual tutoring in and practicing of the targeted content and skills, and the two remaining ones (4 and 8) were essentially about computer-supported collaborative learning.

As far as the mathematical content is concerned, more than half of the studies (5, 7, 8, and 10) were specifically about mathematical word problem-solving, one about arithmetic computation (9), one about geometry (4), and one covered the whole mathematical curriculum (6).

As for the kindergarten level, the majority of the studies (4, 5, 6, 7, 9, and 10) were (quasi-)experimental intervention studies comparing various experimental and control groups. One of these studies (10) also had a case-study component (10b), and one other study (8) was an independent case study. As far as the experimental studies are concerned, four of them (4, 6, 9, and 10a) involved a comparison between an ICT-based learning environment with and without a metacognitive pedagogy, whereas the two other intervention studies (5 and 7) compared an ICT-based version with a metacognitive pedagogy with a “business as usual” mathematics education control condition. Interestingly, one pair of studies (5 and 7) consisted of an evaluation of the effectiveness of a metacognitively oriented computer-based word problem-solving training under rather strictly controlled experimental conditions and realized by the researchers themselves (7), followed by an examination of its effectiveness in a more naturalistic setting (5).

Most of the time (4, 5, 7, and 9), the metacognitive pedagogy was implemented in the ICT-based learning environment, whereas in two other studies (6 and 10), it was the teacher who was responsible for the extra metacognitive instruction and support through the organization of the lessons and/or additional curricular materials (e.g., worksheets), and in one other study, the metacognitive pedagogy was jointly realized by the computer and the teacher. The nature of the metacognitive pedagogy differed according to the mathematical content. For instance, the studies focusing on word problem-solving consisted of a number of general and/or specific metacognitive hints or questions—provided by the computer program or the teacher—that helped the learner to approach and solve each word problem in a thoughtful and systematic way. For instance, in Teong’s study [46] (no. 10 in Table 1), within the computer program were various types of general and specific metacognitive hints that students could request. The general hints were displayed as a staircase containing the following steps (and accompanying hints): read and analyze (“I read carefully”), explore (“I know the type of problem”), plan (“I make a plan”), verify (“I check my answer”), and reflect (“What did I learn?”). The specific hints were about looking within the word problem for what is asked, what numbers are given, and how the problem can be represented.

While in the three kindergarten studies, the assessment was restricted to cognitive learning outcomes related to the content and skills addressed in the training, in all studies with elementary school children, the assessment also involved measurements of a metacognitive (4, 5, 6, 7, 8, 9, and 10) and motivational (6) nature. These metacognitive measures involved (a) general questionnaires about metacognition, such as the “Motivated Strategies for Learning Questionnaire” (MSLQ) [62], being administered at (pretest and) posttest (4 and 6), (b) specifically designed task-specific instruments that are aimed at revealing the use of the intended metacognitive knowledge and/or skills while performing the task during (pretest and) posttest (5, 6, 7, 8, 9, and 10a), and/or (c) log-file analyses or more qualitative evaluations of learners’ metacognitive behavior while working on the mathematical tasks in different experimental conditions (7 and 10b).

In all studies, positive results were reported concerning the superiority of the condition that combined ICT and metacognitive pedagogy, even though the design of several of these studies did not allow straightforward conclusions about the added value of this combination compared to other conditions involving only ICT, only metacognitive pedagogy, or neither of the two. Interestingly, in all cases wherein both cognitive and metacognitive measures were involved, learners in the ICT-based environment with metacognitive guidance demonstrated positive learning effects both for the cognitive outcomes related to the mathematical content and skills addressed in the training and for metacognitive outcomes (4, 5, 6, 7, 8, 9, and 10). Some additional findings obtained in studies that made extra attempts to understand the underlying mechanisms were that learners who made more use of the metacognitive guidance benefitted more from the intervention (7), that extended training provided longer-lasting learning effects (9), and that learning effects were not restricted to immediate effects but lasted at least until five months after the end of the intervention (6).

In sum, various scholars have tried to enhance children’s cognitive and metacognitive development in the context of elementary school mathematics learning, particularly in the context of word problem-solving. In the studies reviewed above, all of which involved regularly developing children from the upper grades of elementary school, the metacognitive pedagogy was either embedded within the ICT environment or provided by the teacher or extracurriculum materials. Taken as a whole, these studies provide ample evidence for the positive role that a metacognitively oriented training combined with an ICT-based learning environment can play in enhancing (upper) elementary school children’s mathematical performance, their metacognitive skills, and—at least in one study—also their motivation. However, flaws in the design of the studies and/or in the analyses of the learning processes during the intervention do not always allow a clear and convincing picture of the contribution of the ICT and the metacognitive pedagogy aspects to the success of the intervention.

3.3. Secondary School

Also with samples of secondary school students, researchers have examined the instructional interventions designed to influence and exploit students’ metacognition during mathematical reasoning and problem-solving in an ICT environment or to make their metacognitively oriented mathematical learning environment more powerful by means of ICT [47–56, 59–61]. The main characteristics of these 12 studies are summarized in nos. 11 to 22 in Table 1.

Except for two studies (20 and 21), which (also) involved students from the upper grades of secondary school, all studies with secondary school students took place in the lower and middle grades of this school level. Moreover, all studies involved regular learners, except for one study with low-achieving students (11) and one with students with autism (19).

Compared to the two lower educational levels, the studies with secondary school students involve a somewhat broader variety of ICT uses. Most of the studies used traditional ICT for individual tutoring in and/or practicing of certain mathematics knowledge and skills (11, 12, 15, 16, and 19), while some others (20, 21, and 22) used intelligent tutoring systems. In another substantive set of studies (13, 14 17, and 18), students were put in the complex (multimedia) learning environment wherein they could explore, in pairs or small groups, certain mathematical topics and/or solve mathematical problems by making use of various search, representation, and communication tools.

The studies covered various parts of the secondary school curriculum. As for secondary school mathematics, (algebra) word problem-solving was the most frequent topic (11, 12, 13, 17, and 18). Other topics were geometry (20, 21, and 22), algebra (15, 16, and 20), and graphing (14).

All studies were (quasi-)experimental studies comparing the efficacy of two or more learning environments with or without certain ICT-based instructional elements and/or metacognitive pedagogies. Several studies had basic experimental designs, such as the comparison of an ICT-based instructional program with or without the provision of extra metacognitive cue cards (15, 16, 15, 17, 19, 20, 21, and 22), whereas others had more complex designs involving three or even four conditions combining the absence or presence of ICT-based and metacognitively oriented instructional elements (11, 12, 13, 14, and 18).

The majority of the studies involving secondary school students had metacognitive hints or guidance tools that were actually embedded in the ICT software (11, 12, 14, 15, 19, 20, and 21). In only one study, the metacognitive pedagogy was provided by the teacher (22), and in the remaining studies (13, 16, 17, and 18), the computer and teacher were jointly responsible for the metacognitive training component. The concrete nature of the (computer-based) metacognitive guidance and support differed according to the type of ICT environment and its mathematical content, but essentially, it consisted of providing cues and questions before, during, and/or after task execution. In the studies of Kramarski and associates, the nature of the metacognitive guidance and support was based on the IMPROVE approach towards metacognitively oriented instruction explained in Introduction. For instance, in their study about e-learning, Kramarski and Gutman [51] (see no. 15 in Table 1) compared the impact of e-learning supported with self-metacognitive questioning based on the IMPROVE approach (EL + IMP) with e-learning without such an explicit support of self-regulation (EL). Students in both environments practiced 10 interactive mathematics problems presented electronically to them in Excel. Students were supposed to interact in pairs about the problems and their solutions before sending their common solution to the teacher via e-mail. Students from both groups were encouraged to work with e-learning, but only the EL + IMP group received the extra metacognitive supports (based on IMPROVE) that appeared in pop-up screens and provided the students with differential metacognitive cues for solving the problems. While the research program of Kramarski et al. was essentially aimed at evaluating various forms of e-learning supported by various kinds of metacognitive instructions built in into the ICT environment or provided from outside, Roll et al. ([58], see nos. 20 and 21 in Table 1) investigated the additive effects of implementing metacognitive scaffolding on an existing intelligent tutoring system for teaching algebra and geometry to secondary school students. More specifically, they designed and implemented a metacognitive tutor for teaching help-seeking students starting from the available algebra or geometry ITS. Based on clear instructional goals with respect to metacognition on the one hand and a good model of the student’s actual metacognitive behavior on the other hand, the auxiliary metacognitive tutor provides help with respect to various aspects of the student’s help-seeking behavior while (s)he is working through the learning materials. Importantly, this adaptive tutor is designed in such a way that it does not merely compensate for the student’s failing metacognitive c.q. help-seeking behavior, but rather it tries to make the student increasingly competent in the intended metacognitive behavior, on top of increasing his/her mathematical knowledge and skills.

As for the investigations with elementary school children, except for one study that only measured the impact on students’ mathematical learning outcomes (11), all studies at the secondary school level involved both cognitive and metacognitive measures (12, 13, 14, 15, 16, 17, 18, 19, 20, 21, and 22), while, except for study no. 14, none included affective/motivational measures as well. The cognitive measures are aimed at assessing content-related learning, which was mostly closely related to the mathematical content being taught but sometimes also involved more distant measures such as specially designed transfer tests or general mathematics achievement tests (e.g., 12, 13, and 15) or assessments aimed at assessing possible changes in students’ cognitive processes and strategies (e.g., 14). The metacognitive measures involved (a) general questionnaires about metacognition or self-regulation being administered at (pretest and) posttest (e.g., 13, 14, 15, 17, 18, and 19), (b) specifically designed task-specific instruments that are aimed at revealing the use of the intended metacognitive knowledge and/or skills while performing the task during (pretest and) posttest (e.g., 20 and 21), and/or (c) log-file analyses or other types of qualitative evaluations of learners’ metacognitive behavior while working individually, in pairs or groupwise on the mathematical tasks in some or all experimental conditions (e.g., 12, 14, 16, and 22).

As far as the results are concerned, generally speaking, the experimental condition wherein the combination of ICT use and metacognitive pedagogy was implemented in the most intensive or most sophisticated way led to better cognitive learning outcomes, at least for some of the cognitive measures, in terms of learning outcomes as revealed either by pre- and posttest comparisons or by evaluation of the quality of the mathematical work during the instructional program (11, 12, 13, 14, 15, 16, 17, 18, 19, 20, and 21b). Moreover, most studies report positive outcomes for at least some of their metacognitive measures (13, 14, 15, 16, 17, 18, 20, 21, and 22). Interestingly, the few studies that performed separate analyses for different levels of mathematical achievement showed that the learning environment was particularly helpful for the low-achieving students (11 and 22).

However, the size and nature of the effects differed depending on the way in which the metacognitive guidance and training were provided. For instance, the studies suggest that group feedback guidance works better than self-explanation guidance (13), that consistently and regularly exposing students to on-screen metacognitive prompts (without learner control) is more efficient than giving students free access to these prompts (14), and that embedding the metacognitive prompts in the online learning environment leads to better learning effects for mathematics than presenting them face-to-face (18). However, not all studies yielded significant positive outcomes for mathematics learning through metacognitive pedagogies. For instance, in studies by Roll et al. ([59, 60], see nos. 20 and 21 in Table 1), improved metacognitive behavior in students working with a metacognitive tutor was not accompanied by enhanced mathematical performance.

4. Conclusion and Discussion

In this article, we have systematically synthesized the research that integrates ICT and metacognitive pedagogies in mathematics learning from preschool until the end of secondary education. This multidimensional research topic has been approached from two different angles. On the one hand, researchers working with new ICT-based environments for mathematics learning became aware of the pitfalls and risks involved with the increased use of ICT for mathematics learning when viewed from a metacognitive perspective. On the other hand, based on the finding that learners of different ages trained in learning to monitor and control their own cognitive processes for mathematics problems and learning mathematics do better than untrained learners, researchers have started to design metacognitive pedagogies for mathematics learning. Initially, these intervention studies were carried out in traditional learning environments that did not make intentional and systematic use of ICT (e.g., [26, 28]). Yet, the very nature of ICT and its widespread use in the educational systems raise the question to what extent such computer-based environments have the potential to enhance the power of metacognitive pedagogies in the domain of mathematics and, in doing so, enhance learners’ mathematics learning. Over the years, various authors have provided arguments in favor of this claim. While some refer to arguments related to the motivational impact of computers and other ICT tools (e.g., [38]), most scholars ground ICT’s potential impact primarily on their capacity to provide metacognitive cues, questions, and other forms of support and to follow and guide learners’ metacognitive thinking and development in a more fine-grained, systematic, adaptive—and therefore more efficient—way. Although it is not always easy to categorize a study into either of these perspectives, it seems clear that, either explicitly or implicitly, the vast majority of studies involved in this review took the former approach, namely, enriching existing ICT mathematics learning environments with metacognitive pedagogies with a view to make the mathematics learning more successful and more efficient.

As a result of these two distinct but complementary developments within research, the past decades have witnessed a productive line of research on the development and use of various ICT-based tools to stimulate and support different aspects of metacognitively guided mathematics problem-solving and learning.

Given the recency of researchers’ interest in young children’s metacognitive development in general [13], it should, first of all, not come as a surprise that most studies were about elementary and secondary school learners rather than kindergartners. For reasons of feasibility, we did not include other educational levels such as higher education and teacher education into our review, but we do know of studies that are situated at these other levels of mathematics education too (see [63] for example of a study with preservice teachers). It would be interesting to elaborate the present systematic review towards these other educational levels. In contrast, the relatively small number of studies focusing on specific subgroups of mathematically weaker learners or learners with special learning needs is notable, particularly since the potential of both ICT and metacognitive pedagogies for these subgroups has been acknowledged widely. More research on these groups of learners is therefore recommendable.

Second, our review yielded great variation in the kind of ICT-based learning environments that have been developed and used in these studies, including computer-supported practice, educational e-books, intelligent tutoring systems, serious games, multimedia, and computer-supported collaborative learning environments, and this variation tends to become larger with increasing educational levels. While the relation with metacognitive pedagogies has been explored and investigated for many kinds of ICT applications in mathematics education, our literature search has not identified studies that have examined the effects of learning mathematics with newer information and communication technologies such as Facebook and Twitter, or virtual and augmented reality. Given the increasing use of these newer ICT applications and technologies, also in (math) educational contexts [64, 65], it would be interesting to explore their potential for learning mathematics in metacognitively oriented ICT-based learning environments too.

Third, our review revealed that the studies that investigated the interrelation between ICT, metacognition, and mathematics have focused mainly on certain mathematical topics, with an emphasis on early numerical skills in kindergarten and arithmetic and algebra word problem-solving in, respectively, elementary and secondary schools. The strong emphasis on word problem-solving in this review is not surprising, given that mathematical word problem-solving has always been conceived as a privileged topic for assessing and stimulating metacognition and self-regulation [13, 66]. Anyhow, taking into account that each subdomain of mathematics (arithmetic, algebra, geometry, calculus, statistics, etc.) involves different problem types, reasoning styles, and solution methods [11], it is recommendable that future research on learning mathematics in metacognitively oriented ICT-based learning environments would cover a broader spectrum of mathematical subdomains. To give just one example, the metacognitive aspects of learning about mathematical argumentation and proof may differ substantially from those that play a role in learning to become an accurate and quick arithmetic calculator or a better solver of authentic mathematical modeling and application tasks.

Fourth, our review also yielded a wide variety of forms of metacognitive support at all educational levels. Overviewing the studies, a common characteristic seems to be the explicit teaching of the metacognitive skills within the content area through one or more powerful instructional techniques such as modeling and articulation of the intended metacognitive skills and the application of various forms of metacognitive scaffolding (including pop-up screens with metacognitive questions, cues, or scripts) or coaching during intensive and deliberate practice, either individually or in small groups. In contrast, only few learning environments make systematic use of the instructional technique of fading, i.e., the gradual removal of the metacognitive support as the learner progresses, and even fewer do so in an adaptive or intelligent way based on an individualized analysis of the level of metacognitive functioning of the learner. Also, explicit, informative, and individualized feedback on the learners’ metacognitive functioning during the mathematical task completion is typically lacking, although the intelligent tutoring system for learning secondary school algebra and geometry learning in [56, 60] is a prominent exception. Taking into account the well-documented research evidence for the importance of adaptive feedback and support in ICT-based learning environments in general [36, 67], more attention to the design and evaluation of advanced forms of adaptivity in metacognitive pedagogies is warranted. Finally, our review revealed that these powerful instructional techniques are either embedded in the ICT itself or provided by the teacher or otherwise. Except for the study in [54], direct comparisons of ICT-based or human-teacher-based delivery of the (same) metacognitive pedagogy are lacking. So, a remaining question for further research is whether it is better to implement the metacognitive pedagogy into the ICT, to leave it in the hands of the teacher, or to look for blended forms of metacognitive support.

Fifth, while our search terms allowed for a wide variety of research methods, including case studies, observational studies, correlational studies, design experiments, and (quasi-)experimental intervention studies, the vast majority of investigations were of the latter type. Many of those intervention studies were characterized by good internal validity, given that they involved both pre- and posttests and an experimental and control group or several experimental groups, allowing for valid conclusions about the contribution of the ICT and/or metacognitive element to the learning outcomes. However, in a significant number of studies, the research design did not allow to derive grounded conclusions as to what elements or aspects of the computer-based environment were exactly responsible for the obtained (positive) effects. Moreover, many studies involved only pre- and posttest measures, without additional attempts to assess the mathematical and metacognitive learning processes during the intervention, which further jeopardizes a deep understanding of the instructional factors and learning mechanisms that caused the obtained learning effects. Echoing Schunk and Greene’s [6] plea for more research aimed at the exploration of “the operation of self-regulation processes as learners employ them and relate moment-to-moment changes in self-regulation to changes in outcome measures” (p. 6), there is a need for studies with methodologies that also allow to capture this dynamic and reciprocal nature of metacognitive thinking and learning, such as think-aloud protocols, looking for observable traces of metacognition, and microanalytic methods examining learners’ behavior and cognitions in real time as they engage in the learning tasks. Similarly, the additional use of unobtrusive physiological measures during math learning in an ICT-based learning environment might provide a more appropriate picture of learners’ metacognitive thinking and learning. Wrist-worn wearables tracking a.o. skin conductance, electrodermal activity, and heart rate might be promising in this regard [68]. As far as external/ecological validity is concerned, many studies involved rather short metacognitively oriented ICT-based interventions on a specific mathematical content, organized and realized by the researchers outside the context of the regular mathematics lessons. This raises the question if such learning environments would also improve learners’ mathematical and/or metacognitive competence in (more) natural class settings. The scarcely available findings (e.g., [41]) suggest that this is the case but probably with a smaller impact. This is not surprising. Indeed, the effective and sustained implementation of innovative learning environments places high demands on the teachers, requires substantial changes in their traditional roles and practices, and pragmatic adaptations to the concrete demands of the local conditions. This is in line with the more general well-documented findings from meta-analyses of the effects of metacognitive studies on elementary and secondary education, which have made clear that the general impact becomes much smaller once teachers instead of the researchers start to work with the program [2]. Given the complexity of metacognitive development and its complex relation to mathematics learning, on the one hand, and given that ICT is being increasingly widely used in education, on the other hand, longitudinal intervention studies covering large parts of the mathematics curriculum and taking place in regular mathematics classes are needed.

Sixth, the reviewed studies also differ in terms of their dependent variables. While, as a direct result of the scope of our systematic review, all studies involved at least some kind of measure of mathematics achievement, there was great variety in the nature of this measure: some studies used a general measure (i.e., a standardized achievement test or an available overall grade score for mathematics), others made use of a specific measure that was particularly tailored to the mathematical content (e.g., arithmetic or algebra word problem-solving, and geometry) being addressed in the learning environment, and still others also included specifically selected transfer measures. Furthermore, in most studies, particularly at the elementary and secondary school levels, these mathematical assessments were combined with metacognitive measures. These measures were either metacognitive questionnaires or tests administered before and after the intervention or observations, self-reports, and log files collected during the intervention. While all authors explicitly or implicitly assume a positive mediating role of metacognitive activity in mathematics learning, quite remarkably, we did not encounter studies that explicitly tested this mediating effect. Given the assumed positive role of metacognition in ICT-supported mathematics learning, it is highly recommended that this mediating hypothesis is tested in an explicit and appropriate way. Finally, studies investigating the effect of ICT or metacognitive pedagogy on motivational-affective outcomes, such as students’ interest or self-esteem in mathematics, were very rare. Given the well-documented longstanding problems with the affective and motivational aspects of mathematics learning, from the elementary school level onwards [69], more research attention to these aspects is needed.

Finally, as far as the research findings are concerned, taken together, the available research evidence indicates that ICT-based mathematics learning environments supported by metacognitive pedagogies enhance both mathematics learning and achievement and metacognitive thinking and learning, more than environments with no ICT and/or metacognitive support. However, this does not imply that all interventions were successful on all dependent variables being assessed and that all effect sizes were large. But given the literature search method being applied in our systematic review, we cannot exclude a publication bias towards successful studies with significant positive results in favor of the condition(s) with ICT and/or metacognitive support.

Conflicts of Interest

The authors declare that there are no conflicts of interest regarding the publication of this paper.

References

J. D. Bransford, A. L. Brown, and R. R. Cocking, How People Learn. Brain, Mind, Experience, and School, National Academia Press, Washington, DC, USA, 2000.

C. Dignath and G. Büttner, “Components of fostering self-regulated learning among students. A meta-analysis on intervention studies at primary and secondary school level,” Metacognition and Learning, vol. 3, no. 3, pp. 231–264, 2008.

J. A. C. Hattie, Visible Learning: A Synthesis of over 800 Meta-Analyses Relating to Achievement, Routledge, New York, NY, USA, 2008.

O. López-Vargas, J. Ibáñez-Ibáñez, and O. Racines-Prada, “Students’ metacognition and cognitive style and their effect on cognitive load and learning achievement,” Educational Technology & Society, vol. 20, no. 3, pp. 145–157, 2017.

A. H. Schoenfeld, “Learning to think mathematically. Problem solving, metacognition and sense-making in mathematics,” in Handbook of Research on Mathematics Teaching and Learning (A Project of the National Council of Teachers of Mathematics), D. A. Grouws, Ed., pp. 334–370, MacMillan, New York, NY, USA, 1992.

D. H. Schunk and J. A. Greene, Handbook of Self-Regulation of Learning and Performance, Routledge, New York, NY, USA, 2018.

Y.-h. Tsai, C.-h. Lin, J.-c. Hong, and K.-h. Tai, “The effects of metacognition on online learning interest and continuance to learn with MOOCs,” Computers & Education, vol. 121, pp. 18–29, 2018.

J. H. Flavell, P. H. Miller, and S. A. Miller, Cognitive Development, Prentice-Hall, Upper Saddle River, NJ, USA, 4th edition, 2002.

A. Brown, “Metacognition, executive control, self-regulation and other more mysterious mechanisms,” in Metacognition, Motivation and Understanding, F. Weinert and R. Kluwe, Eds., pp. 65–116, Erlbaum, Hillsdale, NJ, USA, 1987.

G. Schraw and R. S. Dennison, “Assessing metacognitive awareness,” Contemporary Educational Psychology, vol. 19, no. 4, pp. 460–475, 1994.

E. De Corte, B. Greer, and L. Verschaffel, “Learning and teaching mathematics,” in Handbook of Educational Psychology, D. Berliner and R. Calfee, Eds., pp. 491–549, Macmillan, New York, NY, USA, 1996.

E. Baeten, A. Desoete, and M. Praet, “The relevance and efficacy of metacognition for instructional design in the domain of mathematics,” ZDM International Journal on Mathematics Education, vol. 49, no. 4, pp. 613–623, 2017.

Z. Mevarech, L. Verschaffel, and E. De Corte, “Metacognitive pedagogies in mathematics classrooms: from kindergarten to college and beyond,” in Handbook of Self-Regulation of Learning and Performance, D. H. Schunk and J. A. Greene, Eds., pp. 109–123, Routledge, New York, NY, USA, 2nd edition, 2018.

M. V. J. Veenman, B. H. A. M. Van Hout-Wolters, and P. Afflerbach, “Metacognition and learning: conceptual and methodological considerations,” Metacognition and Learning, vol. 1, no. 1, pp. 3–14, 2006.

D. Whitebread and P. Coltman, “Aspects of pedagogy supporting metacognition and self-regulation in mathematical learning of young children: evidence from an observational study,” Zentralblatt für Didaktik der Mathematik, vol. 42, no. 2, pp. 163–178, 2010.

Z. R. Mevarech and B. Kramarski, Critical Maths in Innovative Societies: The Effects of Metacognitive Pedagogies on Mathematical Reasoning, OECD, Paris, France, 2014.

W. Schneider and C. Artelt, “Metacognition and mathematics education,” Zentralblatt für Didaktik der Mathematik, vol. 42, no. 2, pp. 149–161, 2010.

G. Stillman and Z. Mevarech, “Metacognition research in mathematics education: from hot topic to mature field,” Zentralblatt für Didaktik der Mathematik, vol. 42, no. 2, pp. 145–148, 2010.

M. V. J. Veenman and M. A. Spaans, “Relation between intellectual and metacognitive skills: age and task differences,” Learning and Individual Differences, vol. 15, no. 2, pp. 159–176, 2005.

A. H. Schoenfeld, Mathematics Problem Solving, Academic Press, New York, NY, USA, 1985.

M. Cardelle-Elawar, “Effects of metacognitive instruction on low achievers in mathematics problems,” Teaching and Teacher Education, vol. 11, no. 1, pp. 81–95, 1995.

A. Desoete, H. Roeyers, and A. De Clercq, “Can offline metacognition enhance mathematical problem solving?” Journal of Educational Psychology, vol. 95, no. 1, pp. 188–200, 2003.

D. Lucangeli, C. Cornoldi, and M. Tellarinin, “Metacognition and learning disabilities in mathematics,” in Advances in Learning and Behavioral Disabilities, T. E. Scruggs and M. A. Mastropieri, Eds., pp. 219–285, JAI Press Inc, Greenwich, UK, 1988.

L. Mason and L. Scrivani, “Enhancing students’ mathematical beliefs: an intervention study,” Learning and Instruction, vol. 14, no. 2, pp. 153–176, 2004.

Z. Mevarech and S. Fridkin, “The effects of IMPROVE on mathematical knowledge, mathematical reasoning and meta-cognition,” Metacognition and Learning, vol. 1, no. 1, pp. 85–97, 2006.

Z. R. Mevarech and B. Kramarski, “Improve: a multidimensional method for teaching mathematics in heterogeneous classrooms,” American Educational Research Journal, vol. 34, no. 2, pp. 365–394, 1997.

A. Panouara, A. Demetriou, and A. Gagatsis, “Mathematical modeling, self-representations and self-regulation,” in Proceedings of the Sixth Congress of the European Society for Research in Mathematics Education (CERME 6), V. Durand-Guerrier, S. Soury-Lavergne, and F. Arzarello, Eds., pp. 94–103, Institute National de Recherche Pédagogique, Lyon, France, January 2010.

L. Verschaffel, E. De Corte, S. Lasure, G. Van Vaerenbergh, H. Bogaerts, and E. Ratinckx, “Learning to solve mathematical application problems: a design experiment with fifth graders,” Mathematical Thinking and Learning, vol. 1, no. 3, pp. 195–229, 1999.

S. Abramovich and M. L. Connell, “Using technology in elementary mathematics teacher education: a sociocultural perspective,” ISRN Education, vol. 2014, Article ID 345146, 9 pages, 2014.

R. Azevedo, M. Taub, and N. V. Mudrick, “Understanding and reasoning about real-time cognitive, affective, and metacognitive processes to foster self-regulation with advanced learning technologies,” in Handbook of Self-Regulation of Learning and Performance, D. H. Schunk and J. A. Greene, Eds., pp. 254–270, Routledge, New York, NY, USA, 2018.

R. M. Zbiek, M. K. Heid, G. Blume, and T. P. Dick, “Research on technology in mathematics education: the perspective of constructs,” in Second Handbook of Research on Mathematics Teaching and Learning, F. K. Lester, Ed., pp. 1159–1207, Information Age Publishing, Greenwich, CT, USA, 2007.

R. E. Mayer, “Learning with technology,” in The Nature of Learning: Using Research to Inspire Practice, H. Dumont, D. Istance, and F. Benavides, Eds., pp. 179–195, OECD Publishing, Paris, France, 2010.

D. Moos, “Emerging classroom technology: using self-regulation principles as a guide for effective implementation,” in Handbook of Self-Regulation of Learning and Performance, D. H. Schunk and J. A. Greene, Eds., pp. 243–253, Routledge, New York, NY, USA, 2018.

J. L. Nietfeld, “The role of self-regulated learning in digital games,” in Handbook of Self-Regulation of Learning and Performance, D. H. Schunk and J. A. Greene, Eds., pp. 271–284, Routledge, New York, NY, USA, 2018.

P. Reimann and M. Bannert, “Self-regulation of learning and performance in computer-supported collaborative learning environments,” in Handbook of Self-Regulation of Learning and Performance, D. H. Schunk and J. A. Greene, Eds., pp. 285–303, Routledge, New York, NY, USA, 2018.

A. Krouska, C. Troussas, and M. Virvou, “Computerized adaptive assessment using accumulative learning activities based on revised Bloom’s taxonomy,” in Proceedings of the Joint Conference on Knowledge-Based Software Engineering, pp. 252–258, Springer Nature, Corfu, Greece, August 2018.

D. Moher, A. Liberati, J. Tetzlaff, D. G. Altman, and The PRISMA Group, “Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement,” Annals of Internal Medicine, vol. 151, no. 4, pp. 264–269, 2009.

A. Elliott and N. Hall, “The impact of self-regulatory teaching strategies on “at-risk” preschoolers’ mathematical learning in a computer-mediated environment,” Journal of Computing in Childhood Education, vol. 8, no. 2-3, pp. 187–198, 1997.

A. Shamir and I. Lifshitz, “E-Books for supporting the emergent literacy and emergent math of children at risk for learning disabilities: can metacognitive guidance make a difference?” European Journal of Special Needs Education, vol. 28, no. 1, pp. 33–48, 2013.

C.-H. Chen and C.-H. Chiu, “Collaboration scripts for enhancing metacognitive self-regulation and mathematics literacy,” International Journal of Science and Mathematics Education, vol. 14, no. 2, pp. 263–280, 2016.

W. D. de Kock and E. G. Harskamp, “Can teachers in primary education implement a metacognitive computer programme for word problem solving in their mathematics classes?” Educational Research and Evaluation, vol. 20, no. 3, pp. 231–250, 2014.

M. Dresel and M. Haugwitz, “A computer-based approach to fostering motivation and self-regulated learning,” The Journal of Experimental Education, vol. 77, no. 1, pp. 3–20, 2008.

A. E. Jacobse and E. G. Harskamp, “Student-controlled metacognitive training for solving word problems in primary school mathematics,” Educational Research and Evaluation, vol. 15, no. 5, pp. 447–463, 2009.

G. Lazakidou and S. Retalis, “Using computer supported collaborative learning strategies for helping students acquire self-regulated problem-solving skills in mathematics,” Computers & Education, vol. 54, no. 1, pp. 3–13, 2010.

S. Y. Okita, “Learning from the folly of others: learning to self-correct by monitoring the reasoning of virtual characters in a computer-supported mathematics learning environment,” Computers & Education, vol. 71, pp. 257–278, 2014.

S. K. Teong, “The effect of metacognitive training on mathematical word-problem solving,” Journal of Computer Assisted Learning, vol. 19, no. 1, pp. 46–55, 2003.

E. Kapa, “A metacognitive support during the process of problem solving in a computerized environment,” Educational Studies in Mathematics, vol. 47, no. 3, pp. 317–336, 2001.

E. Kapa, “Transfer from structured to open-ended problem solving in a computerized metacognitive environment,” Learning and Instruction, vol. 17, no. 6, pp. 688–707, 2007.

B. Kramarski and V. Dudai, “Group-metacognitive support for online inquiry in mathematics with differential self-questioning,” Journal of Educational Computing Research, vol. 40, no. 4, pp. 377–404, 2009.

B. Kramarski and S. Friedman, “Solicited versus unsolicited metacognitive prompts for fostering mathematical problem solving using multimedia,” Journal of Educational Computing Research, vol. 50, no. 3, pp. 285–314, 2014.

B. Kramarski and M. Gutman, “How can self-regulated learning be supported in mathematical E-learning environments?” Journal of Computer Assisted Learning, vol. 22, no. 1, pp. 24–33, 2006.

B. Kramarski and C. Hirsch, “Using computer algebra systems in mathematical classrooms,” Journal of Computer Assisted Learning, vol. 19, no. 1, pp. 35–45, 2003.

B. Kramarski and N. Mizrachi, “Online interactions in a mathematical classroom,” Educational Media International, vol. 43, no. 1, pp. 43–50, 2006.

B. Kramarski and N. Mizrachi, “Online discussion and self-regulated learning: effects of instructional methods on mathematical literacy,” The Journal of Educational Research, vol. 99, no. 4, pp. 218–231, 2006.

K. Maras, T. Gamble, and M. Brosnan, “Supporting metacognitive monitoring in mathematics learning for young people with autism spectrum disorder: a classroom-based study,” Autism, vol. 23, no. 1, pp. 60–70, 2019.

I. Roll, V. Aleven, B. M. McLaren, and K. R. Koedinger, “Designing for metacognition—applying cognitive tutor principles to the tutoring of help seeking,” Metacognition and Learning, vol. 2, no. 2-3, pp. 125–140, 2007.

V. Aleven, I. Roll, E. J. Ryu, and K. R. Koedinger, “An architecture to combine metacognitive and cognitive tutoring: pilot testing the Help Tutor,” in Proceedings of the 12th International Conference on Artificial Intelligence and Education, pp. 17–24, IOS, Amsterdam, Netherlands, July 2005.

I. Roll, V. Aleven, B. M. McLaren, E. Ryu, R. S. Baker, and K. R. Koedinger, “The Help Tutor: does metacognitive feedback improve students’ help-seeking actions, skills, and learning?” in Proceedings of the 8th International Conference in Intelligent Tutoring Systems, pp. 360–369, Springer, Berlin, Germany, June 2006.

I. Roll, V. Aleven, B. M. McLaren, and K. R. Koedinger, “Can help seeking be tutored? Searching for the secret sauce of metacognitive tutoring,” in Proceedings of the 12th International Conference on Artificial Intelligence and Education, pp. 203–210, IOS, Amsterdam, Netherlands, March 2007.

I. Roll, V. Aleven, B. M. McLaren, and K. R. Koedinger, “Improving students’ help-seeking skills using metacognitive feedback in an intelligent tutoring system,” Learning and Instruction, vol. 21, no. 2, pp. 267–280, 2011.

R. Schwonke, A. Ertelt, C. Otieno, A. Renkl, V. Aleven, and R. J. C. M. Salden, “Metacognitive support promotes an effective use of instructional resources in intelligent tutoring,” Learning and Instruction, vol. 23, pp. 136–150, 2013.

P. R. Pintrich, D. A. F. Smith, T. García, and W. J. McKeachie, A Manual for the Use of the Motivated Strategies Questionnaire (MSLQ), University of Michigan, National Center for Research to Improve Postsecondary Teaching and Learning, Ann Arbor, MI, USA, 1991.

B. Kramarski and T. Michalsky, “Preparing preservice teachers for self-regulated learning in the context of technological pedagogical content knowledge,” Learning and Instruction, vol. 20, no. 5, pp. 434–447, 2010.

H. Kaufmann and D. Schmalstieg, “Mathematics and geometry education with collaborative augmented reality,” in Proceedings of the ACM SIGGRAPH 2002, Conference Abstracts and Applications, pp. 37–41, ACM, New York, NY, USA, July 2002.

J. Martín-Gutiérrez, C. E. Mora, B. Añorbe-Díaz, and A. González-Marrero, “Virtual technologies trends in education,” EURASIA Journal of Mathematics, Science and Technology Education, vol. 13, no. 1, pp. 469–486, 2017.

L. Verschaffel, F. Depaepe, and W. Van Dooren, “Word problems in mathematics education,” in Encyclopedia of Mathematics Education, S. Lerman, Ed., pp. 641–645, Springer, Berlin, Germany, 2014.

N. Sonwalkar, “Adaptive individualization: the next generation of online education,” in Proceedings of the EdMedia + Innovate Learning, pp. 3056–3063, Association for the Advancement of Computing in Education (AACE), Vancouver, Canada, June 2007.

M.-Z. Poh, N. C. Swenson, and R. W. Picard, “A wearable sensor for unobtrusive, long-term assessment of electrodermal activity,” IEEE Transactions on Biomedical Engineering, vol. 57, no. 5, pp. 1243–1252, 2010.

S. Batchelor, J. Torbeyns, and L. Verschaffel, “Affect and mathematics in young children: an introduction,” Educational Studies in Mathematics, vol. 100, no. 3, pp. 201–209, 2019.