^{1}

^{1}

^{2}

^{1}

^{2}

This article encompasses a systematic review of the research on ICT-based learning environments for metacognitively oriented K-12 mathematics education. This review begins with a brief overview of the research on metacognition and mathematics education and on ICT and mathematics education. Based on a systematic screening of the databases Web of Science and ERIC wherein three elements—ICT-based learning environments, metacognitive pedagogies, and mathematics—are combined, 22 articles/studies were retrieved, situated at various educational levels (kindergarten, elementary school, and secondary school). This review revealed a variety of studies, particularly intervention studies, situated in elementary and secondary schools. Most studies involved drill-and-practice software, intelligent tutoring systems, serious games, multimedia environments, and computer-supported collaborative learning environments, with metacognitive pedagogies either integrated into the ICT software itself or provided externally by the teacher, mainly for arithmetic or algebraic word problem-solving but also related to other mathematical topics. All studies reported positive effects on mathematical and/or metacognitive learning outcomes. This review ends with a discussion of issues for further theoretical reflection and empirical research.

In

This article encompasses a systematic review of the research on learning mathematics in metacognitively oriented ICT-based learning environments: research describing and evaluating attempts to increase the power of ICT-based learning environments for K-12 mathematics education by enriching them with a metacognitive pedagogy, or the reverse, that is, attempts to increase the power of metacognitive pedagogies for K-12 mathematics education by embedding them in ICT-based learning environments. We begin with a brief overview, first, of the research on metacognition and mathematics education. Afterwards, we briefly review the research on ICT and mathematics education. Finally, we explicate the aims and scope of our literature review wherein these three elements—metacognition, ICT, and mathematics education—are combined.

Since the 1980s, it is commonly accepted that the cognitive system depends on higher-order processes that enable it to work efficiently. Examples of metacognitive components are planning, monitoring, control, and reflection (e.g., [

While the early research on metacognition assumed that children younger than 10 years have little or no metacognition, from the late nineties on, researchers started to report evidence demonstrating that kindergarten children can already spontaneously plan, monitor, control, and reflect on their mathematics activities [

The relationship between metacognition and mathematical reasoning and problem-solving is well documented in the research literature (e.g., [

Given the well-documented positive association between metacognition and mathematical reasoning or problem-solving [

Most of these methods were routed in the seminal work of Schoenfeld [

These results about the possibility of improving mathematical competence through metacognitively oriented instruction were confirmed in a number of subsequent studies throughout the nineties and into the 2000s not only with university students but also with learners as young as kindergartners (e.g., [

In particular, Kramarski, Mevarech, and colleagues engaged in a robust program of research related to metacognition instruction in mathematics for more than 20 years (e.g., [

Introducing the new materials, concepts, problems, or procedures using metacognitive scaffolding

Metacognitive self-directed questioning in small groups or individually

Practicing by employing the metacognitive questioning

Reviewing the new materials by teacher and learners, using the metacognitive questioning

Obtaining mastery on higher-order and lower-order cognitive processes

Verifying the acquisition of cognitive and metacognitive skills based on feedback-corrective processes

Enrichment and remedial activities

The core component of the IMPROVE program consists of training the learners to use four kinds of metacognitive self-directed questions:

Comprehension: What is the problem all about?

Connection: How is the problem at hand similar to or different from problems you have solved in the past?

Strategies: What strategies are appropriate for solving the problem and why?

Review: Does the solution make sense? Can you solve the problem differently, how? Are you stuck, why?

However, in many cases, the positive effects of all these interventions were moderate rather than strong, implying that improving learners’ competence in mathematical reasoning and problem-solving by means of metacognitively oriented interventions is a complex and demanding matter, which comes with a cost, in terms of effort, time consumption, and reduction of the curriculum in the traditional sense [

Similar to the research literature on metacognition and mathematics education, the literature on metacognition and ICT is immense. In the early days of ICT, a computer’s potential was mainly seen as a machine for drill-and-practice in basic arithmetic and algebraic skills [

So, already for several decades, ICT is being considered a powerful tool for teaching higher-order processes such as metacognition and self-regulation, in various domains including mathematics. As argued by Mayer (2010, in [

This claim has been reiterated by scholars working in research projects that are aimed at realizing learning outcomes in various curricular domains, including mathematics, through advanced learning technologies such as simulations, serious games, hypermedia, computer-supported collaborative learning environments, and virtual reality. According to these researchers, several design characteristics of these advanced learning technologies (such as learners’ access to nonlinear information, to multiple representations, or to input or ideas from other learners) may play a critical role in the development of their metacognitive or self-regulatory skills and, consequently, lead to improved learning outcomes in the curricular domain at stake [

However, the above-mentioned design features of these advanced learning technologies also require learners to actively, deliberately, and critically consider and monitor the logical relations between various pieces of information, different kinds of representation, or the varied input from co-learners, in relation to the progression in their own learning process. Clearly, all these processes can be characterized as metacognitive or self-regulatory [

Learners’ inability to monitor, control, and reflect on their learning with advanced learning technologies is not the only reason why the potential of advanced technologies in (mathematics) teaching and learning may not be fulfilled. Other reasons for that failure given in the literature are (1) the very nature of ICT, which calls for “trial-and-error” and does not encourage learners to reflect on their problem-solving processes, and (2) the cognitive overload in learners that these ICT-enhanced learning environments might be induced by the multitude of information and extra tools they offer [

This article reviews research on enriching ICT-enhanced environments by metacognitive scaffolds or prompts, or the reverse, that is, attempts to increase the power of metacognitive pedagogies by embedding them in ICT-based learning environments in the domain of K-12 mathematics education. So, our review covers research on how to embed metacognitive pedagogies in available ICT environments to increase the quality and efficiency of mathematics learning, as well as research on how to apply ICT to increase the quality and efficiency of metacognitively oriented mathematics learning environments.

With this systematic review, we hope to answer the following questions concerning the available research on mathematics learning in metacognitively oriented ICT-based learning environments:

What is the educational level of the learners (ranging from kindergarten to secondary education)?

What are the characteristics of the learners (e.g., regular learners, low- or high-achieving learners, or learners with special needs)?

What kind of ICT environment is being used (e.g., computer-supported practice, educational e-books, intelligent tutoring systems, serious games, multimedia, simulations, hypermedia, computer-supported collaborative learning environments, and virtual environments)?

What is the nature of the mathematical subdomain being addressed (e.g., early number sense, arithmetic, geometry, algebra, graphing, and word problem-solving)?

What research methodology is being used (e.g., observational study, case study, correlational study, (quasi-)experimental intervention study, and design research)? and in case of intervention studies, What is the nature of its conditions?

What kind of metacognitive training, guidance, or support is provided (e.g., utilizing software features that help the learner to navigate within a math learning environment, providing assistance in solving problems or processing information by offering metacognitive scaffolds, and increasing the awareness of the problem-solving or learning process with mirroring tools [

What are the targeted learning outcomes of the learning environment (i.e., cognitive knowledge and skills in a given content domain, metacognitive knowledge and self-regulatory skills, improved performance in the specific mathematical subdomain being addressed, or affective or motivational learning outcomes)?

What are the main findings?

We conducted a systematic review of the research literature regarding technology-mediated K-12 mathematical learning environments implementing a metacognitive pedagogy, making use of the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses [

Overview of the search procedure.

First, a systematic search was done through the consultation of the databases ERIC and Web of Science. The databases were browsed for peer-reviewed English articles, based on four combinations—accounting for synonyms in search terms—that is, “metacognition AND mathematics AND online learning” (

Second, the literature findings were narrowed down by excluding duplicates and conference proceedings, resulting in 46 articles.

Third, the dataset was further reduced—by reading the abstracts, and if necessary the full texts—based on four inclusion criteria that were implied by the focus of this article. More particularly, the study has to involve (1) an online learning environment, (2) K-12 learners in regular education, (3) relation to math content, and (4) implementation of a metacognitive pedagogy. After application of these four criteria, 10 articles remained in the dataset.

Fourth, backward and forward reference search was applied to these 10 articles. In case of backward reference searching, reference lists of these 10 articles were systematically screened to identify relevant studies for the scope of this article. Forward reference search was applied to identify more recent articles that cited one of the initial 10 articles in our dataset. This backward and forward reference search resulted in another 12 articles that also met the above-mentioned inclusion criteria. This means that a total number of 22 articles formed the basis of this review study; these articles are marked with an asterisk in the reference list.

To answer the above-mentioned research questions, each article was summarized in terms of the following aspects: (a) the educational level of the learners, (b) other characteristics of the learners, (c) the kind of ICT-based environment being used, (d) the mathematical subdomain being addressed, (e) the metacognitive pedagogy being included, (f) the overall research method and the nature of its conditions, (g) the dependent variables being targeted, and (h) the obtained results.

An overview of the retrieved studies on metacognitively oriented ICT-based learning in K-12 mathematics education is provided in Table

Overview of the retrieved articles on a metacognitively oriented ICT-based learning environment for K-12 mathematics.

No. | Article | School level | Participants | ICT-based environment | Math subdomain | Research method and conditions | Metacognitive training/guidance/support | Targeted learning outcomes | Main findings |
---|---|---|---|---|---|---|---|---|---|

1 | Baeten et al. [ |
K | 167 last-year kindergarten children | Computer-supported practice | Early number sense | Intervention study with 5 computer-based practice conditions: (1) counting; (2) comparison; (3) counting + comparison; (4) counting + comparison with MC support; (5) gaming (control condition) | Training in monitoring and self-regulation of simple memorization skills (computer-based) | Cognitive: early math skills | Metacognitive condition was most effective, even after controlling for intelligence. In all computer-based conditions, lower performing children made higher learning gains, possibly because of ceiling effects. |

2 | Elliott and Hall [ |
K | 54 at-risk children | Computer-supported practice (number farm) | Early number sense | Intervention study with 3 conditions: (1) computer-based practice + MC support; (2) computer-based practice; (3) no computer-based practice (control condition) | Various instructional strategies (i.e., goal identification, active monitoring, modeling, scaffolding, questioning, reflecting, etc.) for the MC skills to be learned (teacher-based) | Cognitive: math achievement | Computer-based practice + MC support group outperformed other groups on math achievement. |

3 | Shamir and Lifshitz [ |
K | 77 at-risk children | Educational e-book | Early number sense | Intervention study with 2 conditions: (1) e-book with MC guidance; (2) e-book without MC guidance | Presentation of MC questions on each screen page, before (questions with respect to planning) and after (questions with respect to monitoring understanding) each activity (computer-based) | Cognitive: emergent literacy (rhyming) and emergent mathematics skills (addition and ordinal numbers) | Both conditions improved on emergent literacy and math skills. |

4 | Chen and Chiu [ |
E | 80 fifth graders (from 4 classes) | Computer-supported collaborative learning | Geometry | Intervention study with 2 conditions: |
Provision of a fixed sequence of prompts for individual MC processes and for structuring pupils’ interactions about MC (based on the “think-pair-share method”) (computer-based) | Cognitive: lower levels of math literacy (cognitive level of understanding and applying) and higher levels of math literacy (cognitive level of analyzing and evaluation) |
MC scripted group outperformed MC nonscripted group on high-level math literacy, but no difference on low-level math literacy. |

5 | de Kock and Harskamp [ |
E | 390 fifth graders (from 18 classes) | Computer-supported practice | Word problem-solving | Intervention study, in the real-school setting, with 2 conditions: (1) computer-based practice with MC support (experimental); (2) practice without computer and without MC support (control) | Provision of a sequence of general MC prompts/questions (i.e., read and analyze, explore, plan, verify, and reflect) and content-specific MC prompts that pupils are free to consult (computer-based) | Cognitive: word problem-solving |
Experimental group outperformed control group on solving word problems and on self-monitoring behavior. |

6 | Dresel and Haugwitz [ |
E | 151 sixth graders | Computer-supported practice (MatheWarp) | Unspecified | Intervention study with 3 conditions: (1) MatheWarp with attributional feedback; |
Provision of worksheets made up of questions addressing MC processes (e.g., planning and self-evaluation) (teacher-based) | Cognitive: achieved math knowledge |
Conditions 1 and 2 outperformed condition 3 on motivation questionnaire. |

7 | Jacobse and Harskamp [ |
E | 49 fifth graders (from 2 classes) | Computer-supported practice | Word problem-solving | Intervention study with 2 conditions: (1) computer-supported practice with MC support (experimental); (2) traditional classroom teaching (control) | Provision of a sequence of general MC prompts/questions (i.e., read and analyze, explore, plan, verify, and reflect) and content-specific MC prompts that pupils are free to consult (computer-based) | Cognitive: word problem-solving |
Condition 1 outperformed condition 2 on word problem-solving and on MC measures. |

8 | Lazakidou and Retalis [ |
E | 24 fourth graders | Computer-supported collaborative learning | Word problem-solving | Case study | Sequence of learning activities for augmenting MC skills: (1) observing an expert; (2) collaboration in small groups; (3) individually solving problems with coaching; (4) individually solving problems without coaching (computer- and teacher-based) | Cognitive: problem-solving performance |
There were a significant increase from pre- to posttest in problem-solving performance, a |

9 | Okita [ |
E | Study 1: 62 children (9–11 yrs) to 40 children |
Computer-supported practice | Arithmetic | 2 intervention studies with 2 computer-supported practice conditions each: (1) self-training: pupils solved all problems on their own; (2) self-other training: pupils worked with computer character | In the self-other condition, pupils had to monitor a computer character displaying correct or erroneous reasoning processes (computer-based) | Cognitive: calculation time and accuracy |
There were longer calculation times but higher accuracies in the self-other group. |

10 | Teong [ |
E | 40 low achievers (11-12 yrs) | Computer-supported practice (WordMath) | Word problem-solving | (a) Intervention study with 2 computer-supported practice conditions: (1) practice without MC guidance; (2) practice with MC guidance (b) Case study | Provision of a card with MC cues and questions related to different phases of the word problem-solving process (i.e., careful reading, recalling possible strategies, implementing possible strategies, monitoring, and evaluation) (teacher-based) | Cognitive: word problem-solving (experimental study) |
Practice with MC guidance condition outperformed condition without MC guidance on cognitive and metacognitive outcomes. |

11 | Kapa [ |
S | 441 eighth graders (high and low achievers) | Computer-supported practice | Word problem-solving | Intervention study with 4 computer-supported practice conditions: MC support (1) during and after problem-solving process, (2) only during problem-solving process, and (3) only after problem-solving process; (4) no MC support | Provision of MC questions during different phases of the word problem-solving process (e.g., What are you asked to find?) and/or MC questions after problem-solving (e.g., Is there any other way to solve the problem you have already solved?) (computer-based) | Cognitive: performance on word problems | The two conditions providing MC support during the solution process were more effective than the learning environments providing MC support after the solution process or no MC support. Low-achieving students were more influenced by MC support than the high-achieving ones. |

12 | Kapa [ |
S | 231 eighth graders (from 14 classes) | Computer-supported practice | Word problem-solving | See Kapa [ |
See Kapa [ |
Cognitive: performance on near-transfer tasks (problems similar to the intervention) and far-transfer tasks (unfamiliar open-ended problems) |
MC support was effective in realizing near and far transfer. |

13 | Kramarski and Dudai [ |
S | 100 ninth graders (from 3 classes) | Computer-supported collaborative learning environment | Word problem-solving and math inquiry | Intervention study with 3 conditions: (1) online learning + group feedback guidance (GFG); (2) online learning + self-explanation guidance (SEG); (3) control group (same learning tasks, but not online and no MC guidance) | GFG and SEG: instruction and practice in 4 self-addressed MC questions (related to comprehension, connections, strategies, and reflection) based on the IMPROVE instructional approach |
Cognitive: students’ mathematical inquiry ability (problem-solving, explanations, mathematical feedback in online forum discussions, and transfer ability) |
GFG students outperformed SEG students in most cognitive and metacognitive measures and control students in all measures. |

14 | Kramarski and Friedman [ |
S | 90 eighth graders | Multimedia environment | Graphing | Intervention study with 3 multimedia conditions: (1) unsolicited, i.e., with automatic MC prompts (system control); (2) solicited, i.e., free access to MC prompts (learner control); (3) control condition: multimedia environment without any MC prompts | Unsolicited or solicited: instruction and practice in 4 self-addressed MC questions (related to comprehension, connections, strategies, and reflection) based on the IMPROVE instructional approach (computer-based) | Cognitive: immediate problem-solving comprehension (think-aloud protocol); delayed transfer problem-solving test |
Unsolicited condition was better than solicited condition and solicited condition was better than control condition on immediate comprehension tasks, delayed transfer, and level of metacognitive discourse, particularly in the planning phase. Unsolicited condition was worse than solicited condition and solicited condition was worse than control condition on perceived mental effort. Unsolicited group consulted more aids. |

15 | Kramarski and Gutman [ |
S | 65 ninth graders (from 2 classes) | Computer-supported practice | Algebra | Intervention study with 2 computer-based practice conditions: (1) practice without MC support; (2) practice with MC support | Instruction and practice in 4 self-addressed MC questions (related to comprehension, connections, strategies, and reflection) based on the IMPROVE instructional approach (teacher-based) |
Cognitive: math test on linear functions: (1) standard, procedural tasks; (2) more complex transfer tasks; (3) providing math explanations |
Both conditions improved in cognitive measures after the intervention. MC condition outperformed non-MC condition on standard procedural tasks, transfer tasks, and providing math explanations. |

16 | Kramarski and Hirsch [ |
S | 43 students (13 yrs; from 2 classes) | Computer-supported practice | Algebra | Intervention study with 2 computer-based practice conditions: (1) with MC support; (2) without MC support | Instruction and practice in 4 self-addressed MC questions (related to comprehension, connections, strategies, and reflection) based on the IMPROVE instructional approach (computer- and teacher-based) | Cognitive: algebraic thinking |
Condition with MC students outperformed condition without MC on algebraic thinking and effective regulation of their learning. |

17 | Kramarski and Mizrachi [ |
S | 43 seventh graders (from 2 classes) | Computer-supported collaborative learning environment | Word problem-solving | Intervention study with 2 CSCL conditions: (1) with MC guidance; (2) without MC guidance | Instruction and practice in 4 self-addressed MC questions (related to comprehension, connections, strategies, and reflection) based on the IMPROVE instructional approach (computer- and teacher-based) | Cognitive + metacognitive: students’ problem-solving of real-life tasks: understanding the task; using math strategies; processing information; math reasoning |
Task performance in the online environment was significantly higher in MC guidance condition than in condition without MC guidance on understanding the task, using math strategies, processing information, and math reasoning. |

18 | Kramarski and Mizrachi [ |
S | 87 seventh graders (from 4 classes) | Computer-supported collaborative learning environment | Word problem-solving | Intervention study with 4 conditions: (1) computer-based environment with MC guidance; (2) computer-based environment without MC guidance; (3) face-to-face environment with MC guidance; (4) face-to-face environment without MC guidance | See Kramarski and Mizrachi [ |
Cognitive: standard test on algebraic knowledge |
Online + MC > F2F + MC > online = F2F for all outcome measures (cognitive and metacognitive). Online > F2F for part of the algebraic knowledge test and solving real-life problems, but no difference for SRL. |

19 | Maras et al. [ |
S | 40 students with autism (13 yrs); 95 typically developing students (13 yrs) | Computer-supported practice (Math Challenge) | Unspecified | Intervention study with 2 computer-based practice conditions: (1) with MC feedback; (2) without MC feedback | Engage in MC monitoring activities before and after solving each mathematical task (e.g., intentions and judgements of accuracy) and negotiate with the system on the level of difficulty (computer-based) |
Cognitive: performance on math test |
No infect of additional feedback on judgement of confidence and intention monitoring. Feedback condition outperformed nonfeedback condition on math test performance for both types of students. |

20 | Roll et al. [ |
S | Study 1: Aleven et al. [ |
Intelligent tutoring system | Algebra and geometry | Intervention study with 2 ITS-based conditions: (1) ITS with MC help tutor; (2) conventional ITS without MC help tutor | Provision of intelligent and adaptive help with respect to students’ MC behavior while working through the math problems. The nature of the MC help is adapted to the level of MC development (computer-based) | Cognitive: math performance |
Help tutor improved students’ help-seeking behavior when working with the cognitive tutor. |

21 | Roll et al. [ |
S | Study 1: 58 tenth/eleventh graders |
Intelligent tutoring system (Geometry Cognitive Tutor) | Geometry | Intervention study with 2 ITS-based conditions: (1) Geometry Tutor with MC help tutor; (2) Geometry Tutor without MC help tutor | Study 1: see Roll et al. [ |
See Roll et al. [ |
Study 1 and study 2: help tutor improved students’ help-seeking behavior while learning with the tutor. |

22 | Schwonke et al. [ |
S | 60 eighth graders (high and low achievers) | Intelligent tutoring system (Geometry Cognitive Tutor) | Geometry | Intervention study with 2 ITS-based conditions: (1) with MC support; (2) without MC support | Provision of MC cue cards encouraging and helping students to use instructional resources in the learning environment (i.e., 3 hints on “How do I solve the problem?” and 3 hints on “What do I do when I get stuck?”) (teacher-based) | Cognitive: effects on procedural and conceptual knowledge; time needed for learning in the environment (learning efficiency) |
Metacognitive condition outperformed only the control condition for low achievers in view of conceptual knowledge. No differences for high achievers or for procedural knowledge. Metacognitive condition improved learning efficiency by reducing the total learning time. |

We retrieved three studies on metacognitively oriented ICT-based learning environments for mathematics education in kindergarten [

Children in these studies had the mean age between 4 and 6 years. Two of the three studies (2 and 3) involved children at risk for (mathematical) learning disabilities.

In two studies, the ICT-based environment involved drill-and-practice (1 and 2), while the other one was built around an educational e-book (3).

In all three cases (1, 2, and 3), the mathematical subdomain was early numerical skills such as counting, numerical magnitude comparison, ordinal number relations, and simple addition.

All three studies were intervention studies with (quasi-)experimental designs comparing an ICT-based environment with and without a metacognitive pedagogy, and in two studies (1 and 2), a non-ICT control condition was added.

The metacognitive training was embedded twice in the ICT environment (1 and 3) and once provided by the teacher as a supplement to the existing ICT environment (2). The metacognitive strategies taught to the children were basic forms of planning, monitoring, and reflection, which were provided by means of instructional techniques such as providing cues and questions, demonstrations, and explanations.

As far as targeted learning outcomes are concerned, all three studies assessed the impact on the early mathematics skills that were addressed in the learning environment, and in one study (3), transfer to the emergent literacy skill of rhyming was assessed too. None of the studies involved a metacognitive measure.

In all three studies, the extra metacognitive pedagogy resulted in a significant positive learning effect compared to the other conditions. Remarkably, in the study of Shamir and Lifshitz [

In sum, even though only few studies have explored the potential of metacognitively oriented ICT-supported mathematics education to preschoolers, the available evidence suggests that it is possible to pay instructional attention to basic metacognitive skills in ICT-based environments from a very early age on, before the start of elementary school. Not surprisingly, at-risk children seem to be seen as a particularly relevant group for this kind of learning environments, and in line with the educational technology for early mathematical education in general, the privileged application domain is drill-and-practice software for basic numerical skills, although one study involved e-book reading. While these kindergarten studies suggest that this instructional attention to metacognitive pedagogy in ICT-based environments may lead to better learning outcomes in emergent mathematical and/or literacy skills, absence of metacognitive pre- and posttest measures and/or process data about the development of children’s metacognitive behavior during the intervention forces to be cautious by making causal claims about the role of metacognition in children’s improved early mathematical and/or literacy skills. Furthermore, the results indicate that these metacognitive pedagogies may either be provided by the ICT itself or be supplemented by the teacher. Finally, reflection on the disappointing results for early mathematics from one study (3) led to the important warning that the added metacognitive support runs the risk of overloading young children’s working memory, especially when these children are at risk for learning difficulties.

We retrieved seven studies on metacognitively oriented ICT-based learning environments for mathematics education in elementary school [

These seven studies involving elementary school children showed a strong tendency to focus on regular children from the upper grades of elementary school. Remarkably, none of the studies involved children in the age group between 6 and 9, and in only one study (10), participants were low achievers in mathematics.

The studies applied a variety of ICT uses. In five studies (5, 6, 7, 9, and 10), the ICT-based learning environment essentially involved individual tutoring in and practicing of the targeted content and skills, and the two remaining ones (4 and 8) were essentially about computer-supported collaborative learning.

As far as the mathematical content is concerned, more than half of the studies (5, 7, 8, and 10) were specifically about mathematical word problem-solving, one about arithmetic computation (9), one about geometry (4), and one covered the whole mathematical curriculum (6).

As for the kindergarten level, the majority of the studies (4, 5, 6, 7, 9, and 10) were (quasi-)experimental intervention studies comparing various experimental and control groups. One of these studies (10) also had a case-study component (10b), and one other study (8) was an independent case study. As far as the experimental studies are concerned, four of them (4, 6, 9, and 10a) involved a comparison between an ICT-based learning environment with and without a metacognitive pedagogy, whereas the two other intervention studies (5 and 7) compared an ICT-based version with a metacognitive pedagogy with a “business as usual” mathematics education control condition. Interestingly, one pair of studies (5 and 7) consisted of an evaluation of the effectiveness of a metacognitively oriented computer-based word problem-solving training under rather strictly controlled experimental conditions and realized by the researchers themselves (7), followed by an examination of its effectiveness in a more naturalistic setting (5).

Most of the time (4, 5, 7, and 9), the metacognitive pedagogy was implemented in the ICT-based learning environment, whereas in two other studies (6 and 10), it was the teacher who was responsible for the extra metacognitive instruction and support through the organization of the lessons and/or additional curricular materials (e.g., worksheets), and in one other study, the metacognitive pedagogy was jointly realized by the computer and the teacher. The nature of the metacognitive pedagogy differed according to the mathematical content. For instance, the studies focusing on word problem-solving consisted of a number of general and/or specific metacognitive hints or questions—provided by the computer program or the teacher—that helped the learner to approach and solve each word problem in a thoughtful and systematic way. For instance, in Teong’s study [

While in the three kindergarten studies, the assessment was restricted to cognitive learning outcomes related to the content and skills addressed in the training, in all studies with elementary school children, the assessment also involved measurements of a metacognitive (4, 5, 6, 7, 8, 9, and 10) and motivational (6) nature. These metacognitive measures involved (a) general questionnaires about metacognition, such as the “Motivated Strategies for Learning Questionnaire” (MSLQ) [

In all studies, positive results were reported concerning the superiority of the condition that combined ICT and metacognitive pedagogy, even though the design of several of these studies did not allow straightforward conclusions about the added value of this combination compared to other conditions involving only ICT, only metacognitive pedagogy, or neither of the two. Interestingly, in all cases wherein both cognitive and metacognitive measures were involved, learners in the ICT-based environment with metacognitive guidance demonstrated positive learning effects both for the cognitive outcomes related to the mathematical content and skills addressed in the training and for metacognitive outcomes (4, 5, 6, 7, 8, 9, and 10). Some additional findings obtained in studies that made extra attempts to understand the underlying mechanisms were that learners who made more use of the metacognitive guidance benefitted more from the intervention (7), that extended training provided longer-lasting learning effects (9), and that learning effects were not restricted to immediate effects but lasted at least until five months after the end of the intervention (6).

In sum, various scholars have tried to enhance children’s cognitive and metacognitive development in the context of elementary school mathematics learning, particularly in the context of word problem-solving. In the studies reviewed above, all of which involved regularly developing children from the upper grades of elementary school, the metacognitive pedagogy was either embedded within the ICT environment or provided by the teacher or extracurriculum materials. Taken as a whole, these studies provide ample evidence for the positive role that a metacognitively oriented training combined with an ICT-based learning environment can play in enhancing (upper) elementary school children’s mathematical performance, their metacognitive skills, and—at least in one study—also their motivation. However, flaws in the design of the studies and/or in the analyses of the learning processes during the intervention do not always allow a clear and convincing picture of the contribution of the ICT and the metacognitive pedagogy aspects to the success of the intervention.

Also with samples of secondary school students, researchers have examined the instructional interventions designed to influence and exploit students’ metacognition during mathematical reasoning and problem-solving in an ICT environment or to make their metacognitively oriented mathematical learning environment more powerful by means of ICT [

Except for two studies (20 and 21), which (also) involved students from the upper grades of secondary school, all studies with secondary school students took place in the lower and middle grades of this school level. Moreover, all studies involved regular learners, except for one study with low-achieving students (11) and one with students with autism (19).

Compared to the two lower educational levels, the studies with secondary school students involve a somewhat broader variety of ICT uses. Most of the studies used traditional ICT for individual tutoring in and/or practicing of certain mathematics knowledge and skills (11, 12, 15, 16, and 19), while some others (20, 21, and 22) used intelligent tutoring systems. In another substantive set of studies (13, 14 17, and 18), students were put in the complex (multimedia) learning environment wherein they could explore, in pairs or small groups, certain mathematical topics and/or solve mathematical problems by making use of various search, representation, and communication tools.

The studies covered various parts of the secondary school curriculum. As for secondary school mathematics, (algebra) word problem-solving was the most frequent topic (11, 12, 13, 17, and 18). Other topics were geometry (20, 21, and 22), algebra (15, 16, and 20), and graphing (14).

All studies were (quasi-)experimental studies comparing the efficacy of two or more learning environments with or without certain ICT-based instructional elements and/or metacognitive pedagogies. Several studies had basic experimental designs, such as the comparison of an ICT-based instructional program with or without the provision of extra metacognitive cue cards (15, 16, 15, 17, 19, 20, 21, and 22), whereas others had more complex designs involving three or even four conditions combining the absence or presence of ICT-based and metacognitively oriented instructional elements (11, 12, 13, 14, and 18).

The majority of the studies involving secondary school students had metacognitive hints or guidance tools that were actually embedded in the ICT software (11, 12, 14, 15, 19, 20, and 21). In only one study, the metacognitive pedagogy was provided by the teacher (22), and in the remaining studies (13, 16, 17, and 18), the computer and teacher were jointly responsible for the metacognitive training component. The concrete nature of the (computer-based) metacognitive guidance and support differed according to the type of ICT environment and its mathematical content, but essentially, it consisted of providing cues and questions before, during, and/or after task execution. In the studies of Kramarski and associates, the nature of the metacognitive guidance and support was based on the IMPROVE approach towards metacognitively oriented instruction explained in Introduction. For instance, in their study about e-learning, Kramarski and Gutman [

As for the investigations with elementary school children, except for one study that only measured the impact on students’ mathematical learning outcomes (11), all studies at the secondary school level involved both cognitive and metacognitive measures (12, 13, 14, 15, 16, 17, 18, 19, 20, 21, and 22), while, except for study no. 14, none included affective/motivational measures as well. The cognitive measures are aimed at assessing content-related learning, which was mostly closely related to the mathematical content being taught but sometimes also involved more distant measures such as specially designed transfer tests or general mathematics achievement tests (e.g., 12, 13, and 15) or assessments aimed at assessing possible changes in students’ cognitive processes and strategies (e.g., 14). The metacognitive measures involved (a) general questionnaires about metacognition or self-regulation being administered at (pretest and) posttest (e.g., 13, 14, 15, 17, 18, and 19), (b) specifically designed task-specific instruments that are aimed at revealing the use of the intended metacognitive knowledge and/or skills while performing the task during (pretest and) posttest (e.g., 20 and 21), and/or (c) log-file analyses or other types of qualitative evaluations of learners’ metacognitive behavior while working individually, in pairs or groupwise on the mathematical tasks in some or all experimental conditions (e.g., 12, 14, 16, and 22).

As far as the results are concerned, generally speaking, the experimental condition wherein the combination of ICT use and metacognitive pedagogy was implemented in the most intensive or most sophisticated way led to better cognitive learning outcomes, at least for some of the cognitive measures, in terms of learning outcomes as revealed either by pre- and posttest comparisons or by evaluation of the quality of the mathematical work during the instructional program (11, 12, 13, 14, 15, 16, 17, 18, 19, 20, and 21b). Moreover, most studies report positive outcomes for at least some of their metacognitive measures (13, 14, 15, 16, 17, 18, 20, 21, and 22). Interestingly, the few studies that performed separate analyses for different levels of mathematical achievement showed that the learning environment was particularly helpful for the low-achieving students (11 and 22).

However, the size and nature of the effects differed depending on the way in which the metacognitive guidance and training were provided. For instance, the studies suggest that group feedback guidance works better than self-explanation guidance (13), that consistently and regularly exposing students to on-screen metacognitive prompts (without learner control) is more efficient than giving students free access to these prompts (14), and that embedding the metacognitive prompts in the online learning environment leads to better learning effects for mathematics than presenting them face-to-face (18). However, not all studies yielded significant positive outcomes for mathematics learning through metacognitive pedagogies. For instance, in studies by Roll et al. ([

In this article, we have systematically synthesized the research that integrates ICT and metacognitive pedagogies in mathematics learning from preschool until the end of secondary education. This multidimensional research topic has been approached from two different angles. On the one hand, researchers working with new ICT-based environments for mathematics learning became aware of the pitfalls and risks involved with the increased use of ICT for mathematics learning when viewed from a metacognitive perspective. On the other hand, based on the finding that learners of different ages trained in learning to monitor and control their own cognitive processes for mathematics problems and learning mathematics do better than untrained learners, researchers have started to design metacognitive pedagogies for mathematics learning. Initially, these intervention studies were carried out in traditional learning environments that did not make intentional and systematic use of ICT (e.g., [

As a result of these two distinct but complementary developments within research, the past decades have witnessed a productive line of research on the development and use of various ICT-based tools to stimulate and support different aspects of metacognitively guided mathematics problem-solving and learning.

Given the recency of researchers’ interest in young children’s metacognitive development in general [

Second, our review yielded great variation in the kind of ICT-based learning environments that have been developed and used in these studies, including computer-supported practice, educational e-books, intelligent tutoring systems, serious games, multimedia, and computer-supported collaborative learning environments, and this variation tends to become larger with increasing educational levels. While the relation with metacognitive pedagogies has been explored and investigated for many kinds of ICT applications in mathematics education, our literature search has not identified studies that have examined the effects of learning mathematics with newer information and communication technologies such as Facebook and Twitter, or virtual and augmented reality. Given the increasing use of these newer ICT applications and technologies, also in (math) educational contexts [

Third, our review revealed that the studies that investigated the interrelation between ICT, metacognition, and mathematics have focused mainly on certain mathematical topics, with an emphasis on early numerical skills in kindergarten and arithmetic and algebra word problem-solving in, respectively, elementary and secondary schools. The strong emphasis on word problem-solving in this review is not surprising, given that mathematical word problem-solving has always been conceived as a privileged topic for assessing and stimulating metacognition and self-regulation [

Fourth, our review also yielded a wide variety of forms of metacognitive support at all educational levels. Overviewing the studies, a common characteristic seems to be the explicit teaching of the metacognitive skills within the content area through one or more powerful instructional techniques such as modeling and articulation of the intended metacognitive skills and the application of various forms of metacognitive scaffolding (including pop-up screens with metacognitive questions, cues, or scripts) or coaching during intensive and deliberate practice, either individually or in small groups. In contrast, only few learning environments make systematic use of the instructional technique of fading, i.e., the gradual removal of the metacognitive support as the learner progresses, and even fewer do so in an adaptive or intelligent way based on an individualized analysis of the level of metacognitive functioning of the learner. Also, explicit, informative, and individualized feedback on the learners’ metacognitive functioning during the mathematical task completion is typically lacking, although the intelligent tutoring system for learning secondary school algebra and geometry learning in [

Fifth, while our search terms allowed for a wide variety of research methods, including case studies, observational studies, correlational studies, design experiments, and (quasi-)experimental intervention studies, the vast majority of investigations were of the latter type. Many of those intervention studies were characterized by good internal validity, given that they involved both pre- and posttests and an experimental and control group or several experimental groups, allowing for valid conclusions about the contribution of the ICT and/or metacognitive element to the learning outcomes. However, in a significant number of studies, the research design did not allow to derive grounded conclusions as to what elements or aspects of the computer-based environment were exactly responsible for the obtained (positive) effects. Moreover, many studies involved only pre- and posttest measures, without additional attempts to assess the mathematical and metacognitive learning processes during the intervention, which further jeopardizes a deep understanding of the instructional factors and learning mechanisms that caused the obtained learning effects. Echoing Schunk and Greene’s [

Sixth, the reviewed studies also differ in terms of their dependent variables. While, as a direct result of the scope of our systematic review, all studies involved at least some kind of measure of mathematics achievement, there was great variety in the nature of this measure: some studies used a general measure (i.e., a standardized achievement test or an available overall grade score for mathematics), others made use of a specific measure that was particularly tailored to the mathematical content (e.g., arithmetic or algebra word problem-solving, and geometry) being addressed in the learning environment, and still others also included specifically selected transfer measures. Furthermore, in most studies, particularly at the elementary and secondary school levels, these mathematical assessments were combined with metacognitive measures. These measures were either metacognitive questionnaires or tests administered before and after the intervention or observations, self-reports, and log files collected during the intervention. While all authors explicitly or implicitly assume a positive mediating role of metacognitive activity in mathematics learning, quite remarkably, we did not encounter studies that explicitly tested this mediating effect. Given the assumed positive role of metacognition in ICT-supported mathematics learning, it is highly recommended that this mediating hypothesis is tested in an explicit and appropriate way. Finally, studies investigating the effect of ICT or metacognitive pedagogy on motivational-affective outcomes, such as students’ interest or self-esteem in mathematics, were very rare. Given the well-documented longstanding problems with the affective and motivational aspects of mathematics learning, from the elementary school level onwards [

Finally, as far as the research findings are concerned, taken together, the available research evidence indicates that ICT-based mathematics learning environments supported by metacognitive pedagogies enhance both mathematics learning and achievement and metacognitive thinking and learning, more than environments with no ICT and/or metacognitive support. However, this does not imply that all interventions were successful on all dependent variables being assessed and that all effect sizes were large. But given the literature search method being applied in our systematic review, we cannot exclude a publication bias towards successful studies with significant positive results in favor of the condition(s) with ICT and/or metacognitive support.

The authors declare that there are no conflicts of interest regarding the publication of this paper.