Journal of Robotics
Volume 2018 (2018), Article ID 4578762, 11 pages
https://doi.org/10.1155/2018/4578762

Review Article

Learning by Teaching with Humanoid Robot: A New Powerful Experimental Tool to Improve Children’s Learning Ability

Frank Jamet,1,2,3 Olivier Masson,1,2 Baptiste Jacquet,1,2 Jean-Louis Stilgenbauer,1,2,4 and Jean Baratgin1,2,5

¹P-A-R-I-S Association, 25 rue Henri Barbusse, 75005 Paris, France
²Laboratoire CHArt, Université Paris VIII, EPHE, 4-14 rue Ferrus, 75014 Paris, France
³Université de Cergy-Pontoise, 33 boulevard du Port, 95011 Cergy-Pontoise Cedex, France
⁴Facultés Libres de Philosophie et de Psychologie (IPC), 70 av. Denfert-Rochereau, 75014 Paris, France
⁵Institut Jean Nicod (IJN), École Normale Supérieure (ENS), 29 rue d’Ulm, 75005 Paris, France

Correspondence should be addressed to Jean Baratgin; jean.baratgin@univ-paris8.fr

Received 31 August 2018; Revised 26 October 2018; Accepted 1 November 2018; Published 18 November 2018

Academic Editor: Gordon R. Pennock

Copyright © 2018 Frank Jamet et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract.

Browsing the literature shows that an increasing number of authors choose to use the learning by teaching approach in the field of educational robotics. The goal of this paper is, on the one hand, to produce a review of articles describing the effects of this approach on learning and, on the other hand, to review the literature in order to explore the characteristics at the core of this approach. We will only focus on the work using a humanoid robot. The areas of learning studied are writing, reading, vocabulary, and reasoning, but also there are some metacognitive abilities like task commitment and mental state attribution. Their targets are from very young children to preadolescents. We can already notice some studies on pupils with special educational needs. In all of these domains, the results show a nonnegligible effect of learning by teaching both on learning and on metacognitive abilities. If the concept of learning by teaching is clear, a careful investigation of the different studies shows that experimental paradigms do not use the same basic characteristics. For some, it is the robot’s weakness, the care that must be given to it, which is the main requirement for the approach, while for others it is the unbalanced distribution of knowledge which is at the heart of it. The learning by teaching approach we will study has two components: the robot and the child tutor. The characteristics of the robot and what is asked of the child to accomplish his or her task of the tutor will be analyzed.

1. Introduction

“It is by teaching that men learn.” With this sentence, the Greek philosopher Seneca (4BC-65AC) lays the foundations, a bit less than 2000 years ago, of a new teaching method consisting in learning through the act of teaching. This method is now commonly referred to as learning by teaching. Its main interest is that it allows the tutor (the teacher) to conceptually increase his or her level of knowledge [1, 2]. For some years now, we see an increasing number of studies in which a human teaches a humanoid robot [3–11]. Teachings were on a great variety of areas such as reading [4], writing [6, 7, 10], language [5], reasoning [11, 12], but also metacognitive abilities like commitment to a task and mental states attribution [13]. These studies concentrate on young children (3 to 6 years old) [5, 11, 12], children (7 to 8 years old) [4, 6], preteenagers (13 to 14 years old) [7], and adults [8, 14]. They also focus both on typical students and on those with special educational needs: children with visuoconstructive deficits, with poor handwriting, and even poorer self-confidence or with attention disorders [7], with autism spectrum disorders [15], and with potential symptoms of a developmental disability [16]. In this review we will focus on studies with children.

2. A Pupil Named NAO

The use of humanoid robot [for a review of possible robots in education see [17]] with children has numerous advantages [18]. For example, humanoid robot facilitates the consideration of the robot as a social person by children [19–21] and seems to help children concentrate [22]. Furthermore, children are motived and have a positive attitude when seeing robots in their classroom [23–25].

In the majority of these studies on learning by teaching, the robot used was NAO (Figure 1): a bipedal, humanoid robot of 58 cm, with 25 degrees of freedom, cameras, and vocal capacities (synthesis and recognition), which can be programmed to carry out autonomously a set of tasks. This robot has many specific advantages that can be beneficial for experimental work with children, especially in the field of education [31]. By taking up the classification of the types of use of robots in the robotics studies applied by [32], NAO has five advantages when using it for learning by teaching.

Figure 1: NAO and the fruits used in [11].

(1) Height. This characteristic is mainly advanced to justify the choice of NAO to establish proximity, by its resemblance to a child [33].

(2) Movement precision. NAO has 25 degrees of freedom 25 degrees of freedom allowing it to perform movements close to human movements (walk, gestures, tilt its head, and appearing to look at something). NAO is able to manipulate small objects [5, 11], to make deictic gestures useful for learning [4, 11], and to mimic gestures or signs [34].

(3) Verbal interaction abilities. NAO has a speech synthesis module, which allows him to utter sentences aloud and to speak to the child orally [35]. In addition, the rate and tone of his voice are both easily parameterizable, which is particularly useful in learning by teaching, for example, in learning how to read.

(4) Nonverbal and emotional interaction abilities. Although devoid of facial expressions, NAO can express emotions (anger, fear, joy, sadness, and surprise) as much as a robot with this ability, such as iCat [33]. The way NAO can simulate emotions is through body movements and LEDs around the eyes (blinking and changing color). Hence the absence of facial expression of NAO does not constitute a barrier to the proximity felt by the child.

(5) Accessibility in terms of use and programming. NAO can be programmed by robotic and computer science neophytes thanks to the software Choregraphe (Aldebaran, Softbank Robotics), presenting a visual interface by which it is possible to program relatively complex behaviors in a simple way.

The aims of this paper are (1) to list the effects on learning when a human teaches a humanoid robot; (2) to analyze the different experimental characteristics of the humanoid robot; (3) to carefully analyze what is asked of the human; and finally (4) to show how the learning by teaching approach is a new paradigm for developmental and cognitive psychology.

3. Effects of the Learning by Teaching Approach on Learning

In this first part, we will be interested in the concept of learning by teaching by only focusing on the results of experimental studies in educational robotics which relies on the use of humanoid robots with children.

Five great areas of essential learning are set out: reading, writing, language, reasoning, and metacognitive abilities. They are summarized in Table 1.

Table 1: Main studies of Learning by Teaching according to the characteristics of the “pupil robot” and the “child teacher”.

Research
Robots Type
Robot’s characteristics
Children’s characteristic
Collaboration’ phenomenon

Reading

[4]
Nao
Autonomous
6 to 7 years old
Protégé effect

(Deictic gesture, pointing)

Writing  
Handwriting

[6]
Nao V4
Autonomous
6 to 8 years old
Induce agency

7 to 8 years old
anthropomorphizing.

Cognitively engage

[7]
Nao V4, V5
Autonomous
5 to 8 years old
Induce agency

with SEN
anthropomorphizing.

Cognitively engage

[26]
Nao
Autonomous
7 to 9 years old
Corrective feedback as a

peer assessment approach

Alone with NAO

[27]
Nao
Autonomous
5 to 6 years old
Protégé effect

Corrective feedback

[10]
Nao
Autonomous
7 to 9 years old age

Language

[5]
Nao
Tele-operated
3 to 6 years old
Care-receiving robot

[28]
Pepper
Total Physical Response
4 to 5 years old
Care-receiving robot

Reasoning

[11, 12]
Nao V5
Tele-operated, designed to give
5 to 6 years old
Alone with NAO

the impression to be ignorant

NAO is silly

Teacher status

Metacognitive abilities

Feasibility of collaborative learning

Learning time

Unable to answer

[15, 16]
Ifbot
Tele-operated, Limited number
8 to 11 years
correctly

of expressions and its arms and
old
Learning capability

body are immobile.
with SEN

Children’s engagement Attribution of mental states

[29]
Epi
Designed to give the impression
5 to 9 years old

of being a child while still being decidedly robotic.

Handwriting    Children’s engagement

[30]
NaoV4, V5
Tele-operated
5 years old
Protégé effect

6,05 to 8 years old
with SEN

Children with special educational needs: visuoconstructive deficits and poor handwriting and even poorer self-confidence.

Children with special educational needs: autism spectrum disorder and with potential symptoms of developmental disability.

3.1. Reading

To our knowledge, only the study of Yadollahi et al. [4] uses the approach of learning by teaching with a humanoid robot to develop reading abilities. In their setup described below, the authors [4] are interested in the effect of the robot’s deictic gestures during reading on the reading ability of the child. In this study, the deictic gestures consist in the finger moving during reading.

The setup is the following.

The child is seated at a table. In front of them is a book. NAO is on the table. The child and the robot can both read the book. The task of the robot is to read a part of the book and then, indicating that it is getting tired, the child continues reading. If the child finds a mistake, they signal it to the robot by pressing a red button. If they need to hear it again, they press a yellow button, and they congratulate it by pressing a green button. Depending on the feedback of the child, the robot reacts verbally, physically, and emotionally (eyes changing color).

Two groups of children between 6 and 7 years old are defined, based on their reading ability (high reading ability versus low reading ability). Good readers will work on a 36 pages long text, and the others on a 24 pages long text. Both groups are split between two conditions: in the first condition, the robot reads aloud with its finger (pointing), and in the second condition it does not follow with its finger (nonpointing). Two sessions of 30 minutes over two days are offered. If a good reader teaches NAO how to read, NAO reads the first 10 pages then indicates getting tired, and the child continues until page 16. The following day, the child finds NAO telling them that it is eager to know the rest of the story. The task continues following the same principles. NAO reads pages 16 to 26 and then indicates getting tired. The child finishes reading. For less advanced readers, the protocol is identical, except for the number of pages: NAO reads 9 pages and indicates getting tired, and the child continues reading 3 pages. NAO’s reading speed can be adjusted. It is set to 60% of normal speed for weaker readers and 80% for good readers.

Three types of mistakes are studied: mistakes of type I: reading “elephant” instead of “penguins,” mistakes of type II: reading “start” instead of “stop,” and mistakes of type III: pronunciation mistakes or syntax issues. These mistakes correspond to Goodman’s Miscue corpus [36, 37]. The NAO robot makes 12 mistakes during each session.

Results show that there is no effect of the condition (pointing versus nonpointing) on the quality of the correction; that pointing actually improves the detection of mistakes of type I (“elephant” instead of “penguins”); and that children of both levels of reading ability benefit from pointing when they must recognize mistakes that do not correspond to the context picture (type I). For readers with a low reading ability, pointing does not improve text comprehension.

3.2. Writing

Learning how to write is a complex task as it is a lengthy process and requires repeating the same gesture. Discouragement can quickly disturb the acquisition of the correct gesture. The challenge faced is thus consequent. Despite these difficulties, this area has numerous studies conducted with the approach of learning by teaching with the NAO robot [4, 6, 9]

In its design, the experimental setup must allow the child to draw many times and to be motivating in order for it to be efficient. To this we can add the additional difficulty: giving NAO the ability to write, even though it only has three fingers, and as a result, drawing cursive letters is impossible. As a consequence, the questions that need to be answered are the following: How to make a robot that only has three fingers be able to write? Does the setup allow the robot and the child to draw enough? Is it not tiresome? Does the setup allow the child to evaluate the robot? Are the relationships between children and the robot not influenced by the existence or absence of the robot’s progress? Is the child able to adapt his or her teaching? Does the progress of NAO give the child the impression that they are a better teacher?

To address the issue of NAO’s three fingers, the authors [6] synchronize a tablet. Thus, NAO is able to simulate with its finger the drawing of a letter on the tablet. It contains two writing zones: one on the left, spanning the 2/3rd of the tablet for NAO, and the one for the child, on the right, smaller. The child can draw and erase letters.

The task of the child of 6-8 years old is to teach NAO how to draw letters (lowercase and uppercase). NAO draws one of these letters and asks the child to correct it. NAO improves. The effects of feedback [38] and regular evaluation are determining factors [39]. Mistakes made by NAO when drawing letters are not the result of chance; they are observed during the development of the learning task: proportions, breaks, and the alignment [40, 41].

Does the setup allow the robot and the child to draw enough? Is it not tiresome? Results show that over the two sessions (respectively, 65 and 160 minutes long), the interaction is effective and efficient. The robot and the child both produce many drawings of letters: 96 letters, among which 49 in response to demonstrations with 6-7-year-old children, and 335 traces among which 152 in response to interventions of 7-8-year-old children.

In the teaching act, evaluating is an important concept. Was the setup efficient in this aspect? Did the progress of NAO influence the relationship with the child? To answer this question, the authors [10] randomly assign 25 7-9-year-old children in two conditions: one in which NAO makes progress: “NAO learns,” and the other in which it does not: “NAO does not learn.” NAO keeps all of its social qualities. Each condition contains four 15 minutes long sessions. The authors add a pretest and a posttest in which they measure the quality of the child’s writing (the ability to recognize letters and the ability to reproduce them) and ask the child about the progress of NAO along with the quality of their relationships with NAO. Results show that, in the condition “NAO learns,” children after the third session evaluate more favorably the performance of NAO than in the condition “NAO does not learn.” The same difference is observable in the fourth session. The relationships with the robot (attachment) are not influenced by the robot’s performance.

Is the child able to adapt his or her teaching? Does the progress of NAO give the child the impression that they are a better teacher? The authors [26] use the same experimental setup as in previous studies but add to the posttest an evaluation of children’s perceived self-efficacy towards tutoring. The robot is called Michael. The experimenter explains to the children (from 7 to 9 years old) that they will meet Michael four times. It needs help to learn how to write. The experimenter explains the role of Michael as a pupil and the role of the children as teachers. NAO is then placed in the room alone with the child. NAO then politely introduces itself and tells the child about its difficulties with learning how to write by showing them his grades. Children are randomly assigned to the two conditions “NAO learns” and “NAO does not learn.” Each condition contains four 15 minutes long sessions, one per week.

Does the robot’s space arrangement (face-to-face or side-by-side) affect the child’s focus of attention and perception of the robot’s performance in a situation of learning by teaching of writer? The authors [27] use the same experimental setup as in previous studies [4, 6, 9] on the cowriting. The children are distributed under two conditions: the face-to-face condition with the NAO and the condition where NAO is next to the child.

Results show that children do pay attention both to the grades and to the robot’s progress; (2) do notice the robot’s progress starting in the third session only in the condition “NAO learns”; (3) do give a better evaluation during the last session in the condition “NAO learns” compared to the condition “NAO does not learn”; (4) enjoy teaching more and more with the robot’s progress; (5) do feel like better teachers after each interaction; (6) in the condition “NAO learns” do improve their performance between the pretest and the posttest; (7) are not influenced by the robot’s performance in their attachment towards it; (8) only found the robot to be intelligent after the last session in the condition “NAO learns”; and (9) if the face-to-face versus side-by-side position does not affect the quality of the learning, the quality of the feedback changes. Children provide more positive feedback in a side-by-side position.

3.3. Language

The works of Tanaka et al. [5] are interesting, on the one hand because it deals with learning in the domain of language, and on the other hand because it enriches the conceptual framework of learning by teaching with care-receiving-robots (CRR). The framework of CRR was introduced for the first time in 2007 by [42]. The authors [42] show that in this framework, children present many caretaking and caregiving behaviors. They increase the motivation of the child in their teaching task building on the fact that the robot is fragile, and that it needs to be cared for. Subsequently, the authors [28] added to CRR the framework of Total Physical Response (TPR) developed by [43] in the 1960s. The TPR approach is a specific second language teaching method, which consists of associating a sentence with physical movement; for example, to learn the sentence “driving a car,” the student simulates driving by maneuvering a fictional steering wheel with both hands. This teaching method applies to both children and adults. It allows a pleasant learning and stress-free learning.

3.3.1. Learning English Action Verbs

The goal of the experiment of [5] is to evaluate the incidence of learning by teaching on the learning of English action verbs in Japanese children between 3 and 6 years old. The experiment is divided into four phases: (1) pretest; (2) interaction with the NAO robot in two parts; (3) free time; and (4) posttest.

The goal of the pretest is to select, with the child, six action verbs from a set of graphic cards representing an action like drinking, sweeping, brushing, and playing. The task of the child is to associate the word card to the graphic card. If the child succeeds, the experimenter says: “it is correct.” We keep playing until the child makes four mistakes.

The interaction with the NAO robot happens in two steps. In a first step, it is the experimenter who teaches the child, and then the child teaches NAO. The “lesson” happens as follows: the experimenter puts in front of NAO four items corresponding to four action verbs: a glass to drink, a hand brush to sweep, a brush to brush, and a toy to play. NAO, teleoperated, welcomes the child with his name and happily introduces itself by shaking their hand.

The experimenter has four cards at their disposal, among which two have the mention “with CRR” written on them. The experimenter takes a card randomly and asks the child: “Show me how to < verb >.” If the child cannot answer, the experimenter points to an item while pronouncing the verb and continues. If the experimenter takes a CRR card, they ask NAO to answer. NAO nods and says: “Please take …” while opening its hand. The experimenter takes the item and gives it to NAO. NAO mimics the action but makes mistakes. The experimenter tells it: “No, it’s incorrect.” NAO then asks: “Please, teach me.” The experimenter takes the robot’s hand and teaches it step by step. The teleoperator record the learning sequence. Once the learning is done, NAO acts the action while pronouncing the verb. The experimenter tells it: “Yes, it’s correct.” For the three other action verbs, the experimenter proceeds in the same way. During the second part, the protocol is identical but it is the child’s turn to teach the robot.

During the free time, the child is invited to play with NAO. The experimenter observes the child for 10 minutes. Finally, the experimenter begins a first posttest. The content is the same as in the pretest. A second posttest will be done 3 to 5 weeks later.

Results show that (1) performances in learning the action verbs are significantly improved when the child teaches the robot instead of when the experimenter teaches the child; (2) three to five weeks later, performances have increased even further; (3) the child uses different forms of teaching: a verbal teaching (the child states the action in English); a gestural teaching (the child moves NAO to do the action); a direct teaching (the child helps and back NAO step by step). The two first forms of teaching are the most commonly used (verbal and gestural), and (4) the fact of having brothers, sisters, or a dog has a significant influence on learning performance. The question of the incidence of gender should be investigated in future studies.

3.3.2. Learning of Nonnative Language

To allow learning of the nonnative language, at home, [28] proposes two methodologies in using the humanoid robot Pepper. This robot is 1.21m high and 42 cm wide and 48 cm thick. It weighs 28 kg and can move at a maximum speed of 3 km/h. A touch screen is placed on his chest. The English learning course can take place in two ways: (1) with a teacher in real time at the view on a screen. In this situation, Pepper is a student like any other. It is programmed to follow the teacher’s gaze. The results show that he favors the “circle time,” that is to say the moment when students gather around the teacher to listen to the instruction and (2) with a video that children can see on the chest of the robot Pepper. The originality of the learning situation is that it crosses the learning by teaching and the TPR. The child learns English using the TPR. He must also teach the Pepper robot using learning by teaching and TPR. These two learning modalities (learning by teaching and TPR) are added.

3.4. Reasoning

To our knowledge, studies of this kind in educative robotics are rare. Only one study [11] shows an interest in the question of class inclusion, a fundamental notion in the cognitive development of children. Studying class inclusion allows us to understand the links between “all” and “parts.” “All” is the sum of all parts. It is Piaget [44] who is the first to study the development of this notion in his famous flower task. He shows a bunch of flowers composed of four asters and two tulips. He asks the child “Are there more flowers or more asters?” It is necessary to wait until the age of 8 years old to have children answering correctly “there are more flowers.” Before the age of 7, the answer given is “There are more asters.” Piaget et al. [45] explain the incorrect answer of the children by the absence of reversibility in the child’s thinking. If the class flowers = asters + tulips, then tulips + asters = flowers. Politzer [46] suggests a new pragmatic linguistic relevance-theoretic explanation of children’s answer “more asters.” The task is a difficult one [45] because it requires the child to be mastering the logic of classes but also judgments of relations (more than, less than, same as). Furthermore, the question is ambiguous [46] in the sense that two interpretations are possible and relevant. Indeed, if the child interprets the notion of flowers as being the “all,” then there are indeed more flowers than asters, but if the child interprets the term flowers as what is not an aster then the answer “there are more asters” is actually correct. From the linguistic point of view, these two interpretations are correct. From the point of view of the pragmatics of language, comparisons generally happen on classes at the same level (in our case, between hyponyms apple and pear or asters, and tulips). The answer of the 7-year-old children illustrates this: “there are more apples.” For psychologists, this is an important question: do children master or not the notion of class inclusion at the age of 6-7? Age when, for Piaget et al. [44], the majority of subjects gives the answer “more asters?” Indeed, children of 5-6 years old are perfectly capable of understanding who is intelligent and who is not, who is qualified and who is not [47].

The originality of the work of Masson et al. [11, 12] is twofold: the authors first investigate the evolution of this central question in 1921 and then suggest an experimental paradigm to remove the pragmatic ambiguity within the question itself: after having indicated that the robot is called NAO, the experimenter explains to children their task: teaching new things to NAO, who knows nothing. The experimenter explicitly indicates to the children that they know much more than NAO. Here, children do not know beforehand what they will teach NAO. The experimenter tells the children that they will have to be careful as NAO’s questions may seem silly to them. The children will have to answer spontaneously without thinking too much about the question itself or the robot’s intention. The experimenter insists that if children do not agree with a suggestion from the robot, they are welcome to correct it. It is important to emphasize that children will be left alone with the robot. The experimental design here is a semiteleoperated situation where the experimenter is in another room without the child being able to see him or her. NAO is programmed to ask in an initial stage questions that should reinforce the children’s belief in them and in the robot’s great ignorance. Hence the child is in a position of a teacher, a person who knows, in opposition to NAO who is the “young” pupil.

The authors have integrated into the robot a program to execute the inclusion task. The important aspect is here. It is NAO during its questioning that will drive the child to teach it the link between the class and the subclass. It concerns the class fruit: 5 apples and 3 pears. Through a computer, the robot is teleoperated without the child’s knowledge by one of the authors hidden behind a wall from the child’s sight. The second experimenter brings the child from their classroom to the room where the experience takes place. On the way, the experimenter explains to the child the didactic contract. The child enters the room and sits at a table on which NAO is placed in a sitting position. The child and NAO are in front of each other, with the robot at the same level as the child so that it does not feel dominating. The experimenter that accompanied the child then leaves the room. The child is alone with the robot. NAO introduces itself, asking for the child’s name and if they want to be its teacher. NAO will use the child’s name to address them. NAO tells them that it knows nothing and that it might ask really silly questions. NAO points at the different items on the table and asks what they are. The child answers. NAO makes sure that it has understood by pointing at the different items again and repeating their name, but it makes mistakes. The child repeats the teaching phase until NAO is able to identify the class (fruits) and the two subclasses (apples and pears). In order to perfect its knowledge, NAO tries to know if “there are more fruits than apples?” The child answers. NAO when reformulating the answer makes a mistake: if the child tells it: “there are more fruits,” NAO answers: “Oh ok so there are more apples.” If the child says “there are more apples,” NAO continues by saying: “Oh ok so there are more fruits.” The child corrects the robot’s mistake. The dialogue between the child and NAO cannot be reduced to a system of preestablished question-answers, but we are in a true conversation between two conversation partners which explains why the robot is teleoperated.

This experiment has been carried out with 40 children in preschool aged between 5 and 6. They were randomly assigned to two conditions: one condition human-human and one condition NAO-human. The authors notice that one child out of five answers correctly in the human-human condition, and more than six children out of ten do in the condition NAO-human. The difference is significant. This result shows that (1) children master the notion of class inclusion earlier than Piaget et al. [45] indicated and (2) the pragmatic factors identified by Politzer [46] constitute a real obstacle in the original experiment.

A qualitative analysis shows that children are actually immersed in a conversation rather than in a list of questions and answers. If at the beginning of the conversation children are rather calm and patient, accepting that NAO makes mistakes in the identification of the different fruits (“this is an apple, this is a pear”), it should not take too long. Indeed, children will usually take the initiative and show NAO that an object is an apple and that another is a pear by pointing at them. Although they remain polite, the tone of their voice is firm and decisive: “Listen little robot, this is an apple and this is a pear.” That way they put an end to any ambiguity. They implicitly tell NAO that it must pay attention and try to understand! Similar expressions are found when NAO has some trouble with the concept of fruit. After the robot fails a couple of times to answer the question: “are there more apples or more fruits?,” the children move closer to NAO and tell it: “Listen little robot, it’s all fruits.” In all the conversations, the children remain polite and show respect to the robot.

3.5. Metacognitive Abilities

In this section, we will investigate the question of the effect of learning by teaching through three points of view: task commitment, motivation and self-confidence, and mental states attribution. These studies are interested in both typical children and those with special educational needs.

3.5.1. Task Commitment

Task commitment is an essential question for teachers [5, 30, 48]. It is one of the factors on which the efficiency of learning relies. The learning by teaching approach results in a well-known “protégé effect” [13]. In the teacher-pupil relationship, the teacher is responsible for the pupil. This responsibility manifests itself through a stronger time investment of the teacher in the tasks. This phenomenon is observed in children (10-11 years old) and in preteenagers (13-14 years old) spending more time on one task when they must teach a teachable agent than if they focused on learning for themselves [13]. Between a teachable agent and a social robot, children are more interested in the social robot [29]. The spatial position of the robot in the interaction also has an influence. It is a powerful way of communicating nonverbal information [49, 50]. When face-to-face, we learn better [51].

Commitment is an even more delicate matter with children with special educational needs. Indeed, these pupils often show specific troubles associated with difficulties in focusing their attention or with an actual attention disorder. A preliminary question arises: Is the learning by teaching approach relevant? Does it allow to decrease the specific learning difficulties? Can the time dedicated to learning be increased for these children?

Is the learning by teaching approach relevant? In a first paper [15], authors wonder about the possibility for children with autism spectrum disorders to work collaboratively such as in a task of learning by teaching. They suggest an experimental setting in which two people (an 11-year-old child with an autism spectrum disorder, and a teacher) are with the conversational robot Ifbot. This robot has a limited number of expressions and does not move neither its arms nor its body. Ifbot is teleoperated and is programmed to make mistakes. The session takes place in two stages, with at first a time of familiarization with the robot (10 minutes), in which the child, the teacher and the robot play a game. Then the session continues with a time of collaborative work (40 minutes). The teacher asks alternatively to the child and to the robot to read a page of a comic book on the History of Japan aloud. Each time the robot makes a mistake, it says: “This question is difficult for me” or “Please teach me the right answer.” It will then answer: “It’s amazing” or “Yes, the answer was right. Very good” and might ask for clarifications: “What is it?”

Results show that (1) when Ifbot makes a mistake, the child says: “That is not correct.” The child explains the robot its mistake quite willingly. To do this, the child comes closer to the robot and (2) if the robot makes too many consecutive mistakes, the child will tend to be a bit more reluctant in correcting it.

In another experiment, these same authors [15] use the same experimental design described above, but this time with two children with an autism spectrum disorder, and a teacher. The collaborative work consists in building, using Lego blocks, a story. During the conception of the story, the robot asks: “What is it?” and “Please explain that?” Each time the child answers the robot, the machine answers: “It is very good.”

The results show that children are engaged in the task and that they explain the story to the robot. Thus, children with an autism spectrum disorder communicate with the robot. They are able to teach it and can explain the meaning of correct answers. We can conclude that the learning by teaching approach with a humanoid robot is perfectly relevant to pupils with specific educational needs. It is now up to us to see if it could be possible to increase the time these pupils dedicate to learning.

Results on learning of how to write show that these pupils with special educational needs are able to remain committed to a task during the entire length of the sessions: over two weeks, each child participated to a three hours long session [7]. In a case study [7], the authors show that a participant sent, months later, a letter to NAO to ask how far it had progressed in its learning. These pupils were able to evaluate the performance of NAO not only positively but also negatively. Thus, they show their ability as teachers, and this even while being very young (5 years old and 5 months). If these results are promising, it should not be dismissed that an adult (experimenter or therapist) was also with the child during the interaction

To answer the question of the increase of the time dedicated to learning in children with potential symptoms of a developmental disability, authors [16] once again use Ifbot. Learning sessions were distributed across three weeks, but their actual duration is not reported. Children were distributed between three groups: a learning by teaching group (G1), a group where the child is alone (G2), and a group where children learn together. Effective learning time during sessions is measured. They show that (1) learning time in G1 is higher than in the two other groups (G2 and G3). In particular, there was a significant difference between group G1 (learning by teaching) and group G2 (child learning alone); (2) the duration of children teaching is higher in group G1 (learning by teaching) than in group G3 (children learning together); and (3) children consider the collaborative experience of teaching the robot quite enjoyable.

3.5.2. Motivation and Self-Confidence

The learning by teaching paradigm is known to increase motivation and self-confidence, and this independently from educational contents [48]. For pupils with specific educational needs, learning can only be done with a strong motivation and great self-confidence. It is by using the concept of the “Protégé Effect” [13] that authors [30] have designed an experimental setup allowing to motivate pupils with specific educational needs with long-term writing learning, despite the fact that they encounter difficulties in this area.

The experimental setup is as follows: NAO and the child both have a tactile tablet with a customized program. This program responds to the specific needs of the child (his or her drawing errors). The child draws and then moves the tablet towards NAO. NAO then simulates drawing on the tablet. The child gets the tablet back and corrects the drawing and presses a button to give it to NAO. NAO then simulates a new drawing, and so on. In order to build up the conversation with the child, NAO asks: “What do you think of it?” Depending on the abilities of the child, a basic scenario can be used for contextualization (NAO needs to know how to write because it tries to communicated with another NAO). To evaluate the robot, two buttons are offered: a red one, with a thumb pointing down for negative evaluations, and a green one with a thumb pointing up for positive evaluations.

Between three and four sessions of 60 minutes have been suggested, one per week to (1) a child with a strong inhibition and a weak motivation for writing; (2) a child with visuoconstructive deficits and focusing difficulties under the care of an occupational therapist; and (3) a group of height children with difficulties to lean cursive writing of different magnitudes. Some are under the care of occupational therapists, others of neurologists.

The results indicated that (1) the number of corrections increases with the sessions even though the drawings that must be realized get more and more complex for the child with writing difficulties and strong inhibition. The child spends more time correcting the robot and even creates an affective relationship with the robot (cries during the last session). The child even sent letters a month after the last session to inquire about the progress of NAO. The parents have reported noticing an evolution in the quality of their child’s writing skill; (2) the child with visuoconstructive defects and attentional difficulties remained engaged in the task more than 40 minutes each of the 4 sessions. The child’s progress is massive (13 out of 26 during the first session, 26 out of 29 in the last one); and (3) all children remained in the activity during all sessions. They gave more positive evaluations to NAO (99) than they gave negative ones (33).

3.5.3. Mental States Attribution

The incidence of learning by teaching on mental states attribution has also been explored. Studies have shown that children 10-11 years old and also preteenagers (13-14 years old) consider a teachable agent as a social entity through the attribution of mental states and responsibility [13]. With younger children (5 to 9 years old), Lindberg et al. [29] observe that mental states attribution to a social robot is similar to a teachable agent. The authors notice wide between-subject variations regarding the recognition of the robot’s intelligence. In contrast, the teachable agent received a higher score in anthropomorphism than the social robot [29].

4. Characteristics of the Learning by Teaching Paradigm

After briefly recalling the historical context of the learning by teaching approach, we will extract from the literature the specificities attributed to the humanoid robot, then we will do the same regarding what is asked of the tutor (the child).

4.1. Brief Historical Context

After Seneca, the idea that it is by teaching that we learn will be used again in the century, in England, with the “monitorial-system.” Facing a lack of teachers, some children or adults are asked, after taking into consideration their abilities, to teach their peer. A similar concept can be found in France with schools of mutual teaching at the beginning of the century [52], and later in the Freinet pedagogy, or of Montessori. Starting in the eighties, Martin, professor in language didactics, used the approach of “Lernen durch Lehren” to develop the teaching of the French language to German students. This educational method will then widely spread in France [53–56] then, during recent years, in educational robotics.

4.2. Features of Robot as Tutee

Three main characteristics can be highlighted. The humanoid robot is programmed to make mistakes [15, 16, 56]. The mistakes are never random. They always correspond to specific mistakes observed in the development of the ability, studied for reading [5], for writing [4], and for reasoning [11, 12]. These mistakes decrease during learning and they increase the tutor’s motivation. The humanoid robot must show learning abilities by improving its skills, displaying as a consequence its intelligence [47]. In all the experiments, the humanoid robot displays a strong social dimension. From our point of view, Tanaka et al. [5] bring a real added value by using the possibility of recording in real time the new behavior that was just learned. Furthermore, it allows for the integration in the robot of different kinds of teaching (verbal, gestural, etc.) and thus to answer more accurately the pupils’ various needs. Masson et al. [11, 12] give the humanoid robot a status of ignorant in order to strengthen the position of the child as being the teacher. Other authors use a fictive grading of the robot’s performance to show its progress [26] and in doing so display the efficiency of the teaching offered by the child.

4.3. Features of the Child as Tutor

The specificities of the tutor, or rather what is asked of the child, are relatively less developed in the literature. In the studies on learning of drawing letters, the experimenter asks the child to learn to draw letters in lowercase and in uppercase. The experimenter tells the child that Michael (name of the robot) needs help to learn how to write. The didactic contract is more implicit than it is explicit. Tanaka et al. [5] add to the approach of learning by teaching the framework of CRR. This concept is interesting for it strengthens the position of teacher for the child. In contrast, the child does not receive explicit instructions to teach.

Masson, et al. [11, 12] bring another clear added value by elaborating a clear and explicit didactic contract between the tutee (the robot) and the tutor (the child). The methodology used can be generalized to any form of learning. This contract clarifies some points: the goal targeted, the role of the child, the difference of knowledge between tutor and tutee is quantified, the position of teaching is prepared (the child is alone with the robot and can express themselves without any judgmental attitude).

The goal targeted: the experimenter only gives the child the instruction that they must teach NAO a lot of things.

The function is defined: “The child is NAO’s teacher.” NAO asks the child if they want to be its teacher.

The difference between the knowledge of the child and the knowledge of NAO is at its maximum: “You know a lot of things, and NAO knows nothing.” The use of the binary quantification is easily understandable for the child. The experimenter adds a value judgment: “NAO is really silly.” The difference is quantified through examples given by the experimenter to the child. This strengthens the role of teacher for the child: a teacher, by definition, knows. From the child’s point of view, the situation suggested becomes believable.

The teaching position is also prepared. The experimenter indicates to the child that they will need to answer surprising questions; some might not even make sense. The child should not be surprised to be facing this kind of questions, because otherwise when facing a nonsensical question, the child might be driven to find a meaning that is not really there. Indeed, asking a question that does not make sense violates Grice’s cooperation principle [57]. In the class inclusion task, the child is driven to answer another question than the one that has been initially asked. “Are there more flowers or more asters” implies a comparison between the number of flowers and the number of asters. If we consider that this question is surprising because there are necessarily more in “all” than in one “part,” the child might infer that the real question was to compare numerically the subclass “asters” to the only other subclass “the other flowers,” or “the flower-tulips.” This question then gains meaning. Furthermore, it is at that age that the child learns how to count and learns to compare quantities. Children evaluate each other on these skills. How many times can we hear things such as: “You do not even know how to count.” This, in turn, triggers in response the counting rhyme. Adults themselves to valorize the child can ask this kind of comparison. The fact that the child is alone in front of the robot, that no adult is present in the room, reduces pragmatic interferences and gives this teaching context all of its authenticity.

The fact that the child is alone with the robot strengthens their status of teacher, liberating them from constraints. The child cannot be judged, and their responses are more natural, more spontaneous.

5. Conclusion

To summarize, we will remember of this work the following points.

The learning by teaching approach with a humanoid robot is truly relevant. As we have shown in our review, it allows children to make real progress in various areas of learning such as reading, writing, language, and reasoning but also allows them to improve their tasks commitment and their focus. This approach is just as efficient in very young children (3 years old) as in older children. Its efficiency is also noticeable in children with special educational needs: children with visuoconstructive deficits, with poor handwriting or even poorer self-confidence, or with attention disorders.

If the characteristics of the tutee (the social robot) are well defined in the literature, it is our belief that a more clearly defined description of the didactic contract with the tutor is required. From our point of view, the targeted goal must be explicitly told to the tutor. The child must know (1) that they are asked to teach NAO; (2) that they have the status of the teacher, and in order to make it believable, it should be quantified; (3) the difference of knowledge between the child and NAO; (4) that the position of the teacher should be anticipated so that they will not be surprised by questions that do not make sense; and (5) that a child alone with the robot considerably decreases the weight of pragmatic implicatures and allows the psychologist to record more objective measures.

For psychologists, the learning by teaching approach with a human-humanoid robot interaction is a new experimental paradigm extremely promising. As we have shown, we can identify the genesis of the building of concepts by reducing effects of the pragmatics of language [58]. It is our belief that this new experimental paradigm is very important since it opens the way to a reexamination of the acquisitions of all the great concepts (number conservation, quantity, matter, weight, and volume) that participate in human intelligence (concepts that can remain problematic in teenagers [59]). New research is also incoming with NAO in the study of cognitive bias and their influence on decision-making [60, 61].

Conflicts of Interest

The authors declare that there are no conflicts of interest regarding the publication of this paper.

References

L. Fiorella and R. E. Mayer, “The relative benefits of learning by teaching and teaching expectancy,” Contemporary Educational Psychology, vol. 38, no. 4, pp. 281–288, 2013.

W. J. Mchugh, “Team Learning in Skills Subjects in Intermediate Grades,” Journal of Education, vol. 142, no. 2, pp. 22–51, 2018.

G. Biswas, K. Lee, L. K. Belynne et al., “Incorporating self regulated learning techniques into learning by teaching environments,” in Proceedings of the 26th Annual Meeting of the Cognitive Science Society, pp. 120–125, 2004.

E. Yadollahi, W. Johal, A. Paiva, and P. Dillenbourg, “When deictic gestures in a robot can harm child-robot collaboration,” in Proceedings of the the 17th ACM Conference, pp. 195–206, New York, NY, USA, June 2018.

F. Tanaka and S. Matsuzoe, “Children Teach a Care-Receiving Robot to Promote Their Learning: Field Experiments in a Classroom for Vocabulary Learning,” Journal of Human-Robot Interaction, pp. 78–95, 2012.

D. Hood, S. Lemaignan, and P. Dillenbourg, “When Children Teach a Robot to Write: An Autonomous Teachable Humanoid Which Uses Simulated Handwriting,” in Proceedings of the 10th Annual ACM/IEEE International Conference on Human-Robot Interaction, HRI 2015, pp. 83–90, New York, NY, USA, March 2015.

S. Lemaignan, A. Jacq, D. Hood, F. Garcia, A. Paiva, and P. Dillenbourg, “Learning by Teaching a Robot: The Case of Handwriting,” IEEE Robotics and Automation Magazine, vol. 23, no. 2, pp. 56–66, 2016.

E. S. Kim, D. Leyzberg, K. M. Tsui, and B. Scassellati, “How people talk when teaching a robot,” in Proceedings of the 4th ACM/IEEE International Conference on Human-Robot Interaction, HRI'09, pp. 23–30, USA, March 2009.

W. Johal, J. Kennedy, V. Charisi, H. W. Park, G. Castellano, and P. Dillenbourg, “Robots for Learning - R4L,” in Proceedings of the Companion of the 2018 ACM/IEEE International Conference, pp. 397-398, Chicago, IL, USA, March 2018.

S. Chandra, R. Paradeda, H. Yin, P. Dillenbourg, R. Prada, and A. Paiva, “Affect of Robot's Competencies on Children's Perception,” in Proceedings of the inProceedings of the 16th Conference on Autonomous Agents and MultiAgent Systems (AAMAS '17, pp. 1490–1492, Richland, CA, USA, 2017.

O. Masson, J. Baratgin, F. Jamet, F. Ruggieri, and D. Filatova, “Use a robot to serve experimental psychology: Some examples of methods with children and adults,” in Proceedings of the 2016 International Conference on Information and Digital Technologies, IDT 2016, pp. 190–197, Rzeszo, Poland, July 2016.

O. Masson, J. Baratgin, and F. Jamet, “NAO robot, a social clues transmitter: what impacts? The example with endowment effect,” in Proceedings of the 30th International Conference on Industrial, Engineering, Other Applications of Applied Intelligent Systems (IEA/AIE), pp. 559–568, Université d’Artois, Arras, France, 2017.

C. C. Chase, D. B. Chin, M. A. Oppezzo, and D. L. Schwartz, “Teachable agents and the protégé effect: Increasing the effort towards learning,” Journal of Science Education and Technology, vol. 18, no. 4, pp. 334–352, 2009.

T. Chaffey, H. Kim, E. Nobrega, N. Lubold, and H. Pon-Barry, “Dyadic Stance in Natural Language Communication with a Teachable Robot,” in Proceedings of the Companion of the 2018 ACM/IEEE International Conference, pp. 85-86, Chicago, IL, USA, March 2018.

F. Jimenez, T. Yoshikawa, T. Furuhashi, M. Kanoh, and T. Nakamura, “Feasibility of Collaborative Learning between Robots and Children with Autism Spectrum Disorders,” in Proceedings of the International Workshop on Intervention of Children with Autism Spectrum Disorders using a Humanoid Robot, 2015.

F. Jimenez, T. Yoshikawa, T. Furuhashi, M. Kanoh, and T. Nakamura, “Collaborative learning between robots and children with potential symptoms of a developmental disability,” in Proceedings of the 2017 IEEE Symposium Series on Computational Intelligence (SSCI), pp. 1–5, November 2017.

O. Mubin, C. J. Stevens, S. Shahid, A. A. Mahmud, and J. Dong, “A review of the applicability of robots in education,” Technology for Education and Learning, vol. 1, no. 1, 2013.

A. K. Pandey and R. Gelin, “Humanoid robots in education: A Short Review,” in Humanoid Robotics: A Reference, A. Goswami and P. Vadakkepat, Eds., Springer Science, 2017.

M. Alemi, A. Meghdari, and M. Ghazisaedy, “The Impact of Social Robotics on L2 Learners’ Anxiety and Attitude in English Vocabulary Acquisition,” International Journal of Social Robotics, vol. 7, no. 4, pp. 523–535, 2015.

A. N. Meltzoff, R. Brooks, A. P. Shon, and R. P. N. Rao, “"Social" robots are psychological agents for infants: A test of gaze following,” Neural Networks, vol. 23, no. 8-9, pp. 966–972, 2010.

C.-W. Chang, J.-H. Lee, P.-Y. Chao, C.-Y. Wang, and G.-D. Chen, “Exploring the possibility of using humanoid robots as instructional tools for teaching a second language in primary school,” Journal of Educational Technology & Society, vol. 3, no. 2, pp. 13–24, 2010.

J. Han, M. Jo, V. Jones, and J. Jo, “Comparative Study on the Educational Use of Home Robots for Children,” Journal of Information Processing Systems, vol. 4, no. 4, pp. 159–168, 2008.

J. Choi, J. Lee, and J. Han, “Comparison of Cultural Acceptability for Educational Robots between Europe and Korea,” Journal of Information Processing Systems, vol. 4, no. 3, pp. 97–102, 2008.

. Han, “The Cross-cultural Acceptance of Tutoring Robots with Augmented Reality Services,” International Journal of Digital Content: Technology and its Applications, vol. 3, no. 2, 2009.

Y. C. Lin, T. C. Liu, M. Chang, and S. P. Yeh, “Exploring children’s perceptions of the robots,” Lecture notes in computer science, vol. 5670, pp. 512–517, 2009.

S. Chandra, R. Paradeda, H. Yin, P. Dillenbourg, R. Prada, and A. Paiva, “Do Children Perceive Whether a Robotic Peer is Learning or Not?” in Proceedings of the the 2018 ACM/IEEE International Conference, pp. 41–49, Chicago, IL, USA, March 2018.

W. Johal, A. Jacq, A. Paiva, and P. Dillenbourg, “Child-robot spatial arrangement in a learning by teaching activity,” in Proceedings of the 25th IEEE International Symposium on Robot and Human Interactive Communication, RO-MAN 2016, pp. 533–538, August 2016.

F. Tanaka, K. Isshiki, F. Takahashi, M. Uekusa, R. Sei, and K. Hayashi, “Pepper learns together with children: Development of an educational application,” in Proceedings of the 15th IEEE RAS International Conference on Humanoid Robots, Humanoids 2015, pp. 270–275, Seoul, Republic of Korea, November 2015.

M. Lindberg, K. Månsson, B. Johansson, A. Gulz, and C. Balkenius, “Does a robot tutee increase children’s engagement in a learning-by-teaching situation?” in Intelligent Virtual Agents, J. Beskow, C. Peters, G. Castellano, C. O'Sullivan, I. Leite, and S. Kopp, Eds., Lecture Notes in Computer Science, pp. 243–246, 2017.

A. Jacq, S. Lemaignan, F. Garcia, P. Dillenbourg, and A. Paiva, “Building successful long child-robot interactions in a learning context,” in Proceedings of the 11th Annual ACM/IEEE International Conference on Human-Robot Interaction, HRI 2016, pp. 239–246, March 2016.

T. Belpaeme, J. Kennedy, A. Ramachandran, B. Scassellati, and F. Tanaka, “Social robots for education: A review,” Science Robotics, vol. 3, no. 21, p. eaat5954, 2018.

F. Erich, M. Hirokawa, and K. Suzuki, “A Systematic Literature Review of Experiments in Socially Assistive Robotics using Humanoid Robots,” 2017, https://arxiv.org/abs/1711.05379.

E. Mwangi, M. Diaz, E. Barakova, A. Catala, and M. Rauterberg, “Can children take advantage of nao gaze-based hints during gameplay?” in Proceedings of the 5th International Conference on Human Agent Interaction, HAI 2017, pp. 421–424, October 2017.

N. Akalin, P. Uluer, H. Kose, and G. Ince, “Humanoid robots communication with participants using sign language: An interaction based sign language game,” in Proceedings of the 2013 IEEE Workshop on Advanced Robotics and Its Social Impacts, ARSO 2013, pp. 181–186, Japan, November 2013.

E. J. G. Van Der Drift, R.-J. Beun, R. Looije, O. A. B. Henkemans, and M. A. Neerincx, “A remote social robot to motivate and support diabetic children in keeping a diary,” in Proceedings of the 9th Annual ACM/IEEE International Conference on Human-Robot Interaction, HRI 2014, pp. 463–470, March 2014.

K. S. Goodman, “Analysis of Oral Reading Miscues: Applied Psycholinguistics,” Reading Research Quarterly, vol. 5, no. 1, p. 9, 1969.

K. S. Goodman, “Miscue analysis: Applications to reading instruction,” ERIC Clearinghouse on Reading and Communication Skills, 1973.

J. Hattie and H. Timperley, “The power of feedback,” Review of Educational Research, vol. 77, no. 1, pp. 81–112, 2007.

J. Bitchener, “Evidence in support of written corrective feedback,” Journal of Second Language Writing, vol. 17, no. 2, pp. 102–118, 2008.

S. Graham, K. R. Harris, L. Mason, B. Fink-Chorzempa, S. Moran, and B. Saddler, “How do primary grade teachers teach handwriting? A national survey,” Reading and Writing, vol. 21, no. 1-2, pp. 49–69, 2008.

S. Graham, N. Weintraub, and V. Berninger, “Which manuscript letters do primary grade children write legibly?” Journal of Educational Psychology, vol. 93, no. 3, pp. 488–497, 2001.

F. Tanaka, A. Cicourel, and J. R. Movellan, “Socialization between toddlers and robots at an early childhood education center,” Proceedings of the National Acadamy of Sciences of the United States of America, vol. 104, no. 46, pp. 17954–17958, 2007.

J. J. Asher, “The Total Physical Response Approach to Second Language Learning,” The Modern Language Journal, vol. 53, no. 1, pp. 3–17, 1969.

J. Piaget, “La notion de partie chez lenfant , [The notion of part in children],” Journal de psychologie normale et pathologique, vol. 18, no. 6, pp. 359–480, 1921.

J. Piaget and A. Szeminska, La genèse du nombre chez l’enfant [The origin of number in children], Delachaux et Niestlée, Neuchâtel, Switzerland, 1941.

G. Politzer, “The class inclusion question: a case study in applying pragmatics to the experimental study of cognition,” SpringerPlus, vol. 5, no. 1, 2016.

J. D. Lane, H. M. Wellman, and S. A. Gelman, “Informants' traits weigh heavily in young children's trust in testimony and in their epistemic inferences,” Child Development, vol. 84, no. 4, pp. 1253–1268, 2013.

C. A. Rohrbeck, M. D. Ginsburg-Block, J. W. Fantuzzo, and T. R. Miller, “Peer-assisted learning interventions with elementary school students: A meta-analytic review,” Journal of Educational Psychology, vol. 95, no. 2, pp. 240–257, 2003.

L. Takayama and C. Pantofaru, “Influences on proxemic behaviors in human-robot interaction,” in Proceedings of the 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2009, pp. 5495–5502, USA, October 2009.

A. Kristoffersson, K. Severinson Eklundh, and A. Loutfi, “Measuring the Quality of Interaction in Mobile Robotic Telepresence: A Pilot's Perspective,” International Journal of Social Robotics, vol. 5, no. 1, pp. 89–101, 2013.

D. Leyzberg, S. Spaulding, and B. Scassellati, “Personalizing robot tutors to individuals' learning differences,” in Proceedings of the 9th Annual ACM/IEEE International Conference on Human-Robot Interaction, HRI 2014, pp. 423–430, Germany, March 2014.

P. Lesage, “La pédagogie dans les écoles mutuelles au XIXe siècle,” Revue française de pédagogie, vol. 31, no. 1, pp. 62–70, 1975.

J.-P. Martin, “Quand les élèves font la classe” [When children teach],” Franτais dans le monde, no. 224, pp. 51–55, 1989.

A. Renkl, Lernen durch Lehren, Deutscher Universitätsverlag, Wiesbaden, 1997.

J.-P. Martin, “Treibhäuser der Zukunft, Wie in Deutschland Schulen gelingen [How German schools succeed],” eine Dokumentation von Reinhard Kahl und der Deutschen Kinderund Jugendstiftung, 2004.

L. Pareto and R. A. Tutee, Robotics in Education, Springer International Publishing, 2017.

B. Jacquet, J. Baratgin, and F. Jamet, “The Gricean Maxims of Quantity and of Relation in the Turing Test,” in Proceedings of the 2018 11th International Conference on Human System Interaction (HSI), pp. 332–338, Gdansk, July 2018.

B. Jacquet, O. Masson, F. Jamet, and J. Baratgin, “On the Lack of Pragmatic Processing in Artificial Conversational Agents,” in Human Systems Engineering and Design, vol. 876 of Advances in Intelligent Systems and Computing, pp. 394–399, Springer International Publishing, Cham, 2019.

F. Jamet, J. Baratgin, and D. Filatova, “Global warming and sea level rise: the intellect development study of preadolescents and adolescents from 11 to 15 years old,” Studia Pedagogiczne, vol. 24, pp. 361–380, 2014.

O. Masson, J. Baratgin, and F. Jamet, “NAO robot and the "endowment effect",” in Proceedings of the IEEE International Workshop on Advanced Robotics and its Social Impacts, ARSO 2015, Lyon, France, July 2015.

O. Masson, J. Baratgin, and F. Jamet, “NAO robot as experimenter: Social cues emitter and neutralizer to bring new results in experimental psychology,” in Proceedings of the 2017 International Conference on Information and Digital Technologies, IDT 2017, pp. 256–264, Slovakia, July 2017.