Recent studies have reported that the inclusion of new technological elements such as augmented reality (AR), for educational purposes, increases the learning interest and motivation of students. However, developing AR applications, especially with mobile content, is still a rather technical subject; thus the dissemination of the technology in the classroom has been rather limited. This paper presents a new software architecture for AR application development based on freely available components; it provides a detailed view of the subsystems and tasks that encompass the creation of a mobile AR application. The typical task of plotting a quadratic equation was selected as a case study to obtain feasibility insights on how AR could support the teaching-learning process and to observe the student’s reaction to the technology and the particular application. The pilot study was conducted with 59 students at a Mexican undergraduate school. A questionnaire was created in order to obtain information about the students’ experience using the AR application and the analysis of the results obtained is presented. The comments expressed by the users after the AR experience are positive, supporting the premise that AR can be, in the future, a valuable complimentary teaching tool for topics that benefit from contextual learning experience and multipoint visualization, such as the quadratic equation.
Even though the first world countries are implementing different technological approaches to encourage the teaching-learning process to be more attractive, motivating, and meaningful to the students [
In their 2011 second-order meta-analysis and validation study, Tamim et al. [
One of the technologies that both researchers and professionals have striven to bring into the classroom is augmented reality (AR). The main idea is to exploit the ability offered by the AR to enhance the user senses by manipulating virtual objects superimposed on top of the real world scenes [
As it was shown by Kerawalla et al. [
Several researchers such as Chang et al. [
Particularly, the teaching-learning process in Mexican schools still relies mostly on students taking notes during lectures, following examples on whiteboards, and having textbook homework. One of the problems with this approach, for example, in mathematics, is having to memorize and master step-by-step arithmetic algorithms instead of focusing on conceptual understanding [
After several research works, it was detected that one of the main obstacles to insert AR as a tool in classrooms is the lack of programming and 3D modeling skills that the teachers outside the computing area often have, as well as the limited number of authoring systems to easily and rapidly deploy the applications [
This paper presents a software architecture based on freely available components, as well as a detailed view of the stages associated with creating an AR application and the final user experience. The plotting of a quadratic equation and its correspondent coefficient manipulation was selected as a case study in order to obtain insights on how AR could support the teaching-learning process and the student’s reaction to the technology and application. The created AR application allows the user to graph and interact with the quadratic equation coefficients. The aim of the study is not only to provide a glimpse of what AR technology could bring into the classrooms in the near future but also to offer a framework that can be used to create compelling experiences; then when implemented effectively, it would allow AR to become a tool in the teaching-learning process for mathematics courses in any curricula.
The rest of the paper is organized as follows. In Section
According to the study made by Martin et al. [
In 2006, Kerawalla et al. [
A pilot study made in 2011 by Mejías Borrero and Andújar Márquez [
The work presented by Billinghurst and Dünser [
Based on the flow theory proposed by Mihaly [
The survey conducted by Wu et al. [
In the work of di Serio et al. in 2013 [
Three AR learning environments used in genuine classrooms contexts were presented in 2013 by Cuendet et al. [
The work presented by Cheng and Tsai [
Given the nature of the proposed architecture and experimental application in this paper, a summary of works regarding image based AR for teaching-learning is shown in Table
A summary of 15 studies using AR for teaching-learning purposes.
Author | Year | Samples | Topic | Level | Technology |
---|---|---|---|---|---|
Kaufmann [ |
2004 | 2 | Geometry | High school | HDM |
Kerawalla et al. [ |
2006 | 133 | Geoscience | Elementary | Webcam |
Rosenbaum et al. [ |
2007 | 21 | Medical science | High school | Mobile device |
Dunleavy et al. [ |
2009 | 12 | Mathematics | High school | Mobile device |
Andújar et al. [ |
2011 | 36 | Remote laboratory | Undergraduate | Webcam |
Mejías Borrero and Andújar Márquez [ |
2012 | 20 | Digital systems | High school | 3D glasses |
Enyedy et al. [ |
2012 | 43 | Physics | Elementary | Camera |
Billinghurst and Dünser [ |
2012 | 10 | Storytelling | Elementary | Books |
Bressler and Bodzin [ |
2013 | 68 | Forensics | High school | Mobile device |
di Serio et al. [ |
2013 | 69 | Art | Middle school | Webcam |
Cuendet et al. [ |
2013 | 16 | Logistics | High school | Camera |
Sommerauer and Müller [ |
2014 | 101 | Mathematics | Math exhibition | Mobile device |
Cai et al. [ |
2014 | 29 | Chemistry | High school | Webcam |
Ibáñez et al. [ |
2014 | 64 | Electromagnetism | High school | Webcam |
Our study | 2014 | 59 | Quadratic equation | Undergraduate | Mobile device |
By the above, this paper presents both a detailed example of an AR application created following the proposed framework and the result of a usage experience in a Mexican classroom.
Frequently, the AR applications are developed using an authoring system that allows building the educational content. However, the commercial tools do not offer a way to easily develop the AR educational content. Typically, AR systems may be quite different in detail, but all of them share a common basic architecture.
Furthermore, there are fundamental components and subsystems that can be found in most of AR applications such as tracking the user’s position, mixing real and virtual objects, processing, and reacting to context changes and user interactions [
In this paper, the first stage consists in developing a simple, practical, and illustrative AR mobile application for plotting a quadratic equation. The main requirements to create the application are the following: (a) it needs to run on devices with android operating system (AOS), (b) offers an alternative tool for undergraduate students to understand the concept of a quadratic equation, (c) graphically shows what happens when the coefficients of the quadratic equation change, and (d) creates an easy to use framework for agile marker based AR. The application was called “pARabola plotting,” writing the letters “A” and “R” in upper case to refer to the AR concept.
The general structure shown in Figure
Proposed marker based mobile AR framework. (a) The proposed global subsystems and (b) tools and components used to create each subsystem.
Simplified framework class view.
The presentation subsystem is responsible for displaying video output of the real world and rendering the 3D augmentations for the user. After exploring and evaluating different tools such as Director, OpenSceneGraph, OGRE, D’Fusion, Papervision3D, and native OpenGL, Unity3D was selected for this task. Unity3D offers rendering capabilities, multiple built-in shaders and effects, a physics engine, a collider detector, and a particle generator that can be used to plot equations in 2D/3D space.
The built-in particle generator is an important aspect of this subsystem because the particles emitted can be organized in such way to follow a predefined path described by a quadratic function. Thus, it is used as the main plotting component. An example of using the particle system to plot a parabola is shown in Figure
Example of using the static position of the particle system to create the pARabola.
The main goal of the world model subsystem is to store and provide access to a digital representation of the real world; this includes the fiducial marker patterns, data about interest points, and the 3D objects that are going to be used in the augmentation. This subsystem is not a standalone component inside the framework; it represents the link between classes from the tracking library and the visual scene editor. The former knows which marker patterns to look for and how to interpret them, while the latter acts as hierarchical container for all the blueprints of the components inside the virtual scene ready to be used by another subsystem inside the main application loop.
When a marker appears inside a scene, it is associated with a plotting prefab to indicate which object should be displayed when the marker is detected on the scene. For this particular case, the object displayed is the graph of the parabola associated with the parameters selected by the users. The marker used for the application is shown in Figure
The marker used for the pARabola plotting application.
Context subsystem is responsible for providing the entire system with contextual information about status; it can be either static or real time information as the application is running. This includes tracking information, although generally this task is delegated to the tracking subsystem. The framework uses scripts attached to empty objects inside Unity3D. This is made by implementing the blackboard pattern where objects can write information to it and can be read by any other object that needs it.
The tracking subsystem is a key component on any AR system; it is responsible for thresholding, filtering, marker detection, and pose estimation. All the information obtained is relayed to the rendering system to add the virtual elements and compose the final scene that will be viewed by the user. In order to maintain the illusion that the real and virtual worlds coexist as one, artifacts and jitter induced by the tracking library should be kept to a minimum without sacrificing response time.
While some of the existing libraries are capable of robust real time marker tracking, they also have some limitations such as cost, platform support, and cumbersome implementation. This drawback was solved using the Vuforia SDK which is a well-developed, stable, documented, and supported framework created by Qualcomm. With the modified particle emitter prefab created, the second step was bringing AR into the mix to do it; Qualcomm’s Vuforia Unity plug-in was used. A new Unity3D project was created with an empty scene; additionally an AR camera object and its associated scripts for initialization were added. At this stage, it is necessary that every frame captured was relayed to the tracking component and ensures the finalization of the capture.
Before reaching the tracking subsystem, Vuforia converts every frame captured by the camera to a suitable format for OpenGL ES rendering and for the tracking operations. The tracking subsystem contains the computer vision algorithms that detect and track real world objects inside the video frames captured by the camera. As a result, the tracking component stores the information obtained in a state object (part of the context subsystem) used by the renderer module to ensure a correct positioning and representation of the 3D object in the video feed of the real world. The tracking-update loop will be executed for every frame that is going to be processed.
The interaction subsystem gathers and processes any input that the user makes deliberately. Many different types of interfaces for AR have been developed over the years, each with its own advantages and limitations. Based on the categorization presented by [
The pose estimation process was handled by the Trackable-Behaviour Class from Vuforia and then relayed to Unity3D in order to change the transformation data for the entity. In order to respond to user inputs, a custom class was implemented and attached to the camera object to ensure that the GUI is always visible to the user and ready to relay information to the presentation subsystem when it detects user interaction with the device.
A tap on the screen will trigger a chain of events to evaluate if any GUI element was touched, verify the presence of a valid marker inside the scene, validate any changes to the exposed properties, and, finally, invoke the corresponding method to update the entity with the new information.
Part of this subsystem was implemented in the form of the C# classes accompanying the plotting prefab and the complementing GUI script responsible for drawing the user controls on the device screen and responding to the interaction. All of this was attached to the AR camera. Thus, the final tracking-interaction-update loop was completed. For every frame, the camera is active on the application, the GUI script will get called and listen for any user interaction, and if a slider is touched and its value changed, it will send a message to the script inside the plotting prefab in order to reposition the particles.
The tracker component will analyze the content and update the state object for the renderer. The renderer will update the scene reflecting both the new pose of the 3D element according to the marker position and any changes that the model may have experienced.
The particle system is capable of rendering individual particles at predefined coordinates, but in order to be able to response to user interaction (e.g., coefficient changes), several objects were created. In order to do this, 2 parameters which regulate the scale and resolution of the plot (2 left and bottom sliders of Figure
The pARabola plotting application running inside Unity3D.
The final application was meant to showcase the technology, yield an AR experience for the students, and serve as visual aid on the quadratic equation topic; the user is free to interact with the GUI sliders to easily replot the equation in real time as the coefficients change, move around the plot to see it from different perspectives, interact in the same physical space with fellow students, and repeat any actions as many times as needed.
Up to this point, the application was ready to be deployed as a single user standalone package (apk file) to any device running Android 2.2 OS and upper, with at least an ARMv6 + VFP CPU. A video demo of the application running on a HTC One S phone has been uploaded to the web (
After reading through the literature and based on our own experience as teachers for different subjects, we believe that the students will find the AR content to be enjoyable and useful and that this technology can become a complementary tool of current classroom dynamics.
The study began with an investigation about the topics in which professors at undergraduate level have more complications in explaining the basic concepts. After performing several interviews with professors, it was detected that mathematics concepts were typically difficult to explain to the students. Due to this finding, the interviews were particularized with mathematics and physics professors.
The analysis of the interviews allows us to detect that, frequently, the topic of quadratic equations is difficult to explain. This is because of the constant necessity of redrawing the plot on the blackboard to show a particular behavior of the parabola when one of its parameters is changed. Additionally, as a result of the interviews, the relevant information about the elements involved with the quadratic equation topic was collected. As a result of the data collection, the pARabola plotting application was designed. Additionally, an instrument with questions about interaction and application content was created to be applied in the classroom with the students.
In elementary algebra, a quadratic equation is any equation having the form
The trial was conducted with the collaboration of three groups of Mexican undergraduate students at the Engineering and Technology Institute of Ciudad Juárez Autonomous University. As previously stated, the scope of this paper is presenting a first approach and assessment of AR technology usage in undergraduate classrooms; thus only a posttesting evaluation is presented.
Two sessions with each group of students were necessary to prepare the students and to obtain the overall data. The first session involved the professor offering the students a typical classroom session and explaining the elements involved in the quadratic equations using textbooks, slides, and whiteboard examples. After that, the second session began with an explanation offered to the students about the use of pARabola plotting application made by the authors of this research. The explanation included information about the inclination of the tablet in front of the marker to detect and display the plot, the functionality of the sliders, and the sections of the questionnaire. After that, the students explore and test the use of the pARabola application.
The students were encouraged to speak up while using the application to collect qualitative responses based on the “thinking aloud” technique, which is a method used to gather data in usability testing in product design and development [
In order to test the pARabola application two tablets and its correspondent markers were provided for the students. Two students would simultaneously test the application; after that, another two students would do the same; this process was repeated until all the students of the class had a chance to test the application. The average time spent by each student was 7 minutes. AR was used by the students to try to enrich the knowledge acquired, during the previous session with their teachers, about quadratic equations. The data collection took place at the Engineering and Technology Institute of Ciudad Juárez Autonomous University during the first week of May 2014.
The pilot study was performed with three different Mexican undergraduate students groups with three different professors. The students belong to several engineering academic programs. The sample application was tested after the students received the typical information offered by the teacher about quadratic equations.
A total of 59 students from different levels and academic programs participated in the study: 27 in the first group, 22 in the second, and 10 in the third. All of the participants stated owning a smart phone and feeling comfortable using mobile apps. The students were informed of the purpose of the research and their right to withdraw at any moment without any reprisal. Informed consent was obtained from every participant. An example of the students performing the proposed experiment and using the tablet device pointing to the marker is shown in Figure
Students performing the pilot study with the pARabola plotting application.
An explanation of the findings obtained after the use of pARabola application and application of the questionnaire are explained in the following subsections.
The questionnaire designed with the information obtained after the interviews with the teachers is shown in the following. It consists of 26 questions divided into six parts.
Questionnaire to obtain information about incorporating AR technologies as a complementary teaching tool.
| |||||
|
|||||
Age | Gender | □ (Male) | □ (Female) | ||
|
|||||
In which semester are you currently enrolled? | What program of study are you currently enrolled in? | ||||
|
|||||
| |||||
|
|||||
(1) Do you know about the quadratic equation? | □ (Yes) | □ (No) | |||
(2) The main features of a parabola are focus, directrix, axis of symmetry, and vertex? | □ (Yes) | □ (No) | |||
(3) The general equation for a parabola that opens up or down to |
□ (Yes) | □ (No) | |||
(4) Is the vertex of the following parabola located at the top with respect to the vertex defined by |
□ (Yes) | □ (No) | |||
(5) In the following graphic, is the value of |
□ (Yes) | □ (No) | |||
(6) The following parabola plot corresponds to |
□ (Yes) | □ (No) | |||
|
|||||
| |||||
|
|||||
(1) The “pARabola plotting” application has shown all the elements explained by the professor concerning the quadratic equation? | □ (Yes) | □ (No) | |||
(2) The results shown by the application were correct? | □ (Yes) | □ (No) | |||
(3) Do you consider that a virtual environment represents in a better way the concepts explained by the professor? | □ (Yes) | □ (No) | |||
(4) Do you consider that, by using the application, you will remember the concepts easily and in a better way? | □ (Yes) | □ (No) | |||
|
|||||
| |||||
|
|||||
Strongly disagree | Disagree | Undecided | Agree | Strongly agree | |
|
|||||
(1) Do you consider the interface of the application attractive? | 1 | 2 | 3 | 4 | 5 |
(2) The color used in the application allows you to see in an adequate way the parabola graphic? | 1 | 2 | 3 | 4 | 5 |
(3) The numbers and the letters presented in the application were legible? | 1 | 2 | 3 | 4 | 5 |
(4) The size of the buttons allows the correct manipulation of the application? | 1 | 2 | 3 | 4 | 5 |
|
|||||
| |||||
|
|||||
Strongly disagree | Disagree | Undecided | Agree | Strongly agree | |
|
|||||
(1) Do you consider the application intuitive? | 1 | 2 | 3 | 4 | 5 |
(2) The application allows detecting the marker in a fast way? | 1 | 2 | 3 | 4 | 5 |
(3) Are you able to quickly locate the image of a parabola inside the screen of the application? | 1 | 2 | 3 | 4 | 5 |
(4) The task of controls manipulation was simple to execute? | 1 | 2 | 3 | 4 | 5 |
|
|||||
| |||||
|
|||||
(1) Had you ever used an augmented reality didactic system? If yes, offer examples. | |||||
(2) Which other topics would you be interested to implement with AR? | |||||
(3) Your augmented reality experience was satisfactory? | |||||
(4) Please specify if any of the following external factors influenced the performance and handling of the application. | |||||
|
|||||
General comments: | |||||
Thank you!!! |
Part A of the instrument was created in order to obtain general information related to the gender, age, academic program, and semester of the students.
From the total sample size
The distribution of the academic programs of the participants corresponds to the following information: aeronautical engineering 11.86% (
This section of the instrument was created to serve as a quick diagnostic. The goal was to discover if the students understood the concepts related to the quadratic equation as taught by the teacher and experienced through the AR application. Despite the fact that the students are at an undergraduate level and the topic of quadratic equations should not be new to them, it was reported that 53 students respond yes to question one about the previous knowledge of the parabola equation and 6 students answered no.
In regard to the parts of the parabola (question 2), 47 students mentioned that they could perfectly identify each part, 11 said no, and 1 student did not respond. With respect to question three, about the general form of the equation that opens to
Referring to question five, 19 students answered yes, 33 answered no, and 7 did not respond. On the final question, 30 students said yes, 25 said no, and 4 did not answer.
In conclusion, more than 65% of the students can perfectly identify the parts of the parabola and its correspondent equation and are familiar with graphic representations. Albeit a higher percentage would be assumed for this sample, the results invite an in-depth comparison study to weigh in the affordance provided by AR as a complementary tool.
This set of four questions was created in order to familiarize the participants with the environment of the pARabola application. All four questions are of dichotomous type.
By referring to question one, 50 students indicated that the 3D model through the parabola AR displayed all the items they had seen in their class with the professor, 8 students mentioned that the application does not show the items, and one student did not answer. For question two, 100% of students indicated that, upon using the application, the result obtained was as expected.
In regard to whether the virtual environment represents in a better way the concepts previously explained by the professor, 53 students indicated yes; the rest indicated a negative response. Finally, 54 students indicated that application helps them remember the parts of a parabola and its plot more easily; the other 5 students responded no.
With the partial results obtained with Part C of the questionnaire, the AR experience begins to show insight on how the applications developed following the proposed architectural model could be tailored to be used in a teaching-learning environment.
This part of the questionnaire is composed of four questions with a Likert scale. A Likert scale is a psychometric scale commonly used for scaling in survey research. The goal of the questions in this section is to obtain information about the level of agreement or disagreement with the design of pARabola application. Five ordered response levels were used, where number five corresponds to strongly agree and number one to strongly disagree.
In Table
Descriptive statistics for the questionnaire Part D items.
Question | 1 | 2 | 3 | 4 | 5 |
---|---|---|---|---|---|
1 | 2 | 1 | 12 | 24 | 20 |
2 | 0 | 0 | 14 | 13 | 32 |
3 | 1 | 2 | 8 | 21 | 27 |
4 | 0 | 5 | 3 | 18 | 33 |
As can be seen from Table
This part of the questionnaire is composed of 4 questions with a Likert scale type. The questions try to measure the level of agreement or disagreement about the functionality of the pARabola application. In Table
Descriptive statistics for the questionnaire Part E items.
Question | 1 | 2 | 3 | 4 | 5 |
---|---|---|---|---|---|
1 | 1 | 1 | 5 | 26 | 26 |
2 | 0 | 1 | 1 | 21 | 36 |
3 | 0 | 1 | 6 | 14 | 38 |
4 | 0 | 2 | 9 | 13 | 35 |
As it was shown in Table
Part F includes 3 open questions of polytomous type and one with multiple options. The questions were created to obtain information about the AR experience and the external factors which influenced the handling of the application presented.
General comments obtained from the students stated that more than 85% had never used augmented reality. The rest explains that their previous interactions with AR were using commercial applications such Aurasma or Wikitude. The results obtained for question 2 were very varied, but one of the repeated answers was artificial intelligence, programming, circuit design, physics, and other mathematical issues like derivation and integration.
For question 3 about the index of satisfaction with the augmented reality experience 58 students are satisfied with the application while one student does not like the application. The comments obtained by the students were the following. “My experience was satisfactory, because the subject of quadratic equations was simple to understand by using the pARabola plotting app.” “The experience was really good; the idea is creative and helps for the better visualization and understanding.” “The application is very innovative and interesting; it has a very intuitive interface.” “With the use of the application, I can handle directly the parabola which cannot be done easily in my notebook.” “For me was a very good experience, cause “With the application I can better understand the concepts explained previously by the professor.” “I really do not like the application, it was difficult to handle.”
With respect to question four, more than 90% of the students mark the four items, which explains that the use of the application was generally good and the environment was adequate. Finally, with respect to general comments, the most frequent suggestion was to change the way to manipulate the values of the equation. The recommendation was to add text boxes to type in the required input instead of the sliders.
After an analysis of the results, several issues were detected. First of all, almost all the students consider the use of the pARabola application a very good way to understand the concept of quadratic equations. The use of the application in the classroom creates an adequate environment for students to increase the enthusiasm to obtain new knowledge.
Since today’s generation of students is much familiarized with the smartphones and in general with Web enabled devices, it can be easy to extend the process of learning to those devices. The capability offered by AR to observe, in an interactively way, the educational content in real time gives the students a richer, different, and more real experience. The students will be engaged and motivated to discover issues and apply them to the real world from diverse points of view that could never be implemented in the real classroom environment.
Even though the survey was administered to students only, some teacher comments were also obtained during the session of using the pARabola application. The professors argue that using the application is easier to explain what happens with the plot if a parameter is changed and avoids the difficulty of redrawing the graph on the whiteboard several times when a parameter is changed.
By all of the above, the study concludes with the argument that using AR in the classroom was a very good experience for the students and for the professors too. However, there are still many obstacles and challenges to be solved in order to easily insert AR technology to the Mexican classrooms.
One of the main obstacles is the lack of programming and 3D models skills that often the teachers have outside the computing area. Because of this, it is imperative, in the future, to work on designing tools to easily build AR environments. This could help professors easily prepare the AR content.
As can be seen in the results of the current study and the success other authors have had using AR as a mean to further enhance learning, we believe that once the framework is completely implemented it can be extended to many other areas aside from mathematical problem visualization, particularly areas that can benefit from contextual information, spatial exploration, and multimedia intensive domains. The proposed framework can become a powerful tool for students and teachers alike; students can use the AR content as a way to practice and review concepts outside the classroom on their own schedule, while teachers can create more engaging and meaningful material. The implementation of AR content can be beneficial for other mathematical topics such as integrals, derivatives, and geometry. Moreover, the framework proposed can be used in other subjects such as chemistry to visually explain a chemical reaction, biology to explore the heart and its main components, electronics to see what the components are doing, and so forth.
At the end of this pilot study we detected that the main challenge for Mexican institutions is trying to introduce in its educational and teaching model the AR technology as a tool to use inside the classroom. This will have some implications such as covering the costs to create and maintain AR solutions (which typically are expensive), reconfiguring the classroom installations to promote the collaborative work, the resistance to the use of new technologies, and the cost of training the professors to use the AR educational contents, to only name a few.
In this paper a new framework approach to create a mobile augmented reality application was presented. The ability of the framework was tested with the so-called pARabola plotting used to explain the concept of a quadratic function to Mexican undergraduate students. Each stage of the framework was explained in detail to finally show how to create the complete application to interact with a graph of a quadratic equation. After the presentation of the framework it was shown that it is possible to create an application from the ground up; in addition, having prefab objects with encapsulated functionality makes the creating process accessible to an even wider audience.
The application was used to observe the impact of an AR experience in Mexican undergraduate students. To aid with this task, a questionnaire was designed as a result of different interviews with several mathematics and physics professors. The results obtained with the application of the questionnaire, about the use of AR technology, show that the use of AR can help enhance the teaching-learning process in Mexican classrooms and motivates the students and can be an alternative technology to revolutionize the learning paradigm in the future. These two types of findings typically are not displayed simultaneously in AR papers, and particularly any indication of this type of work has been reported in Mexico.
There is still work to be done to enhance the current framework, such as including the AR collaborative feature, which allows multiple devices to intuitively discover and connect with each other to share information and interact with scene elements, and creating a more natural and compelling environment for the students. Additionally, in the future, a strategy to detect which learning-teaching activities can be implemented using AR will be designed.
The authors declare that there is no conflict of interests regarding the publication of this paper.
The authors would like to thank Professor Ivonne Robledo for their valuable help in designing the items of the questionnaire.