An Education Application for Teaching Robot Arm Manipulator Concepts Using Augmented Reality

Teaching robotics is a challenge inmany universities due to themathematics concepts used in this area. In recent years, augmented reality has improved learning in several engineering areas. In this paper, a platform for teaching robotic arm manipulation concepts is presented./e system includes a homemade robotic arm, a control system, and the RAR@pp./e RAR@pp is focused on learning robotic arm manipulation algorithms by the detection of markers in the robotic arm and displaying in real time the values based on the data obtained by the control system. Details on the design of the platform are presented, and the related results are discussed. Experimental data about the usability of the application are also shown.


Introduction
In the last decade, Information Technologies and Communication (ITC) have spread to all areas of the society.An example of this is the area of learning.e constant changes in the traditional methodology of teaching seek more productive methods, with the goal to improve the learning experience and increase the intellectual level of the students.Currently, the use of mobile devices has continuously been increased.e growth in the demand for cell phones and tablets is due to the evolution of technologies and the implementation of many functionalities.One of the new technologies in development is augmented reality (AR).
is technology allows the coexistence of real and virtual objects in the same environment.
Among the different qualities of reality, technology increased its adaptability to a high number of scenarios.Augmented reality, due to its portability and usability, may be implemented in various equipment such as personal computers, mobile devices, and smartphones, among others.
Besides, reality technology can be combined with other techniques to enrich the applications, emerging thus a tool to improve the teaching-learning process inside the classroom and laboratories [1].An area with the potential to implement technology augmented reality is that of robotics since there are issues that require a 3D perception [2].Nevertheless, traditional teaching methods make use of material didactic in 2D, which makes it difficult to understand these themes.For this reason, the present work shows the implementation of a robotic platform equipped with augmented reality technology.e purpose is to achieve in an educational setting an understanding of robotics topics which are more interactive.
In this work, a platform for teaching robotic arm manipulation concepts is presented.e system includes a homemade robotic arm, a control system, and the RAR@ pp (Robotics through Augmented Reality Application).
e RAR@pp is focused on learning robotic arm manipulation algorithms by the detection of markers in the robotic arm and the real-time visualization of the angles of each articulation of the homemade robotic arm using augmented reality.
ese angles are obtained from the robot based on the data obtained by the encoders of each motor and transmitted to the RAR@pp using Bluetooth communication.
ese applications allow students to send value to the robotic arm and visualize in real time the response of the arm to the com mands using the RAR@ pp.
is application is specifically designed for mechatronics, information technology, and manufacturing undergraduate programs.
e incorporation of augmented reality technology into the mobile application allows real-time viewing of joint angles through virtual objects corresponding to the real angles in the physical platform.
e robotic platform is programmed with a reference profile using a desktop application.And by means of a Proportional Integral Derivative controller applied to each joint, it is possible that the robot follows the provided trajectory, allowing the joints to reach the required angles.
is paper is structured as follows.In Section 2, a concise review of the state of the art is given.In Section 3, the design and implementation of the proposed system is described.In Section 4, the experimental setup and results are described.Finally, in Section 5, final recalls are established, and future work is outlined.

State of the Art
Nowadays, augmented reality represents a potential solution to problems in several areas such as robotics [3], teaching [4], mobile apps [5], or medicine [6].For instance, Clemente et al. [7] proposed an improvement in sensing feedback of a robotic hand by visual elements.Although other options are typically used as sensors in prostheses such as vibro-or electrotactile stimulations, one advantage of AR implementation is the increase in sensibility resolution and easiness to adapt in the real world.Medical procedures could also be benefited of AR and robotics combination.In [8], a robotic system was proposed for treatment of tumors with the ablation technique.AR implementation helped to increase precision and consistency, which could be reflected in better medical outcomes.Augmented reality can also be used for safety purposes.Quercioli [6] developed a new approach to visualize lasers without any risk to eye damage.e mechanism consists of a modified smartphone camera without the infrared filter and a Google cardboard-type viewer.
e system was successfully implemented to obtain real-time images of a Nd:YAG laser and a Ti:sapphire oscillator, both at near-infrared emission which is widely used in medical procedures.
In [2], a desktop application named Build-A-Robot was developed to help students to understand the forward kinematics of serial robots arms.e app improves the visualization and configuration of a 3D robotic arm in a controlled (virtual) environment.Jara et al. [9] developed an e-learning system based on a robotic platform and a graphical user interface with an integrated AR module.
is platform can be used to control a robot arm through the Internet.e system is capable of planning a path remotely utilizing augmented reality.
In a work oriented to improve students learning, Ibañez et al. [10] developed an electromagnetism experiment using augmented reality (AR).Questions associated with fundamental topics, for example, Coulomb's law, electric field, and Ohm's law, were prepared.A test was carried out in high school students comparing a web-based tool and the AR app in an experimental/control group.Although results showed that the students' outcomes were statistically similar in both cases, the motivation to study was better for the AR app. is factor is also mentioned, along with creativity by Wei at al [4].
is is a very interesting finding since the students' attitude is fundamental during the learning process.Higher education also has a big potential to implement augmented reality as a learning tool.Martín-Gutiérrez et al. [11] developed an app for training students in real electrical labs using AR.One advantage of this work is that it was designed to promote independent as well as collaborative work in laboratories, which could be helpful mainly in engineering environments.
AR has significant growth potential in advanced manufacturing developments; for example, Ni et al. [12] created the haptic robot for programming welding paths.
e main advantages of this prototype were a user-friendly interface and the possibility to improve this work through seam tracking sensors.In another work related to programming, Collett and MacDonald [13], design a system to test AR debugging.
Speed of processing is a main concern in mobile applications.Ruan and Jeong [14] proposed and implemented the substitution of traditional markers that elaborated with ARToolkit [15] for the Quick Response (QR) code in an AR system.A related work of applying QR to AR apps was developed by Kan et al. [16].Barcode has also been used in combination with AR for commercial purposes [17].AR had also been used for broadcasting enhancement as mentioned by Yan and Hu [18] which in general depends on AR display, AR tracking, and robotic AR broadcast.

Proposed System
In this section, the proposed system is described.e system includes three main components: a homemade physical robotic platform, a 2 DoF robotic arm, and the required hardware for moving the robotic arm, including a control module which receives signals from desktop application and sends back the angles obtained by the encoders of the robotic arm.
e second component is the desktop application, focused on allowing students to prototype different control algorithms, and the last one is the RAR@pp, for the visualization of the angles in real time.
e dataflow of the proposed system is shown in Figure 1. e details of each component are described below: (1) e desktop application sends commands to the robotic arm control using a USB serial protocol.case, due to the design of the robotic arm, only two markers were used, but this design can be extended to robotic arms with more than two articulations.(5) e mobile application receives the degree of each articulation and displays them using a virtual degree protractor.

Main Blocks of the Desktop App.
e desktop application is designed for sending commands to the control system of the robotic arm using a serial USB connection and gets feedback about the angle of each articulation.is application was developed using Matlab/Simulink ® , and the position of each articulation was independently programmed using a proportional-integral-derivative (PID) control.
e desktop application also allows the user to perform simulation and generates comparison plots between real and desired trajectory, taking as input the times and angles for each articulation.

Main Blocks of the Control Module.
In Figure 2, a ow diagram including the software routines of the control module is shown.ese software modules are hosted on an Arduino board, with an HC-05 Bluetooth shield.Each one of these modules is described below: (i) Con guration routine. is module establishes the con guration parameters of both serial connections (Bluetooth and USB), the transfer speeds, and the pin modes required for external communication.
(ii) Connection request manager. is module detects if there is at least one connection request by the mobile application or by the desktop application.(iii) Get base angle routine. is module translates the lecture obtained by the encoder located in the base articulation to digital values in order to generate the angle of the base. is angle is sent to the mobile application using the previous Bluetooth connection.(iv) Get arm angle routine. is module translates the lecture obtained by the encoder located in the arm articulation to digital values in order to generate the angle of the arm. is angle is sent to the mobile application using the previous Bluetooth connection.

Main Blocks of the Mobile App.
e main blocks of the mobile application are shown in Figure 3. e details of each block are described below: (i) Wireless connection module (WCM). is module must allow users to detect if any compatible robot arm Bluetooth connection is available.If it is the first time connection, this module asks the user for linking the device with the platform.Once the platform has been linked, the module is ready to get data from the platform to the application, activating the rest of the application functionalities.(ii) Image acquisition module (IAM).
is module serves as a bridge between the physical sensor and the application.e IAM accesses the camera buffer for accessing the image obtained by the camera sensor.

(iii) Marker localization module (MLM).
is module obtains the input video from the IAM and performs the marker localization using the previously obtained data in the training phase for the specific markers located in the platform.Once the marker has been located, then the obtained coordinates are stored and sent to the ARIM.(iv) Articulation degree module (ADM).
is module performs the articulation degree requests depending on the located marker by the MLM.
e WCM returns the angles for each located marker and passes to the ARIM.(v) Augmented reality integrator module (ARIM).is module takes as input the dimensional localization of the marker obtained by the MLM and the degree of each marker obtained by the ADM and generates the final image where the protractor with the obtained angles is overlaid on the image obtained by the IAM.Finally, this image is shown to the user on the device screen.(vii) A mobile device with back camera and Bluetooth adapter.In this device, the developed application is deployed, and several tests were performed in order to validate the localization of the markers and image visualization.e proposed application has been validated on devices with several Android versions.In Table 1, a list of devices used and its specifications used for testing the proposed application are shown.e application worked without problems in the listed devices.

Hardware Tools
In Figure 5, connections required among the Arduino board, the driver, and the power source are shown.e Arduino board hosts the control module and sends commands to the motor driver, and each motor is connected to the dual motor driver.e Arduino board receives commands for the robot arm from the desktop application through USB connection, and the angles of each arm articulation are sent to the mobile application using Bluetooth USB connection.

Software Tools.
e desktop application was developed using Matlab and Simulink, on a desktop computer with Windows OS.
e mobile application was developed using Android Studio IDE with Android Studio SDK and Java SE Development Kit. e modules of the application were written in Java.In addition, ARToolkit was used for the marker recognition and integration of virtual and real objects in live capture obtained by the camera sensor of the device [15].

Designed Application.
In Figure 6, the base (Figure 6 In Figure 7, the desktop application and the mobile application are shown.
In Figure 8, two frames of the obtained angles of each articulation and its visualization in the APP are shown.From the obtained images, it is possible to conclude that the recognition of each marker is performed successfully, and the display of the protractor with the obtained angles is visible in each of the tested angles.

Time and Angle Results
. In order to verify the alignment precision of the two axes (base and shoulder), measurements were carried out using Matlab.Figure 9(a) shows results from base angle trajectories, where the position error ranges from 0.75 °to 3.8 °.In the case of shoulder axis (Figure 9(b)), variations are from 0.08 °to 7.28 °. ese variations could be reduced by using some control strategies.With respect to time lag between sending the instructions and the display of the response, a delay from 1 to 2 seconds was observed in the experiments.

Final Recalls and Future Work
In this paper, an approach for teaching robotic arm manipulator concepts was proposed.
e system includes a homemade robotic arm, a control system, and the RAR@ pp. e RAR@pp is focused on learning robotic arm manipulation algorithms by the detection of markers in the 4 Mobile Information Systems   Mobile Information Systems robotic arm and the real-time visualization of the angles of each articulation of the homemade robotic arm using augmented reality.ese angles are obtained from the robot based on the data obtained by the encoders of each motor and transmitted to the RAR@pp using Bluetooth communication. is application allows students to send con guration parameters to the robotic arm and visualize in real time the response of the arm to the commands using the RAR@pp.e application was tested in a large number of mobile devices, including smartphones and tablet devices.e proposed platform allows capturing the student's attention, facilitating the understanding of complex robotics and kinematics concepts.e implementation of a smartphone-based application will allow a large number of students to access to this type of educational resources, improving their performance and their understanding of key concepts of robotics and kinematics.
In this work, basic techniques combining augmented reality have been applied for the visualization of the articulation arm angles.Future work could be oriented to replace the use of markers for the direct recognition of the shape of the object for AR applications.e direct object recognition capabilities could be used for working with di erent types of arms and for larger degrees of freedom.Moreover, several sensors could be added to the robotic platform in order to expand its capabilities.For example, current sensors could be used to visualize the e ect of torque in the axis motors.Implementation of pressure sensors in an end e ector or inertial sensor to study the device behavior could also been implemented.It also can be added to the RAR@pp the  Mobile Information Systems capability to display information about motor's temperature and the measurement of execution time for a specific task.
(2) e robotic arm control sends back the angles of each articulation, and these angles are used for plotting a comparison graph between desired and real path.erobotic control generates the 2 Mobile Information Systems movement commands for each articulation, based on the information obtained from the desktop application.(3)e mobile application sends a connection request to the robotic arm using a Bluetooth channel.e robotic arm receives this request and grants the connection.Once the connection is established, the mobile application identi es the markers located in the robot arm and sends a request to the robotic arm controller.(4) Each robot articulation has a marker.In this speci c

Figure 1 :Figure 2 :
Figure 1: Main components and data ow of the proposed system.
(i) Two 12 V DC motors with optical encoders.(ii) A L298N dual H-bridge motor driver.(iii) A 12 V 10 A power supply.(iv) A HC-05 Bluetooth module for communicating Arduino via Bluetooth communication with the AR application.(v) An Arduino ONE device for hosting the communication modules.(vi) Homemade robotic arm designed using Solid-Works ® and built using material form local providers.e SolidWorks design is shown in Figure 4(a), and the final design is shown in Figure 4(b).
(a)) and arm (Figure 6(b)) markers are shown.ese markers were used for training the ARK toolkit classification model for its recognition.

Figure 4 :
Figure 4: Design and realization of the robotic arm: (a) robotic arm design with SolidWorks; (b) robotic arm prototype.

Figure 3 :
Figure 3: Main blocks of the mobile application.

Figure 6 :Figure 5 :Figure 9 :
Figure 6: Marker for the robot arm articulations and localization of each market by the AR demo: (a) base articulation marker; (b) arm articulation marker.

Table 1 :
Devices utilized for testing the proposed application.