Comparative Effectiveness of Simulation versus Serious Game for Training Nursing Students in Cardiopulmonary Resuscitation: A Randomized Control Trial

Research and Community Health Departments, Faculty of Nursing, Aja University of Medical Sciences, Tehran, Iran Student Research Committee and Department of Critical Care Nursing, Faculty of Nursing, Aja University of Medical Sciences, Tehran, Iran Department of Psychiatry, Harvard Medical School, Children’s Hospital Boston, Massachusetts, USA Department of Military Nursing, Faculty of Nursing, Aja University of Medical Sciences, Tehran, Iran Women’s Health Research Institute, Department of OBGYN, University of British Columbia, Canada School of Nursing and Midwifery, Tehran University of Medical Sciences, Iran


Background
Cardiac arrest (CA) is one of the most common causes of mortality and morbidity throughout the world. In North America and Europe, the annual incidence of sudden CA is approximately 50 to 100 per 100,000 in the general population [1]. Evidence shows that despite advocacy to improve CA treatment in recent years, overall survival rates following CA remain low [2]. Immediate initiation of cardiopulmonary resuscitation (CPR) is necessary to preserve and resuscitate the heart and brain [3]. The International Liaison Committee on Resuscitation that describes data on systems of care and outcomes following out-of-hospital cardiac arrest (OHCA) from nine national and seven regional registries across the world reported that the provision of bystander CPR was 19.1-79.0% in all registries. Survival to hospital discharge or 30-day survival after emergency medical service-treated OHCA was 3.1-20.4% across all registries [4].
Studies show that the CA and CPR knowledge in nursing and medical students needs to be improved [5,6]. So the European Resuscitation Council promotes a guidance note to improve CPR knowledge among healthcare students [7]. Studies have pointed positive impact of using new training methods [8,9]. Researchers suggest that new and alternative methods to improve CPR knowledge and skill among healthcare students could be useful [5,6,10]. The success rate of CPR depends on appropriate and effective training. Currently the most popular form of CPR training involves a model of theoretical introduction and individual mannequin-based simulation procedural training. This allows the learner to engage actively in their learning process and practice patient care, in a controlled and safe environment. Mannequin-based simulation training has been determined to be effective in training nursing students and medical students [11][12][13].
With increasing computer literacy, online learning and gaming technology for education have increased in popularity [14]. Serious games or applied games are virtually based games designed with a primary purpose other than pure entertainment. Serious game development and implementation for medical education are a growing domain which attempts to deliver affordable, accessible, and usable interactive virtual learning, supporting applications for education [15,16]. The use of serious games and virtual patients engages learners and avoids the risks associated with practicing on real patients. Serious games provide flexible learning, enhance learning outcomes, and improve collaboration across participants [10]. Theoretical benefits of serious games in medical education include increasing availability in remote settings, providing use at a flexible time for the user, and the opportunity for the program or game itself to tailor the educational material to match the learner's level of skill [17]. Preliminary evidence suggests the effectiveness of serious games and virtual patient interventions in medical education [15,18,19]. More specifically, virtual game learning has been rated highly by nursing students with students reporting successful skill acquisition [20].
The use of serious games to provide CPR training includes a virtual emergency scenario where the learner is cued to save the life of a victim requiring CPR. CPR training using the serious game platform has shown feasibility, popularity, increased student concentration, self-efficacy, and a higher level of preparedness in students receiving the game training [21,22].
While research exists noting the relative advantages of both simulation and serious game training methods, to our knowledge, no research directly compares the two. This study was aimed at estimating the effect of a simulation-based CPR training using a mannequin in comparison with serious game training on the smartphone platform in terms of learning outcomes among nursing students. We hypothesized that the knowledge and skills related to CPR performance would be significantly improved in the serious game group or the simulator group.

Study
Design. This randomized control trial was conducted from 2018 to 2019 and registered in the Iranian Registry of Clinical Trials (No. IRCT20150801023446N19).
2.2. Participants and Setting. Through purposive sampling, 56 first semester nursing students (nursing students typically take eight semesters prior to graduation) from a nursing school in Tehran were selected and randomly assigned to one of three groups (simulation using a mannequin, serious game using a smart phone platform, and control group which included no training). Eligibility criteria encompassed agreement and informed consent to participate in the study, had not ever participated in any CPR training, and access to a smartphone. Exclusion criteria included reluctance to continue in the research or missing any of the CPR courses.

Sample
Size. Previous research where education was provided to skill acquisition of nursing students [12] afforded a type I and II error, respectively, of 0.05 and 0.1 and S 1 = 1:63, S 2 = 1:01, μ 1 = 13:13, and μ 2 = 11:58, thus suggesting the appropriate sample size for each group in the current study to be 16 participants. Predicting a 10% dropout rate, we considered a minimum of 18 subjects per group.
Nursing students were randomly assigned into a simulation-based CPR training using a mannequin (n = 18), a serious game on the smartphone platform group (n = 18), or a control group (n = 20) before the baseline assessment. A researcher assistant generated the random allocation sequence by throwing the dice, and she assigned participants to three groups. She was not involved in the intervention or outcome assessment. It should be noted that in the control group, two students did not participate in the posttest and were excluded from the study. A total of 54 students were evaluated.

Instruments and Data
Collection. Before the trial, the students were evaluated individually for their CPR knowledge and skill using three different measurements: a demographic questionnaire, knowledge questionnaire (Appendix A), and an appraisal checklist (Appendix B). The demographic questionnaire provided information on sex, age, living situation (whether the student lived in the dormitory or off campus), marital status, university entrance exam score, history of 2 International Journal of Computer Games Technology theoretical nursing training on CPR (yes/no), history of practical nursing training on CPR (yes/no), and see others doing CPR (yes/no). A 20-item, multiple-choice questionnaire for assessing CPR knowledge was created for this study. The knowledge questionnaire assessed the student's understanding of CA, when to use CPR, understanding of basic life support (BLS), and the steps involved in CPR. The knowledge questionnaire scores ranged between 0 and 20, with 20 as the most proficient score. It was chosen as the arbitrary cut-off to classify three categories: "low level (0-35%)," "moderate level (35%-70%)," and "high level (<70%)". Low-level scores range from 0 to 7; moderate-level scores range from 7.1 to 14; and high-level scores range from 14.1 to 20. Higher scores indicate greater knowledge. A score of 14 (70%) and higher indicates adequate understanding of CPR.
In addition, CPR skill performance was assessed using a live 10-minute simulated scenario of CA on a training mannequin. Each student's performance was scored using a 31item checklist for appraising CPR skill which was created for this study. Scores on this CPR skill appraisal checklist ranged from 0 to 31, with 31 as the most proficient score. We considered three categories: "low level (0-35%)," "moderate level (35%-70%)," and "high level (<70%)". Low-level scores range from 0 to 11; moderate-level scores range from 11.1 to 21; and high-level scores range from 21.1 to 31. Higher scores indicate greater skill. A score of 21 (70%) indicates adequate performance of CPR. The CPR skill appraisal checklist included evaluation of the BLS skill.
The CPR knowledge questionnaire and CPR skill appraisal checklist were developed by the research team based on the 2015 American Heart Association (AHA) guidelines for CPR [23]. To determine the qualitative content validity, ten professors of nursing with expertise in the field of CPR and three experts in the field of instrumentation evaluated the knowledge questionnaire and appraisal checklist. For the face validity of the tools, 15 nursing students previously trained in the emergency course were interviewed face to face regarding the knowledge questionnaire and appraisal checklist. They provided critical information on the items assessed, and changes were made to the measures to reflect and modify any concerns. Pearson correlation coefficient was used to determine the test-retest reliability of the knowledge questionnaire (0.72), indicating strength in association between the measures over time. Cronbach's alpha coefficient was used for the knowledge questionnaire and appraisal checklist, indicating internal consistency, 0.80 and 0.88, respectively. Raters for the appraisal checklist included two research assistants trained in CPR and subject assessment. The interrater reliability of the skill appraisal checklist was determined using Cohen's kappa coefficient with an agreement of 0.93.
The knowledge questionnaire and appraisal checklist were used as both a pretest and a posttest for all three intervention groups. One of the researchers rated the students' CPR performance. The pretest was given 30 minutes before the training intervention and the posttest was given two weeks after the training intervention.

2.5.
Intervention. The first intervention group included CPR training using simulation on a mannequin (KAR/ CPR180S×1PCS, Made in China). The mannequin was specifically designed for CPR instruction and included electronic verification of proper CPR operation. It therefore allowed students to correct mistakes as they were engaging with the mannequin. The training scenario was developed according to the 2015 AHA guidelines [23]. The CPR simulator trainer was a postgraduate critical care nurse with ten years' experience in critical care nursing, and certified in CPR. Training was conducted in the simulation lab of the nursing school. Before each training, study mannequins were examined and calibrated. Students were divided into two groups of nine students. Training was completed in one day. Students participated in two sessions, each at least 45 minutes. Each student practiced and performed CPR on the simulator until they were determined to master CPR. Students were encouraged to ask questions of the instructor during and after each workshop.
The second intervention group included use of a serious game on the smartphone platform. The serious game was based on the latest version of the Basic Critical Action Algorithm and AHA Guide [24]. The serious game was created by the Virtualware Group ( Figure 1) and translated into Persian by the researchers for this study. Firstly, students were instructed on how to install and play the serious game. Then, the player was guided throughout the step-by-step instructions. In fact, it was a self-learning modality without a face-to-face instructor. The player experienced a self-assessment after the completed task and she/he received feedback. When the player failed a task, she/he tried again. Students in this group were able to connect to the researchers and ask questions via group chat on Telegram. Students were given two weeks to engage with the serious game. The serious game training takes approximately two minutes in total to complete.
The third group was the control where no intervention was performed. The control group received the pretest and posttest, completed two weeks apart. The control group received education on CPR through simulation at the end of the data collection. Since most students lived in dormitories and would likely discuss the tests among themselves, we ordered the tests to prevent contamination bias between groups. The pretest and posttest were completed in the control group first, then in the simulator group, and finally in the serious game group. The study process is shown in Figure 2. 2.6. Data Analysis. The data were analyzed in the SPSS software (version 16) using the descriptive statistics (mean, standard deviation, frequency, percentage) and analytical (Fisher's exact test, Kruskal Wallis, Mood's Median test, Wilcoxon, and Mann-Whitney U). The Kolmogorov-Smirnov test was used to evaluate the normality of the data. Statistical analyst was blinded. She was not aware of the allocation of students to the groups. A p < 0:05 was considered significant. Of note, Bonferroni-adjusted significance tests for pairwise comparisons were used with an alpha set at 0.05. With three comparisons, the LSD p value required for significance is 0.0167.

3
International Journal of Computer Games Technology 2.7. Ethical Approval. The study followed principles of the Helsinki declaration and was approved by the Ethics Committee of the Aja University of Medical Sciences (No. IR.AJAUMS.REC.1396.69) and principles of the Committee on Publication Ethics. All students were informed about the study details, and verbal informed consent was obtained. The researchers assured the students about their right to refuse to participate or withdraw from the study at any time. The researchers also assured the students that they would not be harmed and that their personal information would be anonymous and confidential.

Results
3.1. Background and Demographics. The mean age of the nursing students was 20:09 ± 0:94 years. Slightly over half of the students were male (55.6%), 92.6% were single, 96.3% lived in dormitories, 98.1% had no CPR training (theoretical or practical), and only 7.4% had exposure to CPR prior to this study. None of the students were licensed practical in nursing. There was no significant difference between the three groups in terms of any of the background characteristics (p < 0:05) (see Table 1).  The scores for the pretest CPR knowledge questionnaire ranged from 1 to 12 with a mean of 5:48 ± 2:21, which indicates knowledge below minimum standards. Mood's median test noted no significant difference between the three groups on the pretest knowledge questionnaire (p = 0:214). There was no difference between pretest scores in the simulation and serious game groups (Mann-Whitney U test, p = 0:024, ns). Wilcoxon test showed a significant difference in the simulator and serious game intervention groups between the pretest and posttest on the knowledge questionnaire (p < 0:001 and p = 0:022, respectively), whereas, the control group showed no difference between the pretest and posttest (p = 0:684). Mood's median test indicated a significant difference between the knowledge score of three groups after the training (p = 0:011). On the posttest knowledge questionnaire, there was no significant difference between the simulation intervention group and the serious game group (9:55 ± 2:81 versus 7:77 ± 2:46; Mann-Whitney U test, p = 0:065, ns). On the posttest, mean knowledge scores for all three groups (47.75%% simulation group, 38.85% serious game group, 27.2% control group) were less than the 70% level, which was initially set as a score of adequate CPR knowledge (see Table 2).

CPR Skill.
The Kruskal-Wallis test noted that the CPR skill appraisal checklist was not significantly different across the three groups in the pretest (p < 0:999). The Wilcoxon test showed that CPR appraisal skill scores significantly improved between the pretest and posttest in the simulation and serious game intervention groups (p < 0:001 for both). There was no significant difference in the pre-and posttest scores for the control group (p < 0:999). On the posttest, the simulation and serious game groups were significantly different from the control group (p < 0:001) (see Table 3). However, the simulation and serious game groups were not significantly different from each other at the posttest (27:17 ± 2:81 vs. 25:72 ± 3:98, Mann-Whitney U test, p = 0:988, ns). Mean scores in the intervention groups (87.64% simulation group, 83.06% serious game group) are greater than 70% and determined to be above adequate skill level.

Discussion
The current study demonstrated that nursing students provided with CPR education through a traditional simulation training or a more novel serious game platform improved from baseline in their CPR knowledge and skill development. Both the simulation training and serious game groups showed improved CPR knowledge in comparison to the control group. Overall, the simulation training and serious game groups did not significantly differ in their posttest scores indicating both provided a similar amount of learning for nursing students. Of note, neither group provided enough     International Journal of Computer Games Technology training to increase the nursing students' ability to increase their knowledge score on a questionnaire, but both the simulation and serious game groups showed adequate demonstration of CPR skills. It could be due to insufficient motivation of students to increase knowledge. In the present study, the researchers anticipated that the knowledge and skills related to CPR performance would be significantly improved in the serious game group or the simulator group. This hypothesis was derived from literature suggesting that serious games are more effective than other self-learning methods [25]. We found that the simulator and the serious game groups increased the performance on a test of CPR knowledge and skill demonstration. However, the simulation and serious game groups did not differ in their ability to provide education to nursing students in the current study. Most studies demonstrate the usefulness of either the simulation or the serious game education. For example, Akhu-Zaheya et al. examined the effectiveness of simulation on knowledge acquisition and knowledge retention of nursing students. Their findings showed that students who were trained with traditional teaching and high-fidelity simulation earned higher scores in BLS knowledge as compared to those that received only traditional teaching [11]. Heidarzadeh et al. compared the effect of heart and lung examination training using simulation and lectures on midwifery students' knowledge. Training by simulation was found to be effective in enhancing students' knowledge compared to those that received only lecture teaching [26]. In a study conducted by Li et al., they used 3-dimensional virtual technology (3D CPR game) for freshman medical students who had no previous CPR training experience. This technology was applied to assess its effect on three-month CPR skill retention. The findings of their study showed that the use of 3D games was effective in increasing and retaining the CPR skill for medical students [27]. In another study, Low et al. showed that the use of the iResus application (that is a free application that includes current adult and pediatric algorithms in an interactive and intuitive format while remaining entirely faithful to the original content) significantly enhanced physicians' capability in advanced life support [28]. The previous studies differed in their study population characteristics, specific aspects of training such as if they included direct instruction or were purely self-learning, and the actual platform of the training. These slight differences likely play a role in learning and knowledge gained from the different learning methods investigated. It is possible that in our current study, the use of a group facilitator who encouraged questions from the students leads to more similar results between the groups, making it difficult to tease apart the invention of simulation and serious game from the use of the facilitator/educator.
Of concern with both the simulation and serious groups' training is the lack of knowledge gained, as measured by questionnaire, from both types of education. Neither group was able to demonstrate on the knowledge questionnaire a level above 70% at posttest. However, they were able to demonstrate adequate skill on demonstration. At this time, it is difficult to determine if the knowledge questionnaire was too tough for the students and could not adequately measure their knowledge or whether the two intervention groups were unable to relay the underlying concepts and knowledge of CPR. It is possible that students would have benefited from direct instruction via a lecture format to relay information regarding CPR knowledge. Nevertheless, as many evidences show that CPR knowledge and skill are poor in more nursing and medical students [5,6], it is recommended to use these training methods along with other traditional methods. Further ongoing investigation is needed.

Limitations.
This study is a single-center study, which possibly limits the generalizability of the findings to other centers. The small sample size can be considered another limitation of the study, and a larger sample might find more accurate results. Furthermore, the researcher evaluated the effects of the training two weeks after intervention and it is unclear if the results would continue after a longer period posttraining. The evaluation methods were created and tested for the first time for this study. It is possible that other measurements might find different results and these measures should be evaluated in other studies to determine their effectiveness more clearly. Also, we tried to prevent the contamination bias between groups by order of the interventions. Also, the researchers did not include the nursing students who performed a CPR course before entering in nursing school.

Conclusions
Overall, this study indicated that CPR training for nursing students using the traditional simulation training and the novel serious game training on a smartphone platform both increased CPR knowledge and skill equally. Neither method increased knowledge to a satisfactory level but both increased skill performance to a level that was determined adequate. The serious game is a self-learning modality without faceto-face instructor-led training; while simulation has access to real-time CPR feedback devices. We speculate that creating a CPR training for nurses that includes either simulation or a serious game component would provide adequate CPR skill obtainment. The serious game training can be a substitute for the conventional hands-on training for CPR. In addition, direct instruction is needed to support CPR understanding and knowledge. Further studies with higher sample sizes and longitudinal follow-up are recommended.    Place the patient in an appropriate supine position until help arrives 6 Place the patient on a hard and firm surface 7 Support the patient's head along with their body if a spinal injury is suspected 8 Control the patients' carotid pulse in less than 10 seconds to check for pulses 9 Start chest compressions immediately if there is no pulse or no pulse is suspected 10 Place the heel of the hand on the center of the patient's chest, two fingers above the xiphoid process, for a chest compression 11 Put the other hand and lock the fingers of both hands 12 Care about not contacting the chest by fingers 13 Exert vertical chest compressions using the palms of both hands while keeping elbows straight 14 Start chest compressions at the rate of 100-120 compressions per minute 15 Perform chest compressions with a depth of 5-6 cm 16 Release compressions completely to help return the chest to its original position after each compression 17 Control the patient's airway after 30 consecutive and nonstop chest compressions 18 Open the patient's mouth and clear it up properly in the case of the presence of external objects or secretions 19 Open up the airway through appropriate maneuvers, depending on the patient's conditions 20 Give two breaths to the patient immediately after opening the airway (there is no need to see, hear, and feel their breathing) 21 Pinch the patient's nostrils shut using the thumb and the index finger of the hand placed on the forehead 22 Use the other hand to keep the patient's mouth open 23 Take a deep breath and then start mouth-to-mouth resuscitation after covering the patient's mouth with their owns 24 Give the breaths around a second 25 Pay attention to the patient's chest expansion after each breath 26 Keep on breathing to the extent that the patient's chest expands 27 Leave the lungs to be depleted between the resuscitations for at least 1.5 seconds 28 Complete the compression-ventilation ratio for about 2 minutes with no interruptions (5 cycles of 30 : 2) 29 Recheck the patient's pulse after two minutes of resuscitation for less than 10 seconds 30 Continue compressions and breathing via 30 : 2 cycles if there is no pulse and check the pulse every two minutes 31 Take the above-mentioned steps until the patient's pulse returns, the defibrillator is provided, and the help arrives to start the advanced cardiac life support Number of correct answers (yes) Number of incorrect answers (no) 10 International Journal of Computer Games Technology (d) Continuing to breathe through the mouth to the nose and putting firm pressures on the epigastric region B. Appraisal Checklist (Table 4) Data Availability The data used to support the findings of this study are included within the article.

Disclosure
The funders did not play any role in the conceptualization, design, drafting, and approval of the manuscript.

Conflicts of Interest
The authors declare that they have no conflict of interest.

Authors' Contributions
ZF involved in the study conception, planning, data analysis, interpretation, writing, and critically revising the paper. MY contributed to the planning, data collection, interpretation, and writing of the paper. SB involved in the study interpretation and critically revising the paper. MN contributed to the planning and data collection. JM involved in the revision of the paper. All authors collaborated in the study, and all read and approved the final manuscript.