Flipping a Dental Anatomy Course : A Retrospective Study Over Four Years

Flipped classrooms have been successfully used to increase student engagement and support student learning in a range of educational fields, including health education. These advantages for student learning supported implementation of the flipped classroom in introductory sciences and preclinical courses in dental education. We report on a 4-year retrospective study which compared two methods of delivery of a first-year dental anatomy course. The first method used the traditional method, consisting of face to face contact teaching hours, which was compared to a partial flipped classroom, where lecture contact was maintained but practical classes were flipped. A series of online videos demonstrating different practical tasks such as wax carving and tooth identification. An online digital library and online quizzes for self-reflected learning were developed and trialled. Students’ Evaluations of Course (SEC) and students’ overall performance in practical and theoretical assessments were used to evaluate the impact on student engagement and success, respectively, after implementation of the modified course offerings. This study evidences the success of the transition to a partially flipped course design. Careful design and consideration of implementation of the flipped classroom method in dental education are recommended to ensure that there is reliable availability of online resources and dedicated teaching staff for construction of resources and delivery of relevant in-class activities.


Introduction
Flipped classrooms, also known as "inverted classrooms," are increasingly recognised as valuable for supporting student engagement and learning.The main rationale of this teaching approach is to support and encourage students to explore curricular content before attending classes with the teaching staff, through online resources such as online videos.This approach helps students to undertake self-directed learning prior to scheduled classes to familiarise them with relevant content and identify challenging concepts.Improved student preparation prior to on-campus activities enables teaching staff to use more engaging activities such as problem based learning as well as asking questions and interacting with peers and supervisors in class [1], during on-campus face to face contact hours.Initially, flipped classrooms were used in high schools and involved prerecorded lectures outside of class and homework-like activities in class [2].This approach demonstrated the flipped classroom to be a very important instrument which showed powerful and effective support and stimulus for students' learning and engagement [3], in addition to potential benefits of consistency of curriculum exploration and financial efficiency.The definition of a flipped classroom is also rapidly expanding to mean any approach that requires students to prepare outside of class for active participation in class [4].
Key advantages of flipped classrooms were discussed previously in the literature, including active involvement of students in their learning, teaching staff gaining a better understanding of students' difficulties, accommodation of a range of different learning styles during in-class activities, progression of students at different paces and using class time 2 Education Research International in creativity, innovation, and critical thinking [5,6] rather than content delivery.A more student focused and personalized learning experience that meets students' individual needs and accommodates their different learning styles could be achieved by using flipped classrooms [7,8].Flipped classrooms seem ideal to be used for topics that consist of a number of procedures to be learned rather than factual, conceptual, and metacognitive types of knowledge illustrated in the revised Bloom's Taxonomy [1,9].The predominant disadvantages or potential barriers against application of flipped classrooms previously reported include time restrictions, inability to adequately prepare before sessions due to work or family commitments, requirements to participate in group activities, an overwhelming amount of learning materials, and lack of motivation for preparation for in-class activities [10].
The increasingly detailed and varied literature on the flipped classroom approach clearly emphasises that there is no generic or magic recipe for flipped classrooms.Successful flipping requires active engagement by experienced teaching staff to design curricula and learning environments using a student focused lens.However, some characteristic features are identifiable for a successful flipped classroom and designing a learning environment that allows flexibility and selectivity and promotes active learning [11].Development of online resources and learning processes have been suggested to augment the flipped classroom experience including Pow-erPoint slides, lecture capture recordings, animated-solved assignments, web-based simulation games, case studies, real world applications, and learning management systems as well as before and after class online quizzes [12].It is imperative that the interactive solutions and opportunities offered by flipped classrooms are scaffolded in alignment with adults learning needs, namely, experience and reflection [13] as well as curricular design.Furthermore, new innovative approaches have been used successfully in dental education not only at an undergraduate level but also in professional development programs [14].
With the rapid development of a range of new technologies and teaching approaches such as Massive Open Online Courses (MOOCs), there is an obvious paradigm shift towards online and distant modes of teaching, with the future of face to face lectures continuing to be controversial.Some authors believe that we will always still rely on traditional lecturing methods, especially in practical disciplines [15].This study reports the transition from traditional lecturing methods to a partially flipped model, whereby the traditional lecture program was supplemented using online resources and a flipped classroom approach for practical tasks and acquisition of new skills in a first-year dentistry course.

Aim and Objectives
The aim of this study was to investigate the effect of a flipped classroom approach on the practical course component of our first-year dental science and dental technology course in dental anatomy and tooth morphology.Students' evaluations of the course and their success in completion of course assessments were used to examine the effectiveness of the implementation of the flipped classroom for students enrolled in the School of Dentistry and Oral Health, Griffith University, Gold Coast, Australia.
Our objectives are as follows: (1) To assess students' perception of the flipped classroom through course evaluation surveys as a tool for feedback collection.(2) To assess the value of the flipped classroom in acquisition of discipline specific knowledge through monitoring and comparing students' overall performance in written and practical assessments throughout a period of four years, in which for two years (2012 and 2013) the entire course was delivered using a traditional face to face method, while in the remaining two years (2014 and 2015) the same course was delivered by the same lecturer using flipped classroom for only the practical components.

Study Design
The dental anatomy and tooth morphology course used in the present study was first developed in its current form and curricular content in 2012.It was delivered by the same lecturer for two years (2012 and 2013) using a combination of traditional face to face lectures and in lab class learning environments.In 2014, the practical or laboratory component of this course was flipped, while the remainder of the course design and assessment was retained and delivered by the same lecturer in 2014 and 2015.The didactic lecture content and active presentation style of the lecturer remained unchanged.The course and practical class convenors designed and constructed an online digital library using the Blackboard learning management system that included photos of all natural human extracted teeth used in practical laboratory sessions.At the same time, a number of quizzes, generated from selected images from the library, were designed to review all important concepts covered in practical laboratory sessions.The design and format of the questions were deliberately consistent with those used in the written and practical assessments for the course (Figure 1).In addition, a series of online video demonstrations by the same lecturer who delivered the face to face didactic lectures were designed, filmed, and edited.The online videos covered all practical activities being undertaken, learned, and assessed in practical laboratory sessions (Figure 2).Specifically, the videos demonstrated the process for the different steps for wax carving of teeth and procedures that should be followed for identification of all anterior and posterior permanent teeth (refer to Table 1 for the course map).
In order to specifically evaluate the flipped classroom method used in this study, we added three questions to the Student Evaluation of Course (SEC).All students are invited but not obligated to complete these evaluations online in the last two weeks of semester prior to commencement of the final examination period.These surveys are coordinated centrally by Griffith University and student anonymity is standard.Teaching staff receive group mean summary and comparative data for similar sized courses.The standard Table 1: The semester map for the dental anatomy course which includes the timing and order of didactic lectures, practical laboratory sessions augmented by online videos/digital library, and assessment items.

Week
Practical labs, online videos/digital library Assessment  questions.In their responses, students had the opportunity to reflect on their experience with the flipped classroom, online resources, and the course overall.Comparative statistical analysis of the overall performance of students in written and practical assessments over the period of four years was performed.These analyses investigated whether the flipped classroom method introduced in this study was effective in supporting students' understanding of curricular content and development of practical skills or not.

Sample Size.
Students' enrolment and average students' ages within the dental anatomy courses examined in this study between 2012 and 2015 are summarized in Table 2 and did not differ significantly during the period of study.The tooth morphology activities were also very useful.The tooth ID online quizzes gave us extra practice on identifying particular tooth outside of lab times to further enhancing our identifying skills since unlike the wax carving which could be practiced at home, tooth ID could only be practiced in the lab session.I found that using this online module greatly improved my confidence in identifying anterior and posterior tooth.

Evaluation Questions
Targeting the Implementation of the Flipped Approach to Practical Classes.Three additional questions were included in the SEC, which were specifically directed at evaluation of the importance of the online resources to support student learning and their impact on the student's learning environment/experience.Overall, the average scores for these additional questions were very positive with very low levels of disagreement.The responses to these questions are summarized in Table 4.

Students' Usage and Access to Online
Resources.All students were self-motivated and attempted the online quizzes at least ten times, without encouragement from teaching staff.Indeed, approximately half of the students attempted the online quizzes more than ten times.Overall, the average number of attempts per student was 11.86 and the maximum number of attempts for a single student was 23 attempts.The results obtained after analysis of students' usage of the online dental anatomy resources are summarized in Figure 3.

Overall Students' Performance.
Students' support for the implementation of the flipped approach was clearly evident through their access to the resources; thus the impact on students' learning was investigated by comparison of the students' performance in the mid-semester and final practical examination components of the students' overall grades before and after the implementation of the flipped approach.
Overall, there was a significant improvement in the student cohorts' performance in the mid-semester and final practical exams after implementing the flipped classroom method for the practical components of the course.Interestingly, this positive outcome did not extend to include the end of semester written exam, which assessed the lecture content only and did not include the practical aspects assessed from the laboratory classes.In the cohorts that experienced the flipped classroom approach, the improvement in practical and laboratory marks resulted in increasing the students' overall performance in the course (Table 5), with more students achieving higher grades and, importantly, fewer "Fail" grades were observed (2.4%) (Table 6).The average grade percentages for each student cohort's performance, both before and after implementation of the flipped laboratory class, is shown in Table 5.The grade distributions for each cohort were depicted in Table 6.

Discussion
There is a recognisable shift towards online supplementation or indeed replacement of traditional courses and this has been an ongoing trend for the last decade.Recording of face to face lectures and then publishing them online (also known as lecture capture) is recognised as highly valuable for supporting student learning as students can pause, rewind, and review lectures at their own pace.Unfortunately, if the recorded lecture stimulates student interest and application of relevance of the content to their context, students cannot  explore this interest through active questioning or interaction with the lecturer [16].Implementation of a flipped classroom approach to teaching in dental anatomy over the period of two years significantly improved both students' performance in relevant assessments and their perception of their learning experience in the course in comparison to previous offerings, where the only online resources were lecture captures of the face to face lectures performed during the semester and traditional demonstration laboratory classes were used.These results are consistent with previous studies which reported that no significant improvement in students' performance and knowledge acquisition was identifiable when only online recorded lectures were used [17][18][19][20][21].
In our experience with flipped classrooms, we demonstrated that use of the purpose designed online resources for the laboratory class and practical skills development, followed up in class with in-class discussion, encouraged students to be better prepared to engage in class, having taken notes and engaged with the online material prior to class.Data extracted from the Blackboard learning management system showed that each student accessed each of the online quizzes and the online video demonstrations at least 10 times throughout the semester.Popular access times were before laboratory sessions and before assessment tasks for final revision.The improvement in students' achievements shown by the increased overall grades (performance) of students in the related assessments provides evidence that the human element in the form of the teaching staff and the opportunity for engagement in in-class interactions and activities are crucial factors to support enhancement of students' engagement and learning in this course.The opportunity for students to interact and ask questions during the oncampus sessions and the online resources was also valued by and valuable to students and their successful study, since the course was not delivered as a pure online course (face to face sessions were retained).Thus creation or provision of high quality online resources alone does not automatically result in greater student engagement or success with course curricula and learning compared to presentation of a course using traditional lecture and laboratory learning environments and activities.Key curricular content, approaches to constructive alignment of curricula content, and assessments need to be readily emphasised and linked to professionally relevant application of concepts and knowledge to clinical practice through skilled and knowledgeable educators during direct contact with students.Dentistry is a program with highly competitive entry, yet these academically capable students readily recognised the value of the retention of on-campus laboratory classes and more effective use of their time as evidenced in the student comments on the SEC.This study does not justify the abandonment of the traditional lecture [22] and supports previous evidence that flipped classrooms could potentially encourage students to engage and think more creatively.The flipped classroom approach is not a guaranteed success, since difficulties may arise, particularly for first-year students transitioning to university, due to the significantly different approach to learning used in this approach.The development and implementation of reliable and sufficient online tools, which are readily accessed and used, in authentic and relevant in-class activities are important elements of this approach [23].
Flipped classrooms could easily outperform traditional teaching methods as reported previously [5], although there remains significant variation in the success of the approach, which may be affected by the context of the flipped classroom intervention.A previous study in which the traditional face to face delivery method was compared to a hybrid method which incorporated a mixture of face to face and online lectures in teaching embryology reported no significant difference between both delivery methods in terms of acceptance by students or improving their overall results [24].In addition, another report which examined organic and analytic chemistry course implementation of a flipped classroom approach also showed no significant improvement over traditional methods [25][26][27].The disparity between these reports and the current study may result from the use of a partial flipped model, where only the procedural knowledge from the laboratory class exercises in the dental anatomy course was presented differently.Thus students who may have found the flipped classroom approach more challenging still had traditional learning environments and security to maintain their academic confidence and performance.Teaching practical tasks such as wax carving and tooth identification requiring skill development are very different from exploration of theoretical or conceptual knowledge traditionally forwarded in lectures.It appears from the previous studies and the current findings that flipped classrooms may provide more benefit for teaching procedural types of knowledge and skill development [1].
Another related and important factor that impacted the successful application of the flipped classroom approach in our study was the age and generation of our students.The majority of our student cohorts were millennials students who were born between the 1980s and 1990s.A number of studies have shown the effectiveness of flipped classrooms and their acceptance by millennial students in the fields of wildlife, family, and consumer services [28,29].The key issue for this age group is their familiarity and reliance upon technology based resources and their appreciation of the flexibility these resources provide for self-directed student learning.
The current study also demonstrated significant improvement of the overall results and performance of all students, including students performing in the upper and lower quartiles of the class.These results are consistent with a previous report, where graduate students in physiology courses that utilised a flipped classroom approach scored better in exams than students in equivalent traditional courses [30].We believe that the improvement of student performance results, at least in part, is due to students developing more academic confidence in the course content through their access to online demonstrations for self-directed study or review at any time and self-assessment or sample questions through the online quizzes.Participation in flipped classrooms is reported to generate a sense of confidence in learning independently [31] and, furthermore, flipped classrooms have been used successfully to assist "at risk" students in high school to improve their academic confidence and success [32].In this dental anatomy course, high achieving students also benefitted from the flipped classroom approach, since they can be well prepared for classes before attendance and then utilise class time to interact and discuss critical elements and difficult concepts with their teaching staff, rather than establishing and learning basic concepts and knowledge alone.
An important and reliable source of data for evaluation of the success of any educational approach or strategy is students' perception and acceptance.Being the service users, students are in the best position to provide valuable feedback about the flipped classroom approach used in the current study.The positive student responses shown in our results are in agreement with other studies that were conducted in different faculties and different disciplines including but not limited to medicine, operations management, counsellor education, mathematics, and law [5,[33][34][35][36].It should be noted that using a flipped classroom concept allows students to monitor their own learning.Therefore, when designing a flipped classroom, recognition and opportunity for students to evaluate and reflect on their own learning [28] should be incorporated.It has previously been reported that the initial course evaluations after using a flipped classroom may be reduced or low, since many students may be unfamiliar and uncomfortable with the flipped classroom approach.These students may need time, opportunity, and support to adapt to this learning environment/approach, since they consider that a teacher's role is to lecture and deliver information.These students often consider that they should not be obligated to prepare at home before the class [36].This was not observed in the current study, where academically capable and experienced students evaluated the course more positively in comparison to previous offerings before using the flipped classroom method, despite the course being a core first-year course in their program which was undertaken in their first semester when they are transitioning from high school to university.We believe this is due to the ongoing support our students received throughout the semester from the teaching staff and IT staff.Furthermore, students studying dentistry are typically high achieving students who have enrolled in a very competitive program.Flipping only the practical component of the course and teaching the theoretical component using the traditional lecturing method contributed to a smooth transition for both students and staff.Interestingly, the flipping of the laboratory component of course instruction did not enhance students' success in their written examinations at the end of semester.
Finally, the development of the online quizzes in our study was valuable for the provision of instant feedback assessing the students' self-directed learning, which enabled them to self-identify their weakness points and limitations in their understanding and to focus their study on improving these areas.In the future, we endeavour to develop some game-like simulation based modules for the same purpose to enhance the engagement of students who have increasing Education Research International levels of expectation regarding the quality of technology enhanced learning resources.

Conclusions
Our experience with the implementation of a partial flipped classroom approach in the teaching of dental anatomy was very positive, in that it enhanced students' success and achievement in the practical components of the course and was perceived by students to be a welcome addition to support their learning.Most importantly, this approach enhanced the creation of an interactive learning environment between students and teaching staff which resulted in a more problem based style of teaching in laboratory classes rather than demonstration.Modification of this learning environment better models the collaborative approach used in the professional environment and thus is more authentic than didactic teacher focused learning.Students value the opportunity to identify and rectify weaknesses in their understanding through engagement with experienced staff in the flipped laboratory classes.This approach also provided students with more opportunity for hands-on experience and improved the opportunity to provide "just in time" feedback on the progressive development of their wax carving which led to increased engagement and a better overall course experience for students.
We recommend careful consideration of the advantages and disadvantages of this approach when deciding or planning to flip parts of an existing course.The teaching team deliberately designed and implemented scaffolded curricular design and authentic assessment tasks as key platforms for the design of the flipped approach.Engagement of skilled teaching staff during the course development is vital and educators should be very clear about why they plan to flip all or parts of their course, recognising the challenges and benefits that may occur.To best support the specific context of your course, a combination of readily accessible, relevant, online resources available under creative commons licensing from the Internet could be used, supplemented by the creation of purpose designed and constructed inhouse resources.From our experience, students preferred and enjoyed seeing their own lecturer/teachers using the same or similar models, specimens, and experiences as were used in their laboratory sessions and subsequent assessment tasks.Using or matching a familiar teaching and presentation style in these resources, through use of the same style, pace, format, and type of questions, both in the online resources and in laboratory sessions, was also perceived positively by students and increased students' satisfaction.Use of the purpose built resources reassured students of the extent to which their academic staff are invested in supporting student learning and success and reduced the opportunity for potential confusion that may arise when externally sourced resources include less relevant or contradictory information.
In the future, we plan to redesign the lecture course and intend to experiment with flipping the didactic lecture component of the first-year dental anatomy course as future student cohorts are anticipated to have increasing familiarity and confidence with self-directed, resource supported learning rather than didactic lectures.We also plan to create partnered videos, where a conversation or a dialogue style video for two teachers will be used.One teacher will play the role of a mentor and the other raises some questions or argues some points for further clarification.We hypothesize that this approach will increase students' engagement even more and will allow for a lot of their commonly asked questions to be answered before arriving to the class.

Week 1 :Figure 1 :
Figure 1: An example of the online digital library and online quizzes developed as part of the flipped classroom method used in this study.

Figure 2 :
Figure 2: An example of the online videos developed as part of the flipped classroom method used in this study.
Course (SEC) Data 4.2.1.Five-Point Likert Scale Questions.Students supported and positively evaluated the flipped classroom approach and use of the online resources through their responses to the standard SEC questions.Introduction of the flipped classroom resulted in a significant increased agreement with the responses to all questions ( ≤ 0.05) after introduction

Figure 3 :
Figure 3: A summary of the students' usage of the online dental anatomy resources investigated in this study.The number of students is shown on -axis and the number of attempts is shown on -axis.

Table 2 :
Showing the summary of students' enrolment and average students' ages during the four-year period of the study.

Table 3 .
4.2.2.Student Evaluation of the Flipped Approach from theOpen Ended Comments.Overall, the majority of students commented positively about the value of the online resources that were made available during the flipped classroom experience.The overall numbers of comments in 2014,  = 63, and in 2015,  = 93, represent overall response rates of 48% and 73%, respectively.A single negative comment was made in 2014, which requested the correct answer being shown if a question was answered incorrectly.Similarly, in 2015, a single negative comment was made, which requested improvement of the quality of the 2D images, in addition to recommending the use of 3D animated teeth models which could be controlled and rotated by the mouse.Quotations of representative examples of student feedback or comments regarding the resources are shown as follows.
The videos that were available were exceptional and greatly aided my learning, practical experience in labs was second to none.I found that the online resources, including the tooth morphology library and the videos for tooth ID and wax carving very helpful because we were able to learn the content at a pace we felt comfortable with.

Table 3 :
Showing a summary of the SEC standard questions results obtained over the period of the study before and after using the flipped classroom method.

Table 4 :
Showing a summary of the results of the additional SEC questions related to the online resources developed in conjunction with the flipped classroom, ( = 84 in 2014;  = 113 in 2015).

Table 5 :
Showing a summary of the overall students' performance in all assessment tasks before and after using the flipped classroom method.

Table 6 :
Showing the distribution of the percentage of students awarded each different grade over the four-year period of the study, both before and after using the flipped classroom method.