Cross-Cultural Adaptation and Validation of the Korean Version of the Dundee Ready Education Environment Measure (DREEM)

Internationally, Dundee Ready Education Environment Measure (DREEM) is being used to evaluate and compare students' awareness regarding medical education environment. This study aimed to adapt DREEM into Korean, to evaluate the reliability and validity, and to compare its structure to the original DREEM structure. The DREEM was translated using 6 steps which were suggested in cross-cultural adaptation protocols: translation, synthesis, back translation, expert committee review, pilot test, and psychometric study (N = 451). We performed confirmatory factor analysis including basic analysis. For evaluating the original model's goodness of fit with the acquired dataset, model fit indices and construct validity were discussed. The Korean version was completed upon through cross-cultural adaptation protocols. Statistical analysis with 451 data sets showed that the root mean square error of approximation = 0.06, goodness-of-fit index = 0.75, and Tucker–Lewis index = 0.73. Almost construct reliabilities were all over 0.707. Except for just one pair, all squares of correlation coefficients were greater than the corresponding average variance extracted. In conclusion, we developed the Korean version of DREEM. Although the original 5-factor structure was acceptable, low convergent and discriminant validity indices suggested that further studies for the Korean environment are necessary for the respecified or modified factor structures.


Introduction
Medical education's principal aim is to produce medical doctors who will be ready to serve the fundamental purposes of medicine: curing disease and participating in all social aspects related to medicine and healthcare [1]. erefore, many countries have tried to build effective and meaningful medical education systems. Furthermore, educator-centered and nonintegrated education systems have been transformed into student-centered and integrated learning systems [2][3][4][5][6][7]. Medical education is composed of several factors, of which educational environment is one of the most important [8]. Educational environment is a very complex area that, in a comprehensive sense, encompasses the physical location, cultural curriculum, educational facilities, and methods of education [4,9]. Educational environments are as important as expertise delivery in medical education [8]. Educational environments contribute to students' achievements, satisfaction, healthy competition, independence, self-confidence, critical thinking, aspirations, and so on [4,[10][11][12][13][14]. Accordingly, it is very important to provide an appropriate environment for facilitating effective medical education [15]. erefore, educators and policy makers should consider the educational environments in order to improve education quality, and researchers have developed various tools for evaluating educational environments [9,16,17]. Pace and Stern developed the College Characteristics Index (CCI) to understand students' perceptions of educational environment [18]. Hutchins developed the Medical School Environment Inventory (MSEI) in order to study medical colleges. Fanslow specifically developed the College Environment Inventory for Women (CEIW) for a study on female college students [19,20]. Furthermore, other tools include the Classroom Environment Scales (CES), the Inventory of College Characteristics (ICCS), the Learning Environment Inventory (LEI), the College and University Environment Inventory (CUCEI), the Institutional Goals Index (IGI), and the Institutional Functioning Inventory (IFI) [21].
In Korea, a DREEM-based survey was distributed across 40 medical colleges nationwide in 2013 [37] and in one college of Korean medicine in 2015 [38]. Because the DREEM was originally developed in English, this international tool includes some problems associated with language issues and multicultural populations [39,40]. To avoid the ambiguities and inconsistencies with regard to meaning on word in each version, a strict translation process should be applied for considering the equivalence between the original language version and the target language version. Crosscultural adaptation is a normative methodology used by researchers in order to consider the differences in languages and cultural backgrounds [41].
In this study, we conducted a series of cross-cultural adaptation processes to develop a Korean version of DREEM based on the English version. We also performed a psychometric study with confirmative factor analysis to identify whether the latent variable structure of the original version could be applied to Korean medicine students.
A bipolar 5-point Likert scale with a neutral response is applied (0: strongly disagree; 1: disagree; 2: neither agree nor disagree; 3: agree; 4: strongly agree) to the 50 items. Among the fifty items, nine items (item 4, 8, 9, 17, 25, 35, 39, 48, and 50) are negative sentences; therefore, they should be reversely coded when scoring and analysis are performed. High scores indicate a positive evaluation regarding the education environment. e developers provided an interpretation guideline for each item or each factor. According to that guideline, items or factors with a score equal to or greater than 3.5 indicate a very positively evaluated area; items or factors with a score between 2 and 3 indicate a positively evaluated area but, however, require some efforts to enhance education environment, and items or factors with a score equal to or less than 2.0 indicate that have to demand attention to education environment [42]. e DREEM questionnaire enables researchers to quantitatively assess students' perceptions of the educational environment. It also makes it feasible to compare results across different institutions, various curriculums, and different survey researches [17].
e World Health Organization (WHO) guidelines also recommend a serial process of forward translation, expert panel back translation, pretesting, cognitive interviewing, and fixing the final version [46]. In this study, we referred to the adaptation guidelines written by Beaton [39] as well as Sousa and Rojjanasrirat's [40] six stages.
Before starting this study, the authors informed Dr. Sue Roff, the original developer of DREEM, about this crosscultural adaptation and received her permission along with her kind and helpful advice via e-mail. Figure 1 depicts the overall process underlying the cross-cultural adaptation of DREEM.
During the first stage, two translators independently translated the English DREEM items into Korean. eir mother tongue was Korean, but they could speak English fluently. We provided this research's aims and advance information to translator 1 (T1), while translator 2 (T2) was provided with no information regarding this study. During stage 2, we synthesized the two products obtained from T1 and T2. After we showed the synthesized version to T1 and T2, we received feedback from the two translators and revised the manuscript (early Korean version). During stage 3, two other translators independently translated the early Korean version produced in stage 2 into English. e two back-translators (BT1 and BT2) were not provided with any information on the original version of DREEM and the medical education system. During stage 4, we organized an expert committee where we developed the prefinal Korean version of DREEM after considering and analyzing all the products obtained from T1, T2, BT1, and BT2. A total of eight experts participated in the committee: one survey research methodologist, one healthcare professional, one language professional, one education professional, and four translators (T1, T2, BT1, and BT2). e first two members of the committee, and two translators (T1 and T2), were Korean medicine doctors who worked in the college of Korean medicine. roughout stage 4, the committee members focused on semantic equivalence, idiomatic equivalence, experiential equivalence, and conceptual equivalence in order to maintain the equivalence between the original English version and the new Korean version [39,43]. Dr. Sue Roff's answers and advice were very helpful during this stage. We were able to develop the prefinal Korean version of DREEM in stage 4. During the next stage, we conducted an in-depth interview with two students to confirm the clarity of the items. ey provided advice from a student's point of view, and almost no changes were made in prefinal version. Consequently, the Korean version of DREEM was finalized. During the final stage, we performed a psychometric study using the Korean version of DREEM. is was followed by statistical analysis including descriptive study and confirmatory factor analysis.

Psychometric Testing and Statistical Analysis.
After completing the cross-cultural adaptation, we performed a human subject research using the newly developed Korean version of DREEM. A survey was administered to secondyear to sixth-year students in two universities. e online survey platform Survey Monkey (Survey Monkey, CA, USA) was used for collecting the responses. A total of 218 students and 233 students from two universities, respectively, voluntarily participated in the online survey.
is human subject research was approved by the Institutional Review Board of Kyung Hee University Korean Medicine Hospital (KOMCIRB-161020-HR-059).
We performed descriptive statistical methods in order to summarize the basic characteristics and responses to the survey and confirmatory factor analysis to compare the latent variable structure of this population with the original structure and previous studies. For evaluating the goodness of fit of the original model using the acquired dataset, several model fit indices were calculated and discussed: goodnessof-fit index (GFI), adjusted goodness-of-fit index (AGFI), root mean square error of approximation (RMSEA), normed fit index (NFI), Tucker-Lewis index (TLI), and comparative fit index (CFI). With these model fit indices, we investigated the factor loading of each item, the construct reliability (CR) for the convergent validity, and the average variance extracted (AVE) for the discriminant validity. We used Microsoft Excel Office 365 (Microsoft, Redmond, WA, USA) for performing the descriptive statistics and R 3.6.1. (R Core Team, Vienna, Austria) and R packages "sem" for performing confirmatory factor analysis. Table 1 presents the Korean version of DREEM, which was developed using the cross-cultural adaptation process.

Demographic Statistics and the Overall and Subgroup
Analysis of the Psychometric Test. A total of 451 students from two Korean medicine colleges voluntarily responded to the survey inquiry. All items were answered, and there was no missing data. Tables 2 and 3 present the overall results and subgroup analysis results, respectively. e descriptive statistics provided in the tables show that over half of the items were identified as being problematic, with scores equal to or less than 2.0. e students responded negatively to many items in general. Almost no differences were found between colleges and genders and among school years.  Table 4 shows the correlation coefficients and the average variance extracted (AVE) for evaluating the discriminant validity.

e Cross-Cultural Adaptation Process.
Cultural and linguistic diversity has caused problems with regard to translation, and many researchers have continued to study translation methods [47]. As a result, a general consensus has been formed for the cross-cultural adaptation process.
Numerous multinational studies have contributed toward DREEM's development. It has been translated into various languages, used in various countries for many years, and has been identified as a nonculturally specific questionnaire [21,22]. Some studies differed from consensus of cross-cultural adaptation process, such as multiple translators not participating, missing the back translation, or proceeding in a different order [26,[48][49][50][51]. Some did not elaborate on whether they followed the consensus process or not [15,24,37,[52][53][54][55][56].
is study encountered some difficulties because the expert committee found some expressions to be unclear during the process of creating the prefinal version. In response, the final version was produced after a comparison with other questions, agreements between experts, and communication with the original writer (S. Roff). In the cases of item #12 "timetable," item #31 "empathy in my profession," and item #33 "in class socially," it was necessary to accurately define the meaning of the words. "Timetable" can be interpreted in various ways including a syllabus or a class schedule. Roff's response was that it included all such meanings. e meaning of "my professional" in item #31 also included both doctor-doctor (the ability to form consensuses between doctors, the ability to discuss diagnoses, etc.) and patient-doctor relationships. In the case of item #33, "class" indicated all courses of study, including lectures and seminars. Internationally, the medical curriculum is not unified, and this can result in differences in terminology. e term "ward teaching" is translated into the Korean term "hoe-jin," which refers to clinical practice conducted by professors, which involves seeing and examining patients in the ward. In Korea, ward teaching is usually conducted in the 6 th year of Korean medicine college. In the case of "constructive criticism" in item #32, students may find the term "constructive" vague. Based on the words and examples provided by Roff, a description that could supplement the word was added in the Korean version. In the case of item #34, we needed to grasp the meaning of individual "seminars/tutorials." Roff suggested that the word was basically referred to a small number of people and that it was a concept that contrasted with the lectures. erefore, "seminars/tutorials" was replaced by small group classes/ small group activities in individual practice sessions.  All data were acquired using a 5-point Likert scale (0-4) and expressed as mean values (standard deviation). DREEM: Dundee Ready Educational Environment Measure. N: number of respondents. SPL: students' perception of learning; SPT: students' perception of teachers; SAS: students' academic selfperception; SPA: students' perception of atmosphere; SSS: students' social self-perception.

Basic Statistics and the Evaluation of Construct
Validity. e descriptive statistics of the results showed that many students were not satisfied with the education environments in Korean medicine colleges. e total and subscores of DREEM were almost equal to or less than half score, and 58% of items were identified as being problematic. is result is similar to that of a previous study in Korea [38].
us, education policy on Korean medicine requires further effort and investment.
Next, we examined whether the original 5-factor model of DREEM was suitable for the new dataset obtained from Korean medicine students. We conducted confirmatory factor analysis and calculated several parameters, including goodness-of-fit indices, convergent validity, and discriminant validity.
First, the goodness of fit can be evaluated when the original 5-factor model is applied to the new dataset. Generally, it has been known that RMSEA less than 0.05 is considered to be good, RMSEA between 0.05 and 0.08 is acceptable, RMSEA between 0.08 and 0.1 is marginal, and RMSEA over 0.1 is poor [57]. e RMSEA for our study was 0.06. We concluded that the original 5-factor model was acceptable for the new dataset obtained from the Korean medicine students. However, the other indices for model fit-GFI, AGFI, NFI, TLI, and CFI-were around 0.7. Among them, TLI and CFI, which were relatively sample size independent, were 0.73 and 0.74. In other countries, they also had results of about 0.7 [30,32,35], respectively, although it is suggested that these indices should be over 0.9 for a good model fit [58,59]. Based on these results, we    e squares of correlation coefficients are expressed in parentheses. SPL: students' perception of learning; SPT: students' perception of teachers; SAS: students' academic self-perception; SPA: students' perception of atmosphere; SSS: students' social self-perception; AVE: average variance extracted; CR: construct reliability. 6 Evidence-Based Complementary and Alternative Medicine concluded that the goodness of fit of this model with our dataset was acceptable but not very good. Such less satisfactory results can imply different concept structures between countries or cultures. Second, the convergent validity was examined using factor loadings and CRs. Ideally, they must be equal to or greater than 0.707 to demonstrate desirable convergent validity [60][61][62], but values over 0.5 are also generally accepted [63]. Among 50 items, only 2 items showed factor loading values over 0.707, and 28 items showed factor loading values over 0.5; however, the CRs of each factor (except for latent variable SSS) were all over 0.707. Based on these indices, we can consider the relatively low convergent validity of the model in this study. Low convergent validity indicates that the items cannot be explained by only the corresponding latent variable and that some other factors are influencing the information of the items. To consider good discriminant validity, the AVE of one factor should be larger than the shared variance estimates with any other factor including measurement errors [64,65]. In this study, except for the only SPT-SSS pair, all squares of correlation coefficients were greater than the corresponding AVEs. Low convergent validity and low discriminant validity indicated that the original 5-factor model was not suitable for our dataset, which was obtained from Korean medicine students. is result may have two possible explanations. First, as shown in the basic statistics, many respondents gave low scores across all the items. Such overall low scores may make it difficult to separate the latent variables, and this may induce low construct validity. Second, because of the differences in languages and cultures, the Korean students' perceptions regarding medical education environments may have different structures from the perception of the other countries' students. In order to clarify this issue, further studies with large amounts of samples should be conducted.

Limitations and Further Research.
is study aimed to develop a Korean version of DREEM to promote more active use of DREEM in Korea and to reduce errors in the comparisons of Korean study results and those of other countries. Moreover, we aimed to obtain methodological justification through a strict process of cross-cultural adaptation, not simple translation by a few bilingual translators. Because of differences in terms of languages, cultures, medical systems, and education systems, the meanings of some items had to be modified despite the kind and considerable help of the original developer. We consider this to be an unavoidable aspect of localization that occurs in crosscultural adaptation.
With regard to statistics, the convergence in each factor and the discrimination among factors were not clear. erefore, our data were not considered to be suitable for the original 5-factor model. As has been suggested by some previous studies, five-factor structure of DREEM (original DREEM subscales) was not good model fit and new structure or abridged version was suggested [24][25][26][27][28][29][30][31][32][33][34][35][36].
rough further additional human subject researches in Korea, the structure model can be modified or respecified using consecutive confirmatory factor analysis or exploratory factor analysis. erefore, if researches using the Korean version of DREEM, which has undergone cross-cultural translation in this study, are conducted continuously, it could be possible to make policy decisions by analyzing what resource aspects should be put on to improve the quality of medical educational environments through comparison studies between educational institutions and countries. Moreover, it could be possible to objectively evaluate the effect of used resources and improved education systems if data are collected in time series for a specific population.

Conclusion
In conclusion, we developed the Korean version of DREEM using the guidelines of cross-cultural adaption in order to create an objective evaluation tool for evaluating medical education in Korea. e adaptation process encountered some difficulties; however, we succeeded in finalizing the final Korean version of DREEM with the original author's help and advice. Furthermore, we performed a human subject research and statistical analysis to confirm the construct validity of the translated version. Statistical analysis regarding validity showed that the original 5-factor structure had a somewhat acceptable fit. Nevertheless, with regard to aspects of convergent validity and discriminant validity, low validity indices suggested that further researches should be conducted in Korea in order to study respecified or modified factor structure. If this study's product is used widely in Korea, we expect that the medical educational environment could be improved by identifying and studying the distinct features of the medical education system in Korea.

Data Availability
e datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Ethical Approval
Ethical approval was obtained from the Institutional Review Board of Kyung Hee University Korean Medicine Hospital (KOMCIRB-161020-HR-059) regarding the psychometric research included in this study. All participants could only respond to the online survey if they read the written information about the contents of the survey and gave their informed consent online before taking part in the survey. Consent e completion of survey was taken as implied consent.

Disclosure
Hyunho Kim and Pyeongjin Jeon are the co-first authors.