Level of Evidence of Dental Research in Saudi Arabia (2000–2020)

Objectives The level of evidence (LOE) of Saudi dental research from 2000 to 2020 was evaluated, and factors associated with the LOE were determined. Methods This study was a systematic review. PubMed, Web of Science, and Medline databases were utilized to retrieve available dental articles published in English between January 2000 and May 2020. The inclusion criteria consisted of clinical studies conducted in Saudi Arabia with at least one Saudi dental affiliation. The retrieved eligible articles were evaluated independently by two reviewers using a modified Oxford LOE scale. The LOE of the studies was compared between the last two decades. Results Of the 7237 articles identified, 1557 articles met the inclusion criteria. Approximately 78% of the published articles reported Level IV evidence. A higher trend toward Level I, II, and III publications has occurred in recent years (i.e., 2010–2020). However, no statistically significant difference existed in LOE proportions between the two decades. The presence of international collaboration and high journals' impact factor was significantly associated with a higher LOE. Conclusion Most published dental research studies were low LOE studies (i.e., Level IV). National and international collaboration is highly encouraged as this is a factor, according to our findings, that would be a positive addition toward publishing dental research of a higher LOE in Saudi Arabia.


Introduction
In the early 1990s, evidence-based dentistry (EBD) was introduced and became a fundamental part of dental practice [1]. e primary aim of EBD is to improve the quality of oral health care by integrating dentists' experience, the values of the patients, and the best accessible scientific evidence [2]. erefore, dentists are expected to regularly be updated with new techniques and procedures to provide the best treatment for their patients. In essence, EBD created a need to ensure that dentists can refer to high-evidence, valid, and reliable evidence that will guide them in their clinical decision-making in dental practice. e level of evidence (LOE) is a tool that guides clinicians to search the clinical evidence on certain topics rather than judging the quality of evidence [3]. Based on the level of evidence, the research can be ranked from the highest evidence (Level I, i.e., randomized controlled trials) to the lowest evidence (Level IV, i.e., case reports). Generally, the results from the high level of evidence studies are more reliable and reproducible in clinical practices [4,5]. David Sackett first introduced a ranking system to assess the methodological quality of clinical studies regarding the level of evidence. e Oxford Centre for Evidence-Based Medicine is widely used as the standard guide to rank evidence in medical and dental fields [6].
e Oxford LOE has been modified and adopted by many researchers [7][8][9]. Unlike other LOE assessment tools, Oxford LOE considers both the design and the outcomes of the studies.
National and international researchers conducted bibliometric studies to appraise and assess the quality of dental research [10][11][12][13]. For example, many studies have evaluated authorship, the presence of collaboration, and journal impact factors. Ul Haq et al. [14] conducted a bibliometric analysis of Saudi dentists' published research and analyzed their citation impact [14]. e authors concluded that Saudi researchers need to find more innovative research ideas to obtain global attention and more citations. However, to the best of our knowledge, no previous study has explored the level of evidence of published research in dentistry in Saudi Arabia.
Conducting high-evidence research has challenges; however, the government's directions are allocated to provide financial and the best available resources to support research.
e Saudi government has prioritized health, education, and research in the Kingdom's Vision 2030, declared in April 2016 [15]. Hence, conducting research with a high level of evidence is possible. As the number of studies published by Saudi authors has dramatically increased [14], this study, therefore, aimed to evaluate the level of evidence of published Saudi dental research over the past 20 years and the changes in the level of evidence over time. We also investigated whether the level of evidence of Saudi dental research was affected by journal type, impact factor, number of citations, and the presence of international collaboration.

Search
Strategy. An online search was conducted between June 2020 and December 2020 at Umm al Qura University in Saudi Arabia. e authors accessed PubMed, Web of Science, and Medline databases to search the terms "Saudi Arabia" AND "dentistry oral surgery and medicine" OR "dentistry" OR "oral surgery". After retrieving the available studies, each article was identified by title and abstract screening. e inclusion criteria were applied (described in Section 2.2), followed by accessing the full text to extract more data.

Study Selection.
is study included all clinical-based dental, oral, and maxillofacial studies having at least one author with Saudi dental affiliation (affiliated address in Saudi Arabia). We included studies conducted in Saudi Arabia that were published in English in any medical, dental, public health, or health science journal between January 2000 and May 2020. e authors excluded benchwork studies, animal and cadaveric studies, narrative reviews, qualitative studies, expert's opinion, letters to the editors, editorials, comments, book chapters, studies that recruited population only from outside Saudi Arabia, and studies that had no access to their full text ( Figure 1). e two reviewers independently graded the studies by using the criteria of a modified Oxford Centre for Evidence-Based Medicine [16]. e articles were ranked from Level I (e.g., randomized controlled trials) to Level IV (e.g., case reports), based on the modified Oxford LOE criteria (Table 1). Level V studies (e.g., animal studies) were excluded from the review process, consistent with LOE assessments used in other medical specialties [17]. e reviewers independently conducted the screening procedures and classified the articles independently. Any disagreement was resolved by discussion until a consensus was reached.

Data Extraction.
e following information was extracted from each article: name of the journal, journal type, impact factor, publication year, citations, study design, and whether the study was part of an international collaboration (i.e., authors having international affiliation). All data were collected in Excel (Microsoft, Redmond, WA, USA).

Statistical
Analysis. Stata 23 (StataCorp, LLC., College Station, TX, USA) was used to conduct the statistical analysis. Descriptive statistics such as frequency and percentage were calculated. e chi-square test was used to compare between categorical variables. Statistical significance was set at p < 0.05. e kappa value was calculated to assess the level of agreement between the two reviewers. Kappa values were assessed using the criteria described by Landis and Koch [18].

Results
Of the 7237 articles identified, 1557 articles met the inclusion criteria and were included in the review process ( Figure 1). Among the various study designs, cross-sectional studies were the most commonly published studies and accounted for 64.55% of the total published articles ( Table 2). Systematic reviews and randomized controlled trials were the second most common published articles (20% collectively). e level of agreement between the two reviewers was almost  perfect, kappa � 0.96 (p < 0.001). Table 3 describes the benchmark scale proposed by Landis and Koch (1977). Level IV studies constituted 78% of all publications, followed by Level I studies (10%), Level II studies (9%), and Level III studies (3%). Level IV articles were the most frequently published papers over the past 20 years. A higher trend in Level I, II, and III publications occurred in recent years ( Figure 2). One hundred eighty-one studies were published between January 2000 and December 2010, and 1376 studies were published between January 2011 and May 2020. No statistically significant difference in LOE proportions existed between the 2 decades (Table 4 and Figure 3).
International collaborations with centers outside of Saudi Arabia were reported in 35% of the articles. Studies that had an international collaboration with other countries had statistically significantly higher proportions of Levels I, II, and III studies and a low proportion of Level IV studies, compared to studies without an international collaboration (p ≤ 0.001).
e 1557 articles were published in 306 different journals (Table 5). Approximately, 67% were published in dental journals and 33% were published in nondental journals. No significant association existed between journal type and LOE (p � 0.1). e articles were published in journals with impact factors ranging 0-8.227 (median, 1.33). e results showed a significant association between the LOE and a journal's impact factor (p ≤ 0.001). Journals with high impact factors had significantly higher proportions of articles with a high LOE.
e citation numbers of the articles ranged 0-299 (median, 8). No statistically significant association existed between the LOE and the number of citations (p � 0.082).

Discussion
e primary objective of our study was to evaluate the level of evidence of dental research in Saudi Arabia using a modified Oxford LOE scale. We found that the percentage of the high-evidence research (i.e., LOEs I and II) accounted for approximately one-fifth of all articles included in our study. In contrast, Level IV studies represented more than threequarters of the overall Saudi dental publications during the past 20 years. is finding is not limited to the LOE of Saudi dental literature as most Saudi medical publications consisted of Level IV studies in different specialties in the medical field [19]. Compared to dental literature in other countries, Iranian and Brazilian dental literature studies show similar findings: Level IV evidence represented the highest prevalence of research among all other levels [11,[20][21][22]. In addition, a study published by Natto et al. [10], which analyzed dental articles of 33 ISI journals over the last 50 years, showed that case report and case series (i.e., Level IV) constituted the highest prevalence among all other study designs in the dental literature worldwide [10].
In general, high LOE studies such as randomized controlled trials are not always feasible to conduct because of certain ethical, financial, or other challenging factors. However, study designs with a lower LOE such as crosssectional studies and case reports/series are much easier, faster, and less expensive to conduct. is factor explains why researchers tend to conduct more studies using these lower LOE study designs. In addition, in many clinical situations, observational study designs or clinical trials may be the most convenient methods to answer certain types of research questions, although they are generally graded with lower evidence than the controlled and randomized interventional designs. Some research methods provide better evidence than others because not all study designs are equal regarding the risk of errors and bias. erefore, methodologies that present less bias should always be considered when applicable because they contribute to a higher evidence level while considering the adequacy of the research design in answering the research question [23].
Our study also gave insight regarding the changes in the LOE of Saudi dental research over the past two decades. During the past 10 years, the number of published Saudi dental articles has increased approximately seven times more than in the preceding decade. is remarkable growth in the number of publications is related to the increase in the number of dentists and dental institutions and graduate programs that encourage research in Saudi Arabia. With  International Journal of Dentistry regard to dental research evidence, the percentage of Saudi high LOE dental research (i.e., LOEs I and II) has increased by 5%, whereas the percentage of lower LOE publications (i.e., LOEs III and IV) has reduced by 5% during the period from 2011 to 2020, compared to the period from 2000 to 2010. is desirable change is minor and statistically not significant, although it reflects the awareness of dental researchers to conduct more studies with higher LOE. is improvement in the LOE of Saudi-affiliated dental research over time parallels the positive changes in the LOE of dental publications in dental journals worldwide [10]. Collaboration with international dental institutions and the impact factor (IF) of a journal in which an article is published are potential indicators of the LOE. Our study found a statistically significant association between the LOE and these factors. Approximately, more than one-third of the total Saudi-affiliated dental studies had an international collaboration. e percentage of the high LOE (i.e., LOEs I and II) collaborated studies (27.8%) was approximately twice that of local studies (13.7%) with the same LOE, whereas the percentage of low LOE collaborated studies (LOEs III and IV; 72.2%) was 14.2% lower than the percentage of studies without any international collaboration (86.3%). Our finding was consistent with those of medical studies that showed a statistically significant association between international collaboration and high LOE published medical articles [5,7,17,24]. Hence, international collaboration, which expands the border of experience interchanges between dental institutions and improves the LOE of research, is highly encouraged and should be facilitated. e journal impact factor is commonly used to rank a journal based on the number of citations, to facilitate the comparison between various scientific journals in terms of quality and reputation. It is obtained by calculating the average number of citations of an article published in that journal during the two preceding years [25]. e higher the number of article citations, the greater the IF of the journal. Our study found a statistically significant positive association between the LOE and a journal's IF. A medical study by Amiri et al. (2013) had similar findings, whereas other researchers did not find any statistically significant association between a journal's IF and the LOE [4,7,24].
A debate exists regarding whether the IF should be viewed as a quality measure of a journal because the citation rate of published articles varies and therefore it is not representative of an article's quality [7]. Impact factors are used to assess the overall merits of a journal, but they are inadequate for evaluating the quality of an article because they are not directly related to the LOE of the articles. However, editors of journals with higher IFs are stricter about the LOE of articles they publish. Many editors have statistical advisors to ensure the quality of the articles published. Some journals may request the authors to submit the LOE rating of their clinical study, which indicates that they consider the quality of the publication [7]. No other system is as widely accepted as the IF for assessing a journal's quality [26]. erefore, the IF is not a direct measure of the quality of an article but it can be an indicator of the LOE of publications in scientific journals because the quality of the articles and the quality of journals may influence each other.
Citation counting is one way to measure the impact of the article. However, some concerns exist when using the number of citations to evaluate the LOE of a study [13], which include the fact that recent articles have the disadvantage of receiving fewer citations than older articles. Also, articles published in open access journals may receive higher citations due to their availability and easy accessibility. In addition, published articles in less popular research topics receive fewer citations, despite the LOE of the published study. Bias may also occur as a result of citing already highly cited articles [13].
Neither the category of the journal (i.e., dental vs. nondental) nor the citation count had any statistically significant association with the LOE, as indicated by the results of our study and a study by Jamjoom et al. [24].
Our study searched three of the most commonly used databases in dental research. Each author independently accessed each article with a high interrater agreement. Among the articles found by searching these databases, 332 articles were excluded during assessing the eligibility for inclusion because they were nonaccessible, which may affect the results. Also, some journals that published Saudi-affiliated dental studies may not be indexed in these databases, and therefore they could have been overlooked.  e Oxford level of evidence (LOE) classification system is widely used in the medical literature to simplify the process of determining the strength of the evidence generated by various study designs [23]. However, the Oxford LOE scale was developed, to a large extent, to evaluate study designs that answer primarily the clinical research questions related to therapeutic interventions [27,28]. Research questions that are related to prognostic, diagnostic, and causation studies are best answered by observational study designs; however, not all research designs are included in the Oxford LOE classification. erefore, many previous studies have applied modifications to this classification by integrating other systems such as the Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) [20,29]. Observational studies constitute a substantial portion of dental research. erefore, in our study, we also modified the Oxford LOE scale to evaluate various studies that provide valuable data in Saudi dental literature.
One of the limitations of our study was its focus on clinical research only. e preclinical studies, including in vitro and animal studies, which also provide vital information before designing clinical studies, were excluded when using the modified Oxford LOE system. Such preclinical and experimental study designs represented a significant part of dental research, particularly in the field of dental materials, dental tissue regeneration, and oral biology. erefore, it is recommended for future studies to use a LOE classification scale that can evaluate the dental laboratory and animal studies as well.
Also, the current classification system used for grading the LOE of research gives a general evaluation of the clinical research, but critical appraisal of individual articles is always recommended to precisely determine the quality of a study because not all studies of the same design have the same quality. us, some well-designed observational studies can provide better evidence than poorly conducted RCT studies which generally provide better evidence.
Unfortunately, the dental literature has a general lack of well-designed studies due to errors with control groups, bias, randomization, blinding allocation, sample size calculations, statistical analysis, and results interpretation [10]. erefore,   International Journal of Dentistry the study design and ensuring the proper study structure should be considered in the evaluation of the LOE. Hence, some LOE classifications systems such as the GRADE system rate up the observational studies that have a dramatic effect for more accurate assessment and better application of the LOE [29]. is current study spots light on the level of evidence of current dental research. e LOE by itself is not an indicator of the clinical significance of the study but it is a very useful tool to categorize articles based on evidence level for evidence-based practice and for developing policies for research management and funding. We recommend future studies to objectively assess the reasons for conducting low LOE studies in dental publications in Saudi Arabia and consequently guide dental researchers to design and conduct better evidence studies in future research.

Conclusion
Based on our study's modified Oxford LOE scale, most clinical dental research in the past 20 years in Saudi Arabia had Level IV evidence. Level I was the second most common LOE, based on our study's findings, whereas Level III was the least common LOE. e presence of international collaboration and journal IF had a statistically significant association with the LOE of Saudi dental research. However, neither the category of the journal nor the citation count seems to be associated with the LOE of dental publications. e increase in the number of dental publications in the past decade has been substantial with a slight improvement of the LOE. However, we encourage Saudi researchers to conduct more studies of higher LOE to promote better EBD in Saudi Arabia.

EBD:
Evidence-based dentistry IF: Impact factor LOE: Level of evidence RCT: Randomized controlled clinical trial GRADE: Grading of Recommendations, Assessment, Development, and Evaluation.
Data Availability e data that support the findings of this study are available on request from the corresponding author.

Conflicts of Interest
e authors declare no conflicts of interest in this study.