The Influence of Using Digital Diagnostic Information on Orthodontic Treatment Planning – A Pilot Study

The purpose of this pilot study was to assess whether orthodontic treatment planning is reproducible when carried out using digital records compared with clinical examinations or using standard records. The study also assessed patients’ opinion of face-to-face consultations and potential use of teleorthodontics. The study was designed as a prospective observational crosssectional pilot study and carried out in a UK dental teaching hospital involving 27 subjects. Four consultant Orthodontists carried out treatment planning, firstly following a clinical examination, then using standard records, and then using digital records. Each subject completed a questionnaire. Cohen’s kappa coefficient and Fleiss’ kappa coefficient were used to assess intraobserver reproducibility and inter-observer reproducibility of treatment planning decisions, respectively. A change in the diagnostic information format affected treatment planning reproducibility for half of the observers. Inter-observer reproducibility was greater when using hard copy records in comparison to digital records. No subjects were unsatisfied with their faceto-face consultation.


INTRODUCTION 1.Telemedicine
The advent of telecommunications in the 20 th Century was soon followed by the development of telemedicine, whereby advice and care were provided to patients distant to the clinician. As technology has improved, its potential use has been explored by different health specialties, particularly dermatology, psychiatry, radiology, cardiology and accident and emergency [1].
Two distinct forms of telemedicine have been developed:(a) 'Store and forward' is used by a referrer in a local care facility, gathering appropriate clinical information from a patient and then sending this onto a specialist clinician in a hospital facility. The advantages of this method are that existing personal computers and peripheral hardware can be used; it is cheaper and the information can be viewed at a convenient time by the receiving clinician. (b) 'Real time' telemedicine allows instantaneous transmission from the patient to the receiving clinician, allowing immediate consultation; however, more advanced technology is required at greater expense.
Over the past decade, qualitative research has found that general dental practitioners and specialist Orthodontists have mixed feelings with regard to the potential clinical benefits of using teleorthodontics when weighed against the financial costs incurred in setting up the system [2][3][4].
In terms of quantitative research, Mandall et al. [5] compared the perceived appropriateness of orthodontic referrals when using digital information supplied via 'store and forward' teleorthodontics and clinical examinations of the same patients. The results showed that triaging referrals using teleorthodontics was acceptable.

Remote and Rural
Scottish Government data for 2005 found that out of a total population of just over 5 million, more than one and a half million individuals in Scotland lived within geographic areas of health boards defined as being remote and rural [6].
The 2007 'Needs Assessment Report on Remote and Rural Dentistry' highlighted that remote and rural areas of Scotland are at a disadvantage in terms of dental service provision. This report noted that far fewer orthodontic appliance treatments per 100 courses of NHS treatment were provided within remote and rural health board areas compared to urban areas [7]. Therefore, the inequality of access to orthodontic care in rural areas affects a significant proportion of the population in Scotland. Digital technology using study models, stereophotogrammetry, plain radiographs, cone-beam CT (CBCT) scans, photographs and digital referral letters could address this issue through the development of teleorthodontics, enabling referrals and treatment planning strategies to be developed distant from the remote and rural patients.

Digital Technology
In orthodontics, in addition to clinical examinations, hard-copy records are taken to undertake treatment planning for a case when the patient is not present. These records most commonly consist of a referral letter, plaster study models of the teeth, intra-oral and facial photographs and appropriate radiographs.
The advent of digital technology holds many advantages compared to these physical records. These include ease of storage, ease of records transfer and a reduction in risk of damage and loss [8].
Whetten et al. [19] found that orthodontic treatment planning decisions were not significantly affected if digital study models were used instead of plaster models.

412
The Influence of Using Digital Diagnostic Information on Orthodontic Treatment Planning-A Pilot Study Therefore, prior to the advent of fully electronic referrals, it is appropriate to determine the reproducibility of treatment plans using differing diagnostic information formats, including digital records.

Aims
The present study was conducted to address the following aims: 1.
To assess intra-and inter-observer reproducibility of treatment planning when using different diagnostic information formats: clinical examination, hard copy diagnostic records and digital diagnostic records.

2.
To assess subject satisfaction with the conventional consultation system and perception of a teleorthodontic referral.

METHODS
The East of Scotland Research and Ethics Committee gave approval for the current prospective observational cross-sectional pilot study of diagnostic information formats for orthodontics. Subjects were included in the study if they had been referred to the Orthodontic Department of Dundee Dental Hospital and School for an orthodontic assessment and/or treatment. The majority of these subjects were referred from General Dental Practitioners or other medical/dental specialties from the surrounding area. This strategy was employed to ensure that the subjects had the same range of malocclusions and ages as patients that would be referred through a teleorthodontic pathway. Subjects were excluded if they had previously undergone orthodontic treatment, had no original referral in their clinical notes, a cleft lip/palate or other congenital craniofacial anomaly, aged under 12 years and/or in the mixed dentition. In addition, if subjects and their parents/guardians were unable to understand written and verbal explanations of the study adequately to provide informed consent, they were excluded. Potential subjects were booked onto specially arranged assessment clinics, having had patient information sheets sent to them beforehand. On attending, any remaining questions from the potential subjects were answered and those that wished to take part in the study were consented.
Clinical orthodontic records were obtained, including impressions for construction of plaster study models. Intra-oral digital photographs were taken with a Nikon D90 camera (Nikon, Tokyo, Japan), a Sigma 105 DG Macro lens (Sigma, Welwyn Garden City, UK) and a Sunpak 16R pro ring flash (Sunpak, Tokyo, Japan). Extra-oral stereophotogrammetry images were taken using the 3dMD face System (3dMD, Atlanta, US). Digital radiographs were taken if this was thought clinically beneficial and justified by the named Consultant. Most subjects had conventional two-dimensional orthopantomograms (OPT) and lateral cephalograms taken. Some had cone-beam computed tomograms (CBCT), periapical or occlusal radiographs taken if this was clinically necessary. The subjects (with help from parents/guardians when appropriate) completed a questionnaire to assess patient satisfaction with the face-to-face consultation and their perception of the potential benefits of teleorthodontics. The Questionnaire was based on the patient satisfaction questionnaire developed by the British Orthodontic Society [20,21] and had been deemed as acceptable for use by the East of Scotland Research and Ethics Committee (Appendix). Data produced from the questionnaire were entered into an Excel database (Microsoft, Redmond, California, US).
The observers in the study were four Consultants in the Orthodontic Department at Dundee Dental Hospital and School. They are all on the United Kingdom Specialist List for Orthodontics and have undergone further sub-specialty training in orthodontics. All four observers routinely assess new referrals and undertake new patient consultations.
Subjects were clinically examined to determine the most appropriate treatment plan for managing their malocclusion. The treatment options open to the observers were: no treatment, orthodontic treatment or surgical-orthodontic treatment. One month later, the observers assessed the hard-copy diagnostic records to determine the most appropriate treatment strategies. These consisted of anonymised paper referrals, colour print outs of intra-oral and extra-oral images and plaster study models. In addition, any radiographs that had been taken were saved as JPEGs and paper copies were printed. Paper lateral cephalogram tracings were made available using the Orthognathic Planning and Analysis (OPAL) computer program (Harradine and Birnie, 1985 [British Orthodontic Society, London, UK]).
After a further month, the observers reassessed the cases using digital diagnostic records. The digital records included a scanned referral letter using a Hewlett Packard C4680 scanner (HP, Palo Alto, California, US), electronic intra-oral images available as JPEGs and 3D scans of the face available on 3dMDvultus Software Platform (3dMD, Atlanta, US) ( Figure 1). Also, 3D scans of study models were captured using a CBCT I-CAT Next Generation scanner (Imaging Sciences International, Hatfield, PA 19440)

414
The Influence of Using Digital Diagnostic Information on Orthodontic Treatment Planning-A Pilot Study (e.g., see Figure 2), copies of digital radiographs were available on Picture Archiving and Communication System (PACS, Eastman Kodak Company, Rochester, NY, US) and electronic tracings of lateral cephalograms were available on OPAL as above.
To assess intra-data reproducibility, after a further month, observer 1 carried out another round of treatment planning, using digital records and observer 2 carried out another round of treatment planning, using hard-copy records.

Statistical Analysis
Intra-observer and intra-data reproducibility were calculated using Cohen's kappa coefficient [22]. Inter-observer reproducibility of treatment plans determined using hard copy and digital records was assessed using Fleiss' kappa analysis [23]. This statistical analysis was carried out using the IBM SPSS Statistics 21 programme (IBM, New York, US). Table 1 illustrates the different comparisons of treatment plans carried out by each observer. The kappa coefficients were interpreted according to Altman [24] ( Table 2).
Due to the relatively small number of responses provided from the questionnaires, descriptive analyses of these data were carried out.

Demographics
Twenty-seven subjects were recruited, ages ranging from 12 to 52, with a mean of 25.1 years. Eleven (41%) subjects were male and 16 (59%) were female. Due to clinical space and time constraints, only two observers could assess subjects at each clinical session. Table 3 shows the treatment plans developed by each observer using each format of diagnostic information. Observers 1 and 3 carried out additional rounds of treatment planning using the same digital and hard copy records, respectively.

416
The Influence of Using Digital Diagnostic Information on Orthodontic Treatment Planning-A Pilot Study

Intra-Observer Reproducibility
According to Altman's interpretation (

Intra-Data Reproducibility
Intra-data reproducibility was investigated to determine if treatment planning variation was influenced by observers carrying out the treatment planning process on different occasions, but using the same diagnostic information. To do this, observer 1 was asked to develop treatment plans for all 27 subjects again, using the digital records. These treatment plans were then compared to those developed using digital records initially. Similarly, observer 3 carried out a further round of treatment planning for all 27 subjects, but using hard copy records, and these findings were compared to observer 3's initial treatment plans developed using hard copies. It was not possible to gather the subjects together again for a further round of treatment planning from clinical assessments. Table 5 shows that observer 1's treatment planning reproducibility remained good when using the same diagnostic information format on different occasions. Observer 3's Journal of Healthcare Engineering · Vol. 5 · No. 4 · 2014 417  treatment planning reproducibility was only fair. This illustrates the variation in intraobserver treatment planning that can be possible, even though the same diagnostic information is used.

Inter-Observer Agreement
The inter-observer agreement of treatment plans developed when using hard copy and digital records is illustrated in Table 6. The results show that inter-observer agreement when using the hard copy records was moderate whilst using the digital records was fair. Inter-observer agreement from clinical assessments was not included as observers did not assess every subject clinically.

Questionnaire
All 27 subjects completed the questionnaire. The purpose of the exercise was to establish the subjects' opinions on the process they experienced in order to be clinically assessed, their experiences at their initial appointment and their perception of a teleorthodontic referral. Among the subjects, 93% felt that the face-to-face consultation provided a good quality of care; Figure 3 shows that the subjects were satisfied with this format. Figure 4 illustrates that the majority of subjects expressed a preference for their consultation to be carried out face-to-face. However, a substantial proportion did not mind if teleorthodontic technology was used as an alternative.

418
The Influence of Using Digital Diagnostic Information on Orthodontic Treatment Planning-A Pilot Study  Patient satisfaction with face-to-face consultation.
Most subjects believed that teleorthodontic referrals would save time and there was an almost even split between the subjects who thought that the process would save them money and inconvenience ( Figure 5).

Treatment Plan Reproducibility
Intra-observer treatment plan reproducibility ranged from good to poor. For two of the observers, levels of reproducibility dropped markedly when comparing those treatment plans developed using clinical examinations and digital records. However, reproducibility was consistent for the other two observers. Inter-observer treatment planning reproducibility was moderate when using hard copy records, but dropped to fair when using digital records. These results are disappointing when compared to those from Whetten et al. [19] who showed good to very good agreement when  Subjects' perceived savings through use of teleorthodontics.
comparing treatment plans developed using digital and plaster study models. Moreover, only three, very broad treatment options were available to the observers. A possible reason for this poor treatment planning reproducibility, associated with the digital records, was the lack of familiarity of the observers with using this technology. Observers were familiar with the use of digital photographs and radiographs as these are currently used in the clinical setting. However, there was a variation in observer experience in other aspects of digital technology. For example, observer 1 regularly carried out digital lateral cephalogram tracing for planning orthognathic surgery. Observer 1 had also been involved in the planning of an e-referral system in a rural health board, and therefore, is likely to have been familiar with the technology available. In addition, this observer had research experience with 3D study models and stereophotogrammetry. Observer 2 had little additional experience in digital technology use for teleorthodontics. Observer 3 had taken part in a previous study assessing the triaging of orthodontic patients using only digital photographs and radiographs and comparing this with assessing the patient face-to-face. This previous study used little 3D digital technology when compared to the pilot study. Observer 4 had previously taken part in a study investigating the use of digital photographs of study models of unilateral cleft lip and palate patients to assess treatment outcome. No observers used 3D study models or stereophotogrammetry routinely. Due to the variability in observer's experience in the use of digital records, a tutorial was provided to each observer so that they could view the records for each subject. However, the process of loading the digital records was not particularly user-friendly. This was time consuming and it was not possible to have all records on screen at once. Ideally, in a full-scale study, a dedicated 3D digital record system would be used. Different software programs are already available in order to enable this [25]. These programs can be used to accumulate digital records of 3D study models, intra-oral and facial images derived from stereophotogrammetry scans and radiographs within one, convenient package. As well as ease of retrieval, this accumulation of digital records also holds the benefit of allowing integration of the clinical information in order to produce what some Orthodontists have termed a 'virtual patient'. This integration means that potential treatment plans can be simulated on screen providing patients with a prediction of their end result.
An alternative means of producing a 'virtual patient' has been described by Kau et al. [26] whereby facial images from stereophotogrammetry can be overlaid on CBCT images. In addition, the CBCTs can be used to produce digital radiographs and even study models. This enables the majority of the 3D digital records to be acquired in two scans with easy record integration. However, the increased radiation dose required for the CBCT exposure means that there are obvious disadvantages to this method.
Despite these difficulties, treatment planning reproducibility was relatively constant for two observers when digital records were used. Notably, there is inherent variability in orthodontic treatment planning both between observers and also by individual observers at different times. This has been investigated previously and shows a wide range, with intra-observer agreement in clinical decision making ranging from fair to 420 The Influence of Using Digital Diagnostic Information on Orthodontic Treatment Planning-A Pilot Study very good. Inter-observer agreement in clinical decision making has been found to be fair in previous studies [27,28]. This may be due to observers interpreting data differently on separate occasions. Variability of treatment philosophies among observers can result in inter-observer variability. However, each observer should have taken into account the Index of Orthodontic Treatment Need score [29] in deciding a subject's suitability for treatment.

Questionnaire
No subjects were unsatisfied with their face-to-face orthodontic consultation. This was favourable when compared to other patient satisfaction surveys [30][31][32][33]. Seventy percent (70%) of subjects felt that the face-to-face aspect of the consultation was extremely important and the majority preferred this over the exclusive use of teleorthodontic technology. Most subjects did believe a referral using teleorthodontics would save them time and many felt it would also save them money and inconvenience.
The possible uses and advantages of teleorthodontics were explained to the subjects and their parents prior to attending their consultation through the use of patient information sheets. However, the vast majority of the subjects recruited to the study were from urban areas surrounding the dental hospital. This may have influenced the results of the questionnaire with respect to their lack of enthusiasm towards the potential benefits of teleorthodontics as the desire for a face-to-face consultation outweighed the slight inconvenience of travelling to the dental hospital. However, the results may well have been different if the subjects had been recruited from a remote and rural area. The more marked difficulty in attending appointments at the Orthodontist for these subjects may well have made the prospect of teleorthodontics more appealing. Teleorthodontic technology has the potential to have a significant impact on the provision of orthodontic care in Scotland, particularly as around 30% of the population live within remote and rural health boards [6].

Strengths
The main strengths of this pilot study have been its originality and its relevance to treatment needs within the NHS. Research into teleorthodontics is at an early stage; however a number of papers have been published concluding that the technology used to implement teleorthodontic applications find no clinical difference from routine hard copy records [9][10][11][12][13][14][15][16][17][18].
Previous studies have found that for the purposes of orthodontic treatment planning, digital records are an acceptable alternative to hard-copy records. This feasibility study is unique as it not only compares treatment plans developed using hard copy and digital records, but also from clinical examinations.
The potential benefits of teleorthodontics are particularly relevant to remote and rural areas of Scotland where the use of ever more accessible digital technology could allow much easier orthodontic consultations to be carried out. This pilot study has been the first to attempt to gauge patient opinion on the potential use of teleorthodontics and if those individuals from remote and rural areas appreciate the potential benefits more than patients where access to orthodontic consultation is easier.

Limitations of Study
Ideally, a larger sample of subjects would have been recruited in order to assess the significance of changes in treatment planning due to a change in diagnostic information format. Unfortunately, the resources were not available to enable this; therefore, a pilot study was undertaken. Should a future, full-scale study be carried out in the future, the data from the pilot study will aid in the calculation of an appropriate sample size.
The subjects were recruited from one centre in order that observers had access to carry out clinical examinations. Therefore, the majority of subjects were from urban areas. Ideally, subjects would also have been recruited from at least one more centre where more patients were likely to live in remote and rural areas. This would allow better comparison of urban and rural opinions regarding teleorthodontics.
As mentioned above, constraints on clinic time and observer availability meant two of the four observers were able to develop clinical treatment plans at each of the data gathering sessions. This meant that full intra-observer clinical treatment planning agreement could not be carried out because the observers did not examine every subject clinically. This also meant that an assessment of inter-observer agreement of the clinical treatment plans was not possible between all four observers. Ideally, resources would have been available for each observer to assess every subject clinically.
The use of digital records was designed to replicate the use of teleorthodontic technology to treatment plan cases. The majority of the technology used was acceptable; however, problems were associated with the digital study models. The laser scanner anticipated for use produced scans of such poor quality that an alternative had to be found. The use of the CBCT provided a solution; however, this practice does not replicate the technology that would be used in teleorthodontics. Furthermore, the digital models were only available on one computer and their retrieval was time consuming.

Implications for Clinical Practice
The results of this feasibility study have shown that for some observers, there is greater variation in treatment plans developed digitally compared to those developed clinically or with hard copy records. However, as technology becomes more user friendly and observers become more familiar with the process of treatment planning with digital records, it would be hoped that this variation reduces.
It is likely that the majority of patients would prefer a face-to-face consultation, but remote and rural patients may be more open to the potential of digital technology being used for teleorthodontic referrals. There is the possibility that using this technology could save them time, money and inconvenience of making a long journey for a consultation. This is particularly relevant, as a large proportion of referrals are known to be inappropriate in the first place. However, in order for this remote referral process 422 The Influence of Using Digital Diagnostic Information on Orthodontic Treatment Planning-A Pilot Study to be feasible, conventional records require to be converted to digital format, most likely by the patients' General Dental Practitioner. The time and cost implications in order for this to be viable would require to be thoroughly assessed in comparison to the benefits to the remote and rural patients.

CONCLUSIONS
This prospective observational cross-sectional pilot study assesses whether orthodontic treatment planning is reproducible when carried out using digital records compared with clinical examinations or using standard records, as well as patients' opinion of face-to-face consultations and potential use of teleorthodontics. The results support the following conclusions: 1 Intra-observer treatment planning reproducibility was influenced by a change in the diagnostic information format for half of the observers.

2.
Inter-observer treatment planning agreement was influenced by a change in the diagnostic information format.

3.
No subjects were unsatisfied with the conventional consultation system.