Detecting Prosthetic Restorations Using Artificial Intelligence on Panoramic Radiographs

,


Introduction
In dentistry, a correct diagnosis leads to a correct treatment plan. Prosthodontics is a complex branch of dentistry that ofers the best diagnostic and treatment options. Te longterm success of prosthetic restoration is achieved with an accurate diagnosis and treatment plan [1].
Panoramic imaging is frequently used in dental practice by clinicians to screen teeth and maxillofacial structures in a single image. Using these images, teeth can be evaluated and clinicians can plan the patient's prosthetic rehabilitation. Although panoramic radiography (PR) is a quick and easy application with a low radiation dose, interpreting radiographic images can be challenging owing to superimposition, distortions, and potential artifacts [2][3][4].
Artifcial intelligence (AI) typically requires human intelligence or computers to perform tasks. Tis can be defned as the acquisition and learning of information using machines. AI recognizes speech, makes decisions, and makes medical diagnoses. It detects anomalies in images that the expert eye cannot detect and identifes problems that cannot be solved by humans [5].
Recently, AI and deep learning have been evolving. Deep learning is an AI-based approach that makes automated decisions. In addition, deep learning is a set of computational models with multiple layers of data processing, which can learn by representing these data through several levels of abstraction. Convolutional neural networks (CNNs) are considered a special case of DL, and they are used in image processing and analyzing radiological datasets [6].
Some studies in dentistry applied deep learning to diagnose and make decisions [7][8][9]. Deep learning detects caries and periapical lesions based on periapical and panoramic images [10][11][12]. Applying deep learning to distinguish prosthetic restorations in PR could save more clinical time for clinicians to focus on treatment planning and prosthodontic operations. Additionally, examination results (such as restorations, crowns, and bridge prostheses) can be classifed using AI technology. Furthermore, AI can be used in undergraduate dental education. AI technology provides the opportunity to support dentistry students in prosthetic clinical internships and auxiliary staf who are less experienced.
Although numerous studies on the automated detection of caries and root canal treatments using AI have been conducted, the use of AI technology in prosthodontics is limited [13]. Tis study proposes a prosthodontic restoration analysis model based on CNN with transfer learning. Tis model analyzes prosthodontics using PR and provides clinicians with a more accurate treatment plan. Tis study improves detection accuracy and reduces the workload of dentists, who can thus focus more on treatment planning.

YOLOv4
Architecture. YOLO was proposed as an endto-end neural network in 2015 [16]. With this architecture, the images are entirely handled, and the images are divided into S × S grids during training. YOLO makes predictions for each small area resulting from the division operation and returns probabilities as the output.
With over 64 million parameters, YOLOv4 is among the state-of-the-art architectures owing to high precision and real-time operating performance.
Training was conducted on a server with an Nvidia RTX2080 Ti (11 GB RAM) graphics card and 192 GB RAM. Te parameters used in training for 30 epochs are listed in Table 1.
Te model was developed and trained using the PyTorch deep learning framework in Python. Datasets of the radiology images were randomly divided into training and testing sets. Of the 5126 PRs, 4605 images were used for training, and the remaining 521 images were used for testing. During model training ( Figure 2), a transfer learning study was conducted using preweights that had previously won the 2017 COCO competition [17]. Although the dataset was separated as a training set at 90-10%, each image was resized to 608 × 608 pixels and used during model training and testing ( Figure 3).

Performance Metrics.
To examine the performance of the model, the complexity matrix (Table 2), which is crucial in image processing problems, and the precision (equation (1)), recall (equation (2)), mean average precision (equation (3)), mean average recall (equation (4)), and F1 score (equation (5)) metrics obtained from the complexity matrix were considered. Recall: Mean average precision (mAP): Mean average recall (mAR): where n: number of samples. Table 3 presents values obtained from the test dataset of the trained model and test data. Additionally, it presents the mAP and mAR values obtained when the confdence threshold was set to 0.1and the TP, FP, FN, precision, recall, and F1 score values obtained when the confdence threshold was 0.25. Based on these values, the bridge results appeared to be more successful than the crown results. Figure 4 provides sample crown detection from the test set images, together with the ground truth label. In this image, a crown was correctly detected and the confdence score of detection was extremely high. Figure 5 provides sample bridge detection from the test set images, together with the ground truth labels. In this image, all bridges were correctly detected and the confdence scores of detection were extremely high. Te YOLOv4 inference time was 90 ms on average. Hence, the model could process approximately 11 images per second. Together with 73.12 and 89.18% AP values and close-to-realtime inference speed, the YOLOv4 model demonstrated that accurate results could be obtained more quickly compared with other CNN-based object detection models. Moreover, the precision and recall curves, high precision, and high recall values all indicate that this study was statistically powerful.

Discussion
Dental panoramic radiographs are widely used for diagnosis in dentistry because of their relatively low dose and cost [17,18]. However, studies on restoration are limited. In addition, owing to multiple superimpositions and distortions, challenging interpretations can cause misdiagnosis [2,3].
AI technologies can deal with complex cases with multiple variables; therefore, the application of AI in prosthodontics is of high interest. AI algorithms promote evidence-based decision making in treatment plans, particularly for less experienced clinicians. In addition, AI

Manual labelling of Prosthetic
Restorations by a specialist practitioner ( Fig. 1a/   technology provides an opportunity to easily analyze patient cases [13]. Tis study used the CNN algorithm YOLOv4, a real-time detection system that classifes targeted objects in a single pass. Te trained AI can appropriately distinguish between teeth and restorations. In this study, detection accuracies with the F1 score were 0.76 for the crown and 0.89 for bridge restoration. Te diference in the success of detecting crown and bridge restorations may be due to the width of the bridge prosthesis and the efect of multiple teeth. Crown restorations are more difcult to distinguish on PR than bridge prostheses. A comparison of the fndings obtained by other studies that evaluated the detection of crowns using AI on PRs and those obtained by our study revealed discrepancies. Basaran et al. [19] used 1084 images in their study; however, PRs with artifacts caused by superposition were not included. In their study, the F1 score was found to be 0.91 for the crown and 0.82 for the pontic. Similarly, in the study by Vinayahalingam et al. [20], 2000 images were randomly collected, and then blurred and incomplete PRs were excluded. Teir F1 score was 0.951. Te high values obtained in these studies may be related to the exclusion of radiographic images with artifacts and/or blurring. Abdalla-Aslan et al. [21] reported accuracy values of 100% for crown restorations in their pilot study. Te high accuracy of the crown can be attributed to the analysis of only 83 panoramic images. Compared with previous studies, the width of our dataset and the inclusion of all data increase the reliability of the results obtained in this study.
Compared with studies regarding crown detection, F1 scores and sensitivity values were extremely low. Te fact that the success of AI in detecting crowns and bridges in our study was lower than those in other studies may be related to the fact that the radiographs used in our study were taken with diferent devices. In addition, in our study, clean data were not preferred to show the applicability and adaptability of the program to all X-rays. Moreover, only repetitive images were removed and all standard data were used.
Apart from studies on PRs, some studies have measured the success of fxation on intraoral images [22,23]. Engels et al. [22] aimed to detect and categorize dental restorations. Te diagnostic accuracy was 97.8% for ceramic restorations and 99.4 for gold restorations in studies in which 1761 images were used. Similarly, Takahashi et al. [23] aimed to recognize dental prostheses using a deep learning object detection method. Tey reported that the system could detect silver-colored complete metal crowns, gold-colored complete metal crowns, resin-facing metal crowns,   porcelain-fused-to-metal crowns, and ceramic crowns. Tis study used approximately 1900 images; according to the authors, this number was insufcient for recognizing all types of prostheses used in dental clinical procedures. When evaluating the results, dental prostheses with metallic color can be recognized with an AP of over 0.80, but those with tooth color are recognized with an AP of approximately 0.60 from oral photographic images. Terefore, the combined use of both intraoral images and PR may be required to decrease misdetection. Detection of prosthetic restorations with automated computer processing provides objective information to patients; in this manner, patients will be motivated to receive dental treatment, confdent about the diagnosis of dentists, and less worried about dental operations [24]. Furthermore, AI technology may improve patient management and patient-clinician relationships [21]. AI-based systems may be useful tools for relieving the workload of dentists and auxiliary staf members. In addition, it can save clinicians' time and assist them in deciding appropriate treatment plans. AI can also be used in student education. Tis would improve the student's ability to read and detect dental radiographic images.
Considering these encouraging results, we propose that crowns and bridge restorations can be detected. With the development of additional software, the marginal ft of these prostheses can be examined. Owing to this AI program, the incidence of crown and bridge prostheses in patients can be compared. Future studies should be conducted in which teeth with crown and bridge prostheses and teeth without any restoration are comparable in terms of caries, periapical lesions, or the need for root canal treatment.
In conclusion, AI technologies in prosthodontics are useful in several ways. Prosthetic restorations were detected with high accuracy using the deep learning method. If panoramic images are used with oral photographic images obtained using intraoral scanners, better results can be obtained in terms of the detection of prosthetic restorations in clinics.

Data Availability
Te data presented in this study are available on request from the corresponding author.

Conflicts of Interest
Te authors declare that they have no conficts of interest.