Diabetic retinopathy (DR) has been one of the major causes of visual impairment and blindness. According to Sabanayagam et al., the annual incidence of DR ranges from 2.2% to 12.7%, and the progression ranges from 3.4% to 12.3% [
In contrast, automated image processing has proven to be a promising alternative for retinal fundus image analysis and its future application in eye care. Several recent studies have utilized state-of-the-art deep-learning (DL) algorithms for the automated detection of DR from a large number of fundus images [
Several studies that examined the efficacy of automated detection have used standard fundus cameras that provide 30° or 50° images. In recent years, however, various fundus cameras have been developed, such as the ultra-wide-field (UWF) imaging fundus camera and optical coherence tomography angiography (OCTA).
UWF, otherwise known as Optos (Optos 200Tx; Optos Plc, Dunfermline, United Kingdom), is a non-contact, noninvasive imaging modality that can capture up to 200° of visible fundus and has become essential for understanding and managing the peripheral retinal pathologies of adult diseases such as diabetes and retinal vein occlusions [
OCTA has been devised to noninvasively detect moving objects within the fundus, such as flowing red blood cells, as a flow signal and visualize it as a blood vessel [
However, manual analysis of multiple fundus images for accurate screening in clinical practice requires a substantial effort from ophthalmologists. As such, the objective of the present study was to investigate the accuracy of AI using different composite images.
The study was approved by the Ethics Committee of Tsukazaki Hospital (Himeji, Japan) (no. 171001) and Tokushima University Hospital (Tokushima, Japan) (no. 3079) and was conducted in accordance with the tenets of the Declaration of Helsinki. Informed consent was obtained from either the participants or their legal guardians after the nature and possible consequences of the study (shown in Supplemental Human Studies Consent File 1) were explained to them.
The study dataset comprised 491 images and data from patients with diabetes. The data of those without fundus diseases between 2016 and 2019 were extracted from the clinical database of the ophthalmology departments of Saneikai Tsukazaki Hospital and Tokushima University Hospital. Images were reviewed by three retinal specialists to assess the presence of DR or NDR and registered in an analytical database. All patients underwent Optos (Optos 200Tx®, Nikon), OCTA (OCT Triton plus®, Topcon), and UWF fluorescein angiography. OCTA scans were acquired over a 6 × 6 mm2 region.
En face images of the superficial plexus, deep plexus, outer retina, and choriocapillaris and the density map were extracted (Figure
Identification of each image and stage. Test 1 (no apparent diabetic retinopathy [NDR] or diabetic retinopathy [DR]) and test 2 (NDR or proliferative diabetic retinopathy [PDR]) were performed using the Optos, optical coherence tomography angiography (OCTA), and Optos OCTA images.
The present study examined the results of tests 1 and 2 to identify NDR and PDR. For each verification, Optos, OCTA, and Optos OCTA imaging were performed. We described how Optos OCTA images are created in the Image Processing Section.
This study used K-fold cross-validation (
The aspect ratio of the original Optos images was 3900 × 3072 pixels. For analysis, the aspect ratio of all the images was changed and resized to 256 × 192 pixels.
The size of the concatenated original OCTA images was 640 × 320 pixels. The images of the four en face zones (superficial plexus, deep plexus, outer retina, and choriocapillaris) were extracted. The images of the superficial plexus, deep plexus, outer retina, and choriocapillaris were placed on the upper left, upper right, lower left, and lower right (Figure
Test 1 (no apparent diabetic retinopathy [NDR] or diabetic retinopathy [DR]) and test 2 (NDR or proliferative diabetic retinopathy [PDR]) were performed using the Optos (a), optical coherence tomography angiography (OCTA), Optos (b), Optos OCTA images (A1–A4; B0–B4). A1, B1: Superficial OCTA image; A2, B2: deep OCTA image; A3, B3: other retinal layer of the OCTA image; A4, B4: choriocapillaris layer of the OCTA image; B0: Optos image.
The Optos OCTA image (Figure
Representative images of no apparent diabetic retinopathy (a), mild nonproliferative diabetic retinopathy (b), and proliferative diabetic retinopathy (c) obtained using ultra-wide-field (UWF) imaging and optical coherence tomography angiography (OCTA). The UWF image shows the hemorrhage (white triangle) and neovascularization (white arrow). The OCTA image shows microaneurysm (white long arrow), microvascular tortuosity (white dotted arrow), and capillary non-perfusion (white short arrow).
In this study, a visual geometry group, −16 DCNN (VGG16) (Figure
Visual geometry group (VGG) 16 model. The overall architecture of the VGG16 model is shown. The deep convolutional neural network used ImageNet parameters; the weights of blocks 1−4 and 5 were fixed, while the fully connected layers were adjusted.
VGG16 automatically learns the local features of images and generates a classification model [
The aspect ratio of the original Optos images was 3900 × 3072 pixels, whereas that of the OCTA images was 640 × 320 pixels. For analysis, we changed the aspect ratio of all the input images and resized them to 256 × 192 pixels. Given that the RGB image input ranged from 0 to 255, we normalized it to a range of 0−1 by dividing it by 255. To increase the learning speed and improve performance even with a small amount of data, the initial weight values of the first four convolution blocks were used as parameters learned by ImageNet using the transfer learning method [
This study evaluated the performance of six verifications, namely, tests 1 and 2 for Optos, OCTA, and Optos OCTA images. Receiver-operating characteristic (ROC) curves were created on the basis of the abilities of the DL models to discriminate between NDR and DR images (test 1), and between NDR and PDR images (test 2). These curves were evaluated using the area under the curve (AUC), sensitivity, and specificity. Sensitivity and specificity were considered positive (DR in test 1 and PDR in test 2) when the probability of the neural network output was greater than 0.5. The ROC curve was derived using Python scikit-learn (
To compare patient background, age was analyzed using Student's
For the AUC, the 95% confidential intervals (CIs) were obtained using the following formula [
The mean AUC and SE(
SE(
For sensitivity and specificity, 95% CIs were obtained using the Clopper-Pearson method [
The baseline characteristics of the development and clinical validation datasets are described in Table
Patients’ demographics.
NDR | Mild | Moderate | Severe | PDR | |
---|---|---|---|---|---|
Number of images | 169 | 76 | 54 | 90 | 102 |
Patients | 95 | 52 | 40 | 58 | 71 |
Women (%) | (42.6) | (40.8) | (38.9) | (35.6) | (34.3) |
Mean age, years (SD) | 66.8 ± 9.6 | 67.2 ± 9.7 | 67.4 ± 10.3 | 66.8 ± 8.6 | 59.0 ± 11.6 |
Left fundus (%) | (49.1) | (47.4) | (50.0) | (48.9) | (52.0) |
NDR, no apparent diabetic retinopathy; PDR, proliferative diabetic retinopathy.
In test 1, Optos, OCTA, and Optos OCTA images had an AUC of 0.790 (95% CI: 0.751–0.830), 0.883 (95% CI: 0.854–0.912), and 0.847 (95% CI: 0.814–0.880), respectively.
The ROC curves are shown in Figure
Receiver-operating characteristic curve for test 1 (no apparent diabetic retinopathy [NDR] or diabetic retinopathy [DR]) for Optos, optical coherence tomography angiography (OCTA), and Optos OCTA images. The order of imaging methods used based on the accuracy of their results was as follows: OCTA, Optos OCTA, and Optos.
In test 2, the Optos, OCTA, and Optos OCTA images had AUC of 0.981 (95% CI: 0.962–1.064), 0.928 (95% CI: 0.892–0.964), and 0.964 (95% CI: 0.938–0.990), respectively. The ROC curves are shown in Figure
Receiver-operating characteristic curve for test 2 (no apparent diabetic retinopathy [NDR] vs. proliferative diabetic retinopathy [PDR]) for Optos, optical coherence tomography angiography (OCTA), and Optos OCTA images. The order of imaging methods used based on the accuracy of their results was as follows: Optos, Optos OCTA, and OCTA.
Sensitivity and specificity values and 95% confidence intervals.
Test | Device | Sensitivity | Specificity |
---|---|---|---|
Test 1 | Optos | 80.9 (76.2–85.1) | 55.0 (47.2–62.7) |
OCTA | 83.9 (79.4–87.7) | 71.6 (64.2–78.3) | |
Optos OCTA | 78.6 (73.7–82.9) | 69.8 (62.3–76.6) | |
Test 2 | Optos | 90.2 (82.7–95.2) | 97.0 (93.2–99.0) |
OCTA | 74.5 (64.9–82.6) | 97.0 (93.2–99.0) | |
Optos OCTA | 80.4 (71.4–87.6) | 96.4 (92.4–98.7) |
The present study investigated the efficacy of the DL method in identifying the difference between NDR and DR on the basis of 491 multimodal images. The better DL algorithm showed appropriate sensitivity and specificity (AUC: 0.847; sensitivity: 78.6%; specificity: 69.8%), as well as good results with respect to differentiating NDR from PDR (AUC: 0.964; sensitivity: 80.4%; specificity: 96.4%). The ability to discriminate between NDR and PDR presented herein was comparable with that reported in previous studies [
First, OCTA with DL properly detected the difference between NDR and DR (test 1). The current international classification recommends diagnosis based on the presence of superficial retinal lesions. Therefore, the accuracy of OCTA, whose imaging range is narrower than that of UWF imaging, in determining the DR stage has generally been poor. However, OCTA images showed significant differences between NDR and DR even with an unevenly enlarged acicularity index and foveal avascular zone, indicating a relatively satisfactory staging accuracy [
Second, Optos showed more accurate results in distinguishing NDR from PDR (test 2). Once a patient has developed DR, especially severe cases (e.g., PDR), a wider range of images can increase the diagnosis rate. Retinopathy lesions in DR that predominantly develop around the standard field defined in ETDRS 7 [
Progress in traditional technologies, such as digital fundus photography, along with recent advancements in various imaging modalities, has provided clinicians with new information and improved efficiency. Tran and Pakzad-Vaezi reported the benefits of multimodal imaging of DR and the clinical applications of several imaging techniques in DR including color photography, OCT, OCTA, and adaptive optics [
Furthermore, the use of the combination of DCNN and these multimodal images in diagnosing DR is expected to increase in the future, and the use of DCNN in the analysis of retinal images is appealing given its suitability with the current trend of teleophthalmology and telemedicine [
The present study has several limitations. One of the major issues of this study is the small number of images for training. Many DL researchers agree that such a small number of data in each category is insufficient to test the effectiveness of the proposed method. Deep learning generally requires more than a million samples to train without overfitting. Another limitation is that this cohort included eyes treated with and eyes treated without a PRP laser, which may have confounded our results.
In summary, our study suggests that the use of AI in classifying multimodal images did not always produce accurate results and showed advantages and disadvantages depending on the stage. Although combination of DCNN and multimodal images certainly provides better result, it is not particularly superior to medical examination. Face-to-face examinations by ophthalmologists are indispensable for a definite diagnosis.
Although UWF fundus ophthalmoscopy and OCTA images with a DCNN were effective in diagnosing DR, the use of AI in diagnosing multimodal images did not always produce accurate results.
The data that support the findings of this study are available from the corresponding author, Hitoshi Tabuchi, upon reasonable request.
Approval was obtained from the institutional review boards of Saneikai Tsukazaki Hospital and Tokushima University Hospital to perform this study.
Toshihiko Nagasawa, Hitoshi Tabuchi, Hiroki Masumoto, and Zaigen Ohara are employees of Tsukazaki Hospital (Himeji, Japan).
The authors thank Masayuki Miki and orthoptists at Tsukazaki Hospital for their support in data collection. Hitoshi Tabuchi’s laboratory of Hiroshima University received donations from Topcon Corporation (Tokyo, Japan) and Glory Corporation (Himeji, Japan).
The authors show the research consent form from the patient in the Supplemental Human Studies Consent File 1.