Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Automatic retinoblastoma screening and surveillance using deep learning

British Journal of Cancer volume 129, pages 466–474 (2023)Cite this article

Subjects

Abstract

Background

Retinoblastoma is the most common intraocular malignancy in childhood. With the advanced management strategy, the globe salvage and overall survival have significantly improved, which proposes subsequent challenges regarding long-term surveillance and offspring screening. This study aimed to apply a deep learning algorithm to reduce the burden of follow-up and offspring screening.

Methods

This cohort study includes retinoblastoma patients who visited Beijing Tongren Hospital from March 2018 to January 2022 for deep learning algorism development. Clinical-suspected and treated retinoblastoma patients from February 2022 to June 2022 were prospectively collected for prospective validation. Images from the posterior pole and peripheral retina were collected, and reference standards were made according to the consensus of the multidisciplinary management team. A deep learning algorithm was trained to identify “normal fundus”, “stable retinoblastoma” in which specific treatment is not required, and “active retinoblastoma” in which specific treatment is required. The performance of each classifier included sensitivity, specificity, accuracy, and cost-utility.

Results

A total of 36,623 images were included for developing the Deep Learning Assistant for Retinoblastoma Monitoring (DLA-RB) algorithm. In internal fivefold cross-validation, DLA-RB achieved an area under curve (AUC) of 0.998 (95% confidence interval [CI] 0.986–1.000) in distinguishing normal fundus and active retinoblastoma, and 0.940 (95% CI 0.851–0.996) in distinguishing stable and active retinoblastoma. From February 2022 to June 2022, 139 eyes of 103 patients were prospectively collected. In identifying active retinoblastoma tumours from all clinical-suspected patients and active retinoblastoma from all treated retinoblastoma patients, the AUC of DLA-RB reached 0.991 (95% CI 0.970–1.000), and 0.962 (95% CI 0.915–1.000), respectively. The combination between ophthalmologists and DLA-RB significantly improved the accuracy of competent ophthalmologists and residents regarding both binary tasks. Cost-utility analysis revealed DLA-RB-based diagnosis mode is cost-effective in both retinoblastoma diagnosis and active retinoblastoma identification.

Conclusions

DLA-RB achieved high accuracy and sensitivity in identifying active retinoblastoma from the normal and stable retinoblastoma fundus. It can be used to surveil the activity of retinoblastoma during follow-up and screen high-risk offspring. Compared with referral procedures to ophthalmologic centres, DLA-RB-based screening and surveillance is cost-effective and can be incorporated within telemedicine programs.

Clinical Trial Registration

This study was registered on ClinicalTrials.gov (NCT05308043).

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Workflow diagram for the development and evaluation of DLA-RB.
Fig. 2: Receiver operating characteristic curves and precision-recall curve of DLA-RB performance in the prospective validation dataset.
Fig. 3: Heatmap visualisation of DLA-RB.
Fig. 4: Performance of the combination of ophthalmologists and DLA-RB in the prospective validation dataset.

Similar content being viewed by others

Data availability

The data in this study are available from the corresponding author upon reasonable request.

Materials availability

The materials in this study are available from the corresponding author upon reasonable request.

Code availability

Python scripts enabling the main steps of the analysis are available from https://github.com/Hugo0512/AI4RB, and we provide some sample images for readers’ testing.

References

  1. Dimaras H, Corson TW, Cobrinik D, White A, Zhao J, Munier FL, et al. Retinoblastoma. Nat Rev Dis Prim. 2015;1:15021.

    Article  PubMed  Google Scholar 

  2. Fabian ID, Onadim Z, Karaa E, Duncan C, Chowdhury T, Scheimberg I, et al. The management of retinoblastoma. Oncogene. 2018;37:1551–60.

    Article  CAS  PubMed  Google Scholar 

  3. Luo Y, Zhou C, He F, Fan J, Wen X, Ding Y, et al. Contemporary update of retinoblastoma in China: three-decade changes in epidemiology, clinical features, treatments, and outcomes. Am J Ophthalmol. 2022;236:193–203.

    Article  PubMed  Google Scholar 

  4. Linn Murphree A. Intraocular retinoblastoma: the case for a new group classification. Ophthalmol Clin North Am. 2005;18:41–53.

    Article  CAS  PubMed  Google Scholar 

  5. Stathopoulos C, Lumbroso-Le Rouic L, Moll AC, Parulekar M, Maeder P, Doz F, et al. Current indications of secondary enucleation in retinoblastoma management: a position paper on behalf of the European Retinoblastoma Group (EURbG). Cancers. 2021;13:3392.

  6. Daniels AB, Patel SN, Milam RW, Kohanim S, Friedman DL, Koyama T. Effect of intravenous chemotherapy regimen on globe salvage success rates for retinoblastoma based on disease class-A meta-analysis. Cancers. 2021;13:2216.

  7. Chen Q, Zhang B, Dong Y, Mo X, Zhang L, Huang W, et al. Comparison between intravenous chemotherapy and intra-arterial chemotherapy for retinoblastoma: a meta-analysis. BMC Cancer. 2018;18:486.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  8. Fabian ID, Stacey AW, Johnson KC, Chowdhury T, Duncan C, Reddy MA, et al. Primary enucleation for group D retinoblastoma in the era of systemic and targeted chemotherapy: the price of retaining an eye. Br J Ophthalmol. 2018;102:265–71.

    Article  PubMed  Google Scholar 

  9. Dimaras H, Kimani K, Dimba EA, Gronsdahl P, White A, Chan HS, et al. Retinoblastoma. Lancet. 2012;379:1436–46.

    Article  PubMed  Google Scholar 

  10. Gulshan V, Peng L, Coram M, Stumpe MC, Wu D, Narayanaswamy A, et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA. 2016;316:2402–10.

    Article  PubMed  Google Scholar 

  11. Peng Y, Dharssi S, Chen Q, Keenan TD, Agron E, Wong WT, et al. DeepSeeNet: a deep learning model for automated classification of patient-based age-related macular degeneration severity from color fundus photographs. Ophthalmology. 2019;126:565–75.

    Article  PubMed  Google Scholar 

  12. Li Z, He Y, Keel S, Meng W, Chang RT, He M. Efficacy of a deep learning system for detecting glaucomatous optic neuropathy based on color fundus photographs. Ophthalmology. 2018;125:1199–206.

    Article  PubMed  Google Scholar 

  13. Tan TE, Anees A, Chen C, Li S, Xu X, Li Z, et al. Retinal photograph-based deep learning algorithms for myopia and a blockchain platform to facilitate artificial intelligence medical research: a retrospective multicohort study. Lancet Digit Health. 2021;3:e317–9.

    Article  CAS  PubMed  Google Scholar 

  14. Milea D, Najjar RP, Zhubo J, Ting D, Vasseneix C, Xu X, et al. Artificial intelligence to detect papilledema from ocular fundus photographs. N Engl J Med. 2020;382:1687–95.

    Article  PubMed  Google Scholar 

  15. Brown JM, Campbell JP, Beers A, Chang K, Ostmo S, Chan RVP, et al. Automated diagnosis of plus disease in retinopathy of prematurity using deep convolutional neural networks. JAMA Ophthalmol. 2018;136:803–10.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Redd TK, Campbell JP, Brown JM, Kim SJ, Ostmo S, Chan RVP, et al. Evaluation of a deep learning image assessment system for detecting severe retinopathy of prematurity. Br J Ophthalmol. 2019;103:580–4.

  17. Hui S, Dong L, Zhang K, Nie Z, Jiang X, Li H, et al. Noninvasive identification of benign and malignant eyelid tumors using clinical images via deep learning system. J Big Data. 2022;9:84.

    Article  Google Scholar 

  18. Zhou W-D, Dong L, Zhang K, Wang Q, Shao L, Yang Q, et al. Deep learning for automatic detection of recurrent retinal detachment after surgery using ultra-widefield fundus images: a single-center study. Adv Intell Syst. 2022;4:2200067.

    Article  Google Scholar 

  19. Pan Q, Zhang K, He L, Dong Z, Zhang L, Wu X, et al. Automatically diagnosing disk bulge and disk herniation with lumbar magnetic resonance images by using deep convolutional neural networks: method development study. JMIR Med Inform. 2021;9:e14755.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Zhang K, Liu X, Liu F, He L, Zhang L, Yang Y, et al. An interpretable and expandable deep learning diagnostic system for multiple ocular diseases: qualitative study. J Med Internet Res. 2018;20:e11144.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Li Z, Guo C, Nie D, Lin D, Zhu Y, Chen C, et al. Deep learning from “passive feeding” to “selective eating” of real-world data. NPJ Digit Med. 2020;3:143.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Shorten C, Khoshgoftaar TM. A survey on image data augmentation for deep learning. J Big Data. 2019;6:60.

    Article  Google Scholar 

  23. Lu B, Li H-X, Chang Z-K, Li L, Chen N-X, Zhu Z-C, et al. A practical Alzheimer’s disease classifier via brain imaging-based deep learning on 85,721 samples. J Big Data. 2022;9:101.

    Article  Google Scholar 

  24. Zhang M, Zhang K, Yu D, Xie Q, Liu B, Chen D, et al. Computerized assisted evaluation system for canine cardiomegaly via key points detection with deep learning. Prev Vet Med. 2021;193:105399.

    Article  PubMed  Google Scholar 

  25. Schmidt H, Spieker AJ, Luo T, Szymczak JE, Grande D. Variability in primary care physician attitudes toward medicaid work requirement exemption requests made by patients with depression. JAMA Health Forum. 2021;2:e212932.

    Article  PubMed  PubMed Central  Google Scholar 

  26. Deng J, Dong W, Socher R, Li LJ, Kai L, Li F-F. ImageNet: a large-scale hierarchical image database. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2009. p. 248–55.

  27. Bottou, L. Stochastic gradient descent tricks. In: Montavon G, Orr GB, Müller K-R, editors. Neural networks: tricks of the trade. Lecture Notes in Computer Science. Berlin: Springer; 2012. p. 421–36.

  28. Skalet AH, Gombos DS, Gallie BL, Kim JW, Shields CL, Marr BP, et al. Screening children at risk for retinoblastoma: consensus report from the American Association of Ophthalmic Oncologists and Pathologists. Ophthalmology. 2018;125:453–8.

    Article  PubMed  Google Scholar 

  29. Hutubessy R, Chisholm D, Edejer TT-T. Generalized cost-effectiveness analysis for national-level priority-setting in the health sector. Cost Eff Resour Alloc. 2003;1:8.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D. Grad-CAM: visual explanations from deep networks via gradient-based localization. In: Proceedings of the IEEE International Conference on Computer Vision. 2017. p. 618–26.

  31. Ramirez-Ortiz MA, Ponce-Castaneda MV, Cabrera-Munoz ML, Medina-Sanson A, Liu X, Orjuela MA. Diagnostic delay and sociodemographic predictors of stage at diagnosis and mortality in unilateral and bilateral retinoblastoma. Cancer Epidemiol Biomark Prev. 2014;23:784–92.

    Article  Google Scholar 

  32. Wong ES, Choy RW, Zhang Y, Chu WK, Chen LJ, Pang CP, et al. Global retinoblastoma survival and globe preservation: a systematic review and meta-analysis of associations with socioeconomic and health-care factors. Lancet Glob Health. 2022;10:e380–89.

    Article  CAS  PubMed  Google Scholar 

  33. Munier FL, Beck-Popovic M, Chantada GL, Cobrinik D, Kivela TT, Lohmann D, et al. Conservative management of retinoblastoma: challenging orthodoxy without compromising the state of metastatic grace. “Alive, with good vision and no comorbidity”. Prog Retin Eye Res. 2019;73:100764.

    Article  PubMed  Google Scholar 

  34. Yousef YA, Al-Nawaiseh I, Mehyar M, Sultan I, Al-Hussaini M, Jaradat I, et al. How telemedicine and centralized care changed the natural history of retinoblastoma in a developing country: analysis of 478 patients. Ophthalmology. 2021;128:130–7.

    Article  PubMed  Google Scholar 

  35. Morrison SL, Dukhovny D, Chan RVP, Chiang MF, Campbell JP. Cost-effectiveness of artificial intelligence-based retinopathy of prematurity screening. JAMA Ophthalmol. 2022;140:401–9.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Funding

Beijing Hospitals Authority’ Ascent Plan (DFL20190201); National Natural Science Foundation of China (82141128); The Capital Health Research and Development of Special (2020-1-2052); Science & Technology Project of Beijing Municipal Science & Technology Commission (Z201100005520045, Z181100001818003). The sponsor or funding organisation had no role in the design or conduct of this research.

Author information

Author notes

  1. These authors contributed equally: Ruiheng Zhang, Li Dong, Ruyue Li, Kai Zhang, Yitong Li.

Authors and Affiliations

  1. Beijing Tongren Eye Center, Beijing Key Laboratory of Intraocular Tumor Diagnosis and Treatment, Beijing Ophthalmology & Visual Sciences Key Lab, Medical Artificial Intelligence Research and Verification Key Laboratory of the Ministry of Industry and Information Technology, Beijing Tongren Hospital, Capital Medical University, Beijing, China

    Ruiheng Zhang, Li Dong, Yitong Li, Xuhan Shi, Chuan Zhang, Wenda Zhou, Haotian Wu, Heyan Li, Chuyao Yu & Wenbin Wei

  2. Beijing Institute of Ophthalmology, Beijing Tongren Eye Center, Beijing Tongren Hospital, Capital Medical University, Beijing, China

    Ruyue Li

  3. School of Optometry, The Hong Kong Polytechnic University, Kowloon, Hong Kong SAR, China

    Kai Zhang

  4. Beijing Ophthalmology and Visual Sciences Key Laboratory, Beijing Tongren Eye Center, Beijing Tongren Hospital, Capital Medical University, Beijing, China

    Hongshu Zhao, Jitong Shi, Xin Ge, Xiaolin Xu, Libin Jiang, Liangyuan Xu, Jing Li & Jianmin Ma

Authors
  1. Ruiheng Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Li Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ruyue Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kai Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yitong Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Hongshu Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Jitong Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Xin Ge
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Xiaolin Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Libin Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Xuhan Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Chuan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Wenda Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Liangyuan Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  15. Haotian Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  16. Heyan Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  17. Chuyao Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  18. Jing Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  19. Jianmin Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  20. Wenbin Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

JMM, RHZ, and DL contributed to the concept of the study. JMM and WBW critically reviewed the manuscript. RHZ, LD, RYL, KZ, and YTL designed the study and did the literature search. RHZ, LD, RYL, KZ, YTL, HSZ, JTS, XG, XLX, LBJ, XHS, CZ, WDZ, LYX, HTW, HYL, CYY, and JL collected the data. RHZ, LD, RYL, KZ, and YTL contributed to the design of the statistical analysis plan. RHZ, LD, and KZ did the data analysis and data interpretation. RHZ and RYL drafted the manuscript. JMM and WBW provided research funding, coordinated the research, and oversaw the project. All authors had access to all the raw datasets and the corresponding authors (JMM and WBW) verified the data and had the final decision to submit it for publication. All authors reviewed and approved the final manuscript.

Corresponding authors

Correspondence to Jianmin Ma or Wenbin Wei.

Ethics declarations

Competing interests

The authors declare no competing interests.

Ethics approval and consent to participate

The Medical Ethics Committee of Beijing Tongren Hospital approved the study protocol. Because individually identifiable information was removed during retrospective collection, written informed consent was exempted from the retrospectively collected dataset. In the prospectively collected validation dataset, informed consent was obtained from all caregivers.

Consent for publication

All authors declare that all information and materials in the manuscript are original. No text, table, figure, or other material has been published elsewhere.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, R., Dong, L., Li, R. et al. Automatic retinoblastoma screening and surveillance using deep learning. Br J Cancer 129, 466–474 (2023). https://doi.org/10.1038/s41416-023-02320-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41416-023-02320-z

Search

Advanced search

Quick links