Skip to main content

09-23-2017 | Retinopathy | Review | Article

Crowdsourcing and Automated Retinal Image Analysis for Diabetic Retinopathy

Journal: Current Diabetes Reports

Authors: Lucy I. Mudie, Xueyang Wang, David S. Friedman, Christopher J. Brady

Publisher: Springer US


Purpose of Review

As the number of people with diabetic retinopathy (DR) in the USA is expected to increase threefold by 2050, the need to reduce health care costs associated with screening for this treatable disease is ever present. Crowdsourcing and automated retinal image analysis (ARIA) are two areas where new technology has been applied to reduce costs in screening for DR. This paper reviews the current literature surrounding these new technologies.

Recent Findings

Crowdsourcing has high sensitivity for normal vs abnormal images; however, when multiple categories for severity of DR are added, specificity is reduced. ARIAs have higher sensitivity and specificity, and some commercial ARIA programs are already in use. Deep learning enhanced ARIAs appear to offer even more improvement in ARIA grading accuracy.


The utilization of crowdsourcing and ARIAs may be a key to reducing the time and cost burden of processing images from DR screening.
Russell S, Norvig P. Artificial intelligence: a modern approach. 3rd ed. Essex: Pearson; 2009.
Brabham DC, Ribisl KM, Kirchner TR, Bernhardt JM. Crowdsourcing applications for public health. Am J Prev Med. 2014;46:179–87.CrossRefPubMed
Brabham DC. Crowdsourcing. Massachusetts: MIT Press; 2013.
Brinkmann BH, Wagenaar J, Abbot D, Adkins P, Bosshard SC, Chen M, et al. Crowdsourcing reproducible seizure forecasting in human and canine epilepsy. Brain. 2016;139(Pt 6):1713–22.CrossRefPubMedPubMedCentral
•• Brady CJ, Villanti AC, Pearson JL, et al. Rapid grading of fundus photographs for diabetic retinopathy using crowdsourcing. J Med Internet Res. 2014;16(10):e233. This is the first paper describing the use of crowdsourcing specifically for grading images for diabetic retinopathy CrossRefPubMedPubMedCentral
Mitry D, Peto T, Hayat S, et al. Crowdsourcing as a novel technique for retinal fundus photography classification: analysis of images in the EPIC Norfolk Cohort on behalf of the UKBiobank eye and vision consortium. PLoS One. 2013;8(8):e71154.CrossRefPubMedPubMedCentral
•• Mitry D, Zutis K, Dhillon B, Peto T, Hayat S, Khaw KT, et al. The accuracy and reliability of crowdsource annotations of digital retinal images. Transl Vis Sci Technol. 2016;5(5):6. Erratum in: Transl Vis Sci Technol. 2016;5(6):9. This is the most recent evaluation of crowdsourcing for using in grading retinal images for pathology. It is also the first paper to report using crowdsourcing to annotate retinal images with areas of pathology associated with diabetic retinopathy CrossRefPubMedPubMedCentral
Mitry D, Peto T, Hayat S, et al. Crowdsourcing as a screening tool to detect clinical features of glaucomatous optic neuropathy from digital photography. PLoS One. 2015;10(2):e0117401.CrossRefPubMedPubMedCentral
Wang X, Mudie LI, Baskaran M, Cheng CY, Alward WL, Friedman DS, et al. Crowdsourcing to evaluate fundus photographs for the presence of glaucoma. J Glaucoma. 2017.
de Alfaro L, Shavlovsky M. Crowdsourcing quantitative evaluation: algorithms and empirical results. Technical Report UCSC-SOE-14-03, School of Engineering, UC Santa Cruz. https://​pdfs.​semanticscholar.​org/​cdc8/​5dda05559884af5b​05f5015288cff5fb​aaab.​pdf. Accessed 27 Apr 2017.
Katz N, Goldbaum M, Nelson M, et al. An image processing system for automatic retina diagnosis. SPIE. 1988;902:131–7. https://​doi.​org/​10.​1117/​12.​944774.
Yamamoto S, Yokouchi H. Automatic recognition of color fundus photographs. In: Preston K, Onoe M, editors. Digital processing of biomedical images [internet]. Boston: Springer US; 1976. p. 385–98. https://​doi.​org/​10.​1007/​978-1-4684-0769-3_​24.CrossRef
Trucco E, Ruggeri A, Karnowski T, Giancardo L, Chaum E, Hubschman JP, et al. Validating retinal fundus image analysis algorithms: issues and a proposal. Invest Ophthalmol Vis Sci. 2013;54(5):3546–59. https://​doi.​org/​10.​1167/​iovs.​12-10347.CrossRefPubMedPubMedCentral
Walton OB 4th, Garoon RB, Weng CY, Gross J, Young AK, Camero KA, et al. Evaluation of automated teleretinal screening program for diabetic retinopathy. JAMA Ophthalmol. 2016;134(2):204–9. https://​doi.​org/​10.​1001/​jamaophthalmol.​2015.​5083.CrossRefPubMed
Sim DA, Keane PA, Tufail A, Egan CA, Aiello LP, Silva PS. Automated retinal image analysis for diabetic retinopathy in telemedicine. Curr Diab Rep. 2015;15(3):14. https://​doi.​org/​10.​1007/​s11892-015-0577-6.CrossRefPubMed
Tufail A, Rudisill C, Egan C, Kapetanakis VV, Salas-Vega S, Owen CG, et al. Automated diabetic retinopathy image assessment software: diagnostic accuracy and cost-effectiveness compared with human graders. Ophthalmology. 2017;124(3):343–51. https://​doi.​org/​10.​1016/​j.​ophtha.​2016.​11.​014.CrossRefPubMed
MacGillivray TJ, Cameron JR, Zhang Q, El-Medany A, Mulholland C, Sheng Z, et al. Suitability of UK biobank retinal images for automatic analysis of morphometric properties of the vasculature. PLoS One. 2015;10(5):e0127914. https://​doi.​org/​10.​1371/​journal.​pone.​0127914. eCollection 2015CrossRefPubMedPubMedCentral
•• Gulshan V, Peng L, Coram M, Stumpe MC, Wu D, Narayanaswamy A, et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA. 2016;316(22):2402–10. https://​doi.​org/​10.​1001/​jama.​2016.​17216. This high-profile paper described the use of deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs, and has a good description of deep learning for a general medical audience CrossRefPubMed
Abràmoff MD, Lou Y, Erginay A, Clarida W, Amelon R, Folk JC, et al. Improved automated detection of diabetic retinopathy on a publicly available dataset through integration of deep learning. Invest Ophthalmol Vis Sci. 2016;57(13):5200–6. https://​doi.​org/​10.​1167/​iovs.​16-19964.CrossRefPubMed
Oliveira CM, Cristovao LM, Ribeiro ML, Abreu JR. Improved automated screening of diabetic retinopathy. Ophthalmologica. 2011;226(4):191–7. https://​doi.​org/​10.​1159/​000330285.CrossRefPubMed
Philip S, Fleming AD, Goatman KA, Fonseca S, McNamee P, Scotland GS, et al. The efficacy of automated “disease/no disease” grading for diabetic retinopathy in a systematic screening programme. Br J Ophthalmol. 2007;91(11):1512–7. https://​doi.​org/​10.​1136/​bjo.​2007.​119453.CrossRefPubMedPubMedCentral
Bhaskaranand M, Ramachandra C, Bhat S, Cuadros J, Nittala MG, Sadda S, et al. Automated diabetic retinopathy screening and monitoring using retinal fundus image analysis. J Diabetes Sci Technol. 2016;10(2):254–61. https://​doi.​org/​10.​1177/​1932296816628546​.CrossRefPubMedPubMedCentral

Be confident that your patient care is up to date

Medicine Matters is being incorporated into Springer Medicine, our new medical education platform. 

Alongside the news coverage and expert commentary you have come to expect from Medicine Matters diabetes, Springer Medicine's complimentary membership also provides access to articles from renowned journals and a broad range of Continuing Medical Education programs. Create your free account »