medwireNews: A deep learning algorithm is highly accurate for the detection of diabetic retinopathy in retinal fundus photographs, researchers report in JAMA.
In deep learning, the algorithm is not programmed with a specific set of rules, rather it is “trained” using a large existing dataset – in this case 128,175 retinal images that had each been graded for diabetic retinopathy, diabetic macular edema, and image quality by three to seven licensed ophthalmologists and ophthalmology senior residents from a panel of 54. The panel detected referable diabetic retinopathy in 28.1% of the 118,419 gradable images.
Thus trained, the algorithm detected referable diabetic retinopathy with 99.1% accuracy in 9963 images from 4997 patients in the EyePACS-1 dataset and with 99.0% accuracy in 1748 images from 874 patients in the Messidor-2 dataset.
Using a threshold to maximize specificity (ie, reduce false positives), the sensitivities and specificities were 90.3% and 98.1%, respectively, in EyePACS-1 and 87.0% and 98.5% in Messidor-2, report Lily Peng (Google Inc, Mountain View, California, USA) and co-workers.
With a threshold to maximize sensitivity, which reduces missed cases and more closely reflects screening parameters, the corresponding values were 97.5% and 93.4% for EyePACS-1 and 96.1% and 93.9% for Messidor-2. The algorithm had slightly lower, but still very high, accuracy for all-cause referrals (ie, retinopathy, macular edema, or ungradable image).
In one of three commentary articles accompanying the study, Tien Yin Wong (Singapore National Eye Centre) and Neil Bressler (Johns Hopkins University, Baltimore, Maryland, USA) say that these performance indicators are “substantially better” than what is generally advised for screening. But they caution that a different gold standard for diabetic retinopathy detection, such as standardized, centralized assessment or optical coherence tomography, might have resulted in less convincing sensitivities and specificities.
They also say it is critical for a screening algorithm to be able to detect sight-threatening diabetic retinopathy, and desirable for it to detect other eye conditions such as glaucoma or age-related macular degeneration.
All three commentaries address how artificial intelligence could facilitate screening programs, and impact the wider medical community. Wong and Bressler note that screening for diabetic retinopathy is widely recommended, yet underused. A part of this problem, they say, is the sheer scale of screening – an estimated 32 million retinal images would require evaluation annually in the USA.
But in their article, Andrew Beam and Isaac Kohane, both from Harvard Medical School in Boston, Massachusetts, USA, say that a deep learning system attached to an existing computer system could screen more than that in a single day, all for an initial hardware outlay of around US$ 1000 (€ 940).
“Given that artificial intelligence has a 50-year history of promising to revolutionize medicine and failing to do so, it is important to avoid overinterpreting these new results,” they write.
Nevertheless, they believe that “there are valid reasons to remain cautiously optimistic that the time could now be right for artificial intelligence to transform the clinic into a much higher-capacity and lower-cost information processing care service.”
medwireNews is an independent medical news service provided by Springer Healthcare Limited. © Springer Healthcare Ltd; 2016
See also third commentary – viewpoint article.