Artificial intelligence can deliver specialty-level diagnosis in primary care setting, study shows

The clinical trial, which also was the first study to prospectively assess the safety of an autonomous AI system in patient care, compared the performance of IDx-DR to the gold standard diagnostic for diabetic retinopathy, which is the leading cause of vision loss in adults and one of the most severe complications for the 30.3 million Americans living with diabetes.

IDx-DR exceeded all pre-specified superiority endpoints in sensitivity, the ability to correctly identify a patient with disease; specificity, the ability to correctly classify a person as disease-free; and imageability, or the capability to produce quality images of the retina and determine the severity of the disease.

"The AI system's primary role is to identify those people with diabetes who are likely to have diabetic retinopathy that requires further evaluation by an eye-care provider. The study results demonstrate the safety of autonomous AI systems to bring specialty-level diagnostics to a primary care setting, with the potential to increase access and lower cost," says Michael Abràmoff, MD, PhD, the Robert C. Watzke Professor of Ophthalmology and Visual Sciences with UI Health Care and principal investigator on the study. He is founder and president of IDx, the company that created the IDx-DR system and funded the study.

Early detection may prevent vision loss

More than 24,000 people in the U.S. lose their sight to diabetic retinopathy each year. Early detection and treatment can reduce the risk of blindness by 95 percent, but less than 50 percent of patients with diabetes schedule regular exams with an eye-care specialist.

In the study, 900 adult patients with diabetes -- but no history of diabetic retinopathy -- were examined at 10 primary care sites across the U.S. Retinal images of the patients were obtained using a robotic camera, with an AI assisting the operator in getting good quality images. Once the four images were complete, the diagnostic AI then made a clinical diagnosis in 20 seconds. The diagnostic AI detects disease just as expert clinicians do, by having detectors for the lesions characteristic for diabetic retinopathy, including microaneurysms, hemorrhages, and lipoprotein exudates.

Camera operators in the study were existing staff of the primary care clinics, but not physicians or trained photographers.

"This was much more than just a study testing an algorithm on an image. We wanted to test it in the places where it will be used, by the people who will use it, and we compared it to the highest standard in the world," says Abràmoff, who also holds faculty appointments in the UI College of Engineering.

AI measured against gold standard

Study participants also had retinal images taken at each of the primary care clinics using specialized widefield and 3D imaging equipment without AI operated by experienced retinal photographers certified by the Wisconsin Fundus Photograph Reading Center (FPRC) -- the gold standard in grading the severity of diabetic retinopathy.

Complete diagnostic data accomplished by both the AI system and FPRC readers was available for 819 of the original 900 study participants. FPRC readers identified 198 participants with more than mild diabetic retinopathy who should be further examined by a specialist; the AI was able to correctly identify 173 of the 198 participants with disease, resulting in a sensitivity of 87 percent. Among the 621 disease-free participants identified by FPRC readers, AI identified 556 participants, for a specificity of 90 percent. The AI had a 96 percent imageability rate: of the 852 participants who had an FPRC diagnosis, 819 had an AI system diagnostic output.

In June, following FDA clearance, providers at the Diabetes and Endocrinology Center at UI Health Care-Iowa River Landing in Coralville, Iowa, were the first in the nation to begin using IDx-DR to screen patients.

"We are hoping to do this also for early detection of diseases like glaucoma and macular degeneration. We are working on those algorithms already. The goal is to get these specialty diagnostics into primary care and retail, which is where the patients are," Abràmoff says.

IDx is working with the American Medical Association to ensure that there is clear coding guidance for billing of IDx-DR. Providers, physicians, and suppliers should contact their third-party payers for specific and current information on their coding, coverage, and payment policies. IDx is a licensed distributor of the robotic camera used in the study.

The paper, "Pivotal trial of an autonomous AI-based diagnostic system for detection of diabetic retinopathy in primary care offices," was co-authored by James Folk, MD, UI professor of ophthalmology and visual sciences and the Donald H. Beisner, MD, Professor of Vitreoretinal Diseases and Surgery, as well as an IDx board member. Also involved in the study were Philip Lavin at Boston Biostatistics Research Foundation, Inc, Michele Birch at University of North Carolina School of Medicine, and Nilay Shah at The Emmes Corporation, Rockville, Md.