AI approach outperformed human experts in identifying cervical precancer

To develop the method, researchers used comprehensive datasets to "train" a deep, or machine, learning algorithm to recognize patterns in complex visual inputs, such as medical images. The approach was created collaboratively by investigators at the National Cancer Institute (NCI) and Global Good, a project of Intellectual Ventures, and the findings were confirmed independently by experts at the National Library of Medicine (NLM). The results appeared in the Journal of the National Cancer Institute on January 10, 2019. NCI and NLM are parts of NIH.

"Our findings show that a deep learning algorithm can use images collected during routine cervical cancer screening to identify precancerous changes that, if left untreated, may develop into cancer," said Mark Schiffman, M.D., M.P.H., of NCI's Division of Cancer Epidemiology and Genetics, and senior author of the study. "In fact, the computer analysis of the images was better at identifying precancer than a human expert reviewer of Pap tests under the microscope (cytology)."

The new method has the potential to be of particular value in low-resource settings. Health care workers in such settings currently use a screening method called visual inspection with acetic acid (VIA). In this approach, a health worker applies dilute acetic acid to the cervix and inspects the cervix with the naked eye, looking for "aceto whitening," which indicates possible disease. Because of its convenience and low cost, VIA is widely used where more advanced screening methods are not available. However, it is known to be inaccurate and needs improvement.

Automated visual evaluation is similarly easy to perform. Health workers can use a cell phone or similar camera device for cervical screening and treatment during a single visit. In addition, this approach can be performed with minimal training, making it ideal for countries with limited health care resources, where cervical cancer is a leading cause of illness and death among women.

To create the algorithm, the research team used more than 60,000 cervical images from an NCI archive of photos collected during a cervical cancer screening study that was carried out in Costa Rica in the 1990s. More than 9,400 women participated in that population study, with follow up that lasted up to 18 years. Because of the prospective nature of the study, the researchers gained nearly complete information on which cervical changes became precancers and which did not. The photos were digitized and then used to train a deep learning algorithm so that it could distinguish cervical conditions requiring treatment from those not requiring treatment.

Overall, the algorithm performed better than all standard screening tests at predicting all cases diagnosed during the Costa Rica study. Automated visual evaluation identified precancer with greater accuracy (AUC=0.91) than a human expert review (AUC=0.69) or conventional cytology (AUC=0.71). An AUC of 0.5 indicates a test that is no better than chance, whereas an AUC of 1.0 represents a test with perfect accuracy in identifying disease.

"When this algorithm is combined with advances in HPV vaccination, emerging HPV detection technologies, and improvements in treatment, it is conceivable that cervical cancer could be brought under control, even in low-resource settings," said Maurizio Vecchione, executive vice president of Global Good.

The researchers plan to further train the algorithm on a sample of representative images of cervical precancers and normal cervical tissue from women in communities around the world, using a variety of cameras and other imaging options. This step is necessary because of subtle variations in the appearance of the cervix among women in different geographic regions. The ultimate goal of the project is to create the best possible algorithm for common, open use.