Natural language processing accurately tracks colonoscopy quality

Natural language processing, a linguistic technique using sophisticated software to extract meaning from written language, allows a computer to rapidly "read" and "understand" the free text of reports prepared by the gastroenterologists who perform colonoscopies and the free text reports of pathologists who analyze the composition of growths removed from the colon.

The ability of a computer to understand and extract meaning from human language has been part of science fiction for decades, familiar to anyone who has watched Captain James T. Kirk talk to the ship's computer on Star Trek. Only in recent years has research allowed computer software to catch up with fiction, as famously demonstrated when IBM's Watson computer was able to correctly understand and answer complicated questions on the TV game show Jeopardy. In this new study, Regenstrief and IU School of Medicine researchers evaluated the ability of natural language processing to interpret complex medical information related to colon cancer detection.

"We found that rapid and inexpensive natural language processing, which utilizes free-text data that was previously unusable for efficient computer-based analysis, was extremely accurate in measuring adenoma detection rate during colonoscopy," said Timothy Imler, M.D., a Regenstrief Institute investigator and Indiana University School of Medicine assistant professor of medicine in the Division of Gastroenterology and Hepatology, who led the study. "The presence of adenomas in the colon is predictive of a patient's risk of later developing colon cancer, and the detection rate has been identified as the critical measure of a high-quality endoscopist, the specialist who performs colonoscopy."

The study found that in spite of linguistic variation in the way providers at 13 medical centers across the country used words to express the same concept, 95 percent of the time computerized natural language processing was correct regarding type and location of an adenomatous polyp, compared to human expert reviewers who were assumed to be 100 percent correct. Thus, software can perform the same job in interpreting and correlating colonoscopy and pathology reports as humans, only much faster, reproducibly, and far less expensively.

Of more than 90,000 colonoscopies performed at the Albany, N.Y.; Ann Arbor, Mich.; Bronx, Denver, Detroit, Houston, Indianapolis, Nashville, New York Harbor; Northport, N.Y.; San Francisco, West Haven, Conn. and White River Junction, Vt. Veterans Affairs medical centers, slightly fewer than half had findings that generated pathology reports. Seven hundred and fifty paired colonoscopy and pathology reports from this group were randomly sampled and compared by human reviewers using 19 measurements related to quality and procedure interval determination. The remaining 41,819 paired reports were processed through natural language processing to access performance consistency using the same 19 measurements.

"Confirming that humans and the computer had similar assessments on procedures performed at facilities across the country gives us a powerful, scalable tool to assess quality and determine the appropriate interval between colonoscopies based on the procedure findings. Natural language processing will enable comparison of adenoma detection rates across populations to determine geographic, racial, ethnic or gender disparities," said Dr. Imler. "It also could be used across health systems or colonoscopy centers or doctors to enable referers or patients themselves to make informed decisions about where they wish to go for a colonoscopy."