Facial analysis improves diagnosis

The majority of rare diseases are genetic. The underlying hereditary mutations often cause varying degrees of impairment in different areas of the body. In most cases, these hereditary changes are also expressed by characteristic facial features: for example, because eyebrows, the base of the nose or the cheeks are shaped in a distinctive way. However, this varies from disease to disease. Artificial intelligence (AI) uses these facial characteristics, calculates the similarities, and automatically links them to clinical symptoms and genetic data of patients. "The face provides us with a starting point for diagnosis," says Tzung-Chien Hsieh of Krawitz's team. "It is possible to calculate what the disease is with a high degree of accuracy."

"GestaltMatcher" requires only a few patients

The AI system "GestaltMatcher" described in the current publication is a continued development of "DeepGestalt," which the IGSB team trained with other institutions a few years ago. While DeepGestalt still required about ten non-related affected persons as a reference for training, its successor "GestaltMatcher" requires significantly fewer patients for feature matching. This is a great advantage in the group of very rare diseases, where only a few patients are reported worldwide. Furthermore, the new AI system also considers similarities with patients who have also not yet been diagnosed, and thus combinations of characteristics that have not yet been described. GestaltMatcher therefore also "recognizes" diseases that were previously unknown to it and suggests diagnoses based on this. "This means we can now classify previously unknown diseases, search for other cases and provide clues as to the molecular basis," says Krawitz.

The team used 17,560 patient photos, most of which came from digital health company FDNA, which the research team worked with developing the web service through which the AI can be used. Around 5,000 of the photos and patient data were contributed by the research team at the Institute of Human Genetics at the University of Bonn, along with nine other university sites in Germany and abroad. The researchers focused on disease patterns that were as diverse as possible. They were able to consider a total of 1,115 different rare diseases. "This wide variation in appearance trained the AI so well that we can now diagnose with relative confidence even with only two patients as our baseline at best, if that's possible," Krawitz says.

"We are very happy to finally have a phenotype analysis solution for the ultra-rare cases, which can help clinicians solve challenging cases, and researchers to progress rare disease understanding," says Aviram Bar-Haim of FDNA Inc. in Boston, USA. In Germany, too, the application in doctors' offices, for example, is not far off, adds Krawitz. Doctors can already use their smartphones to take a portrait photo of a patient and use AI to make differential diagnoses, he says. "GestaltMatcher helps the physician make an assessment and complements expert opinion."

Peter Krawitz and his team turned over the data they collected themselves to the non-profit Association for Genome Diagnostics (AGD), to provide researchers with access. "The GestaltMatcher Database (GMDB) will improve the comparability of algorithms and provide the basis for further development of artificial intelligence for rare diseases, including other medical image data such as X-rays or retinal images from ophthalmology," Krawitz says.

Participating institutions and funding:

In addition to the Institute for Genomic Statistics and Bioinformatics and the Institute of Human Genetics of the University Hospital Bonn, the Charité-Universitätsmedizin Berlin, the universities of Greifswald, Tübingen, Düsseldorf, Lübeck, Heidelberg, the Technical University of Munich as well as universities from South Africa, France, the USA and Norway were involved. The study was mainly funded by the German Research Foundation (DFG).