'Mixed Reality' Human Helps Medical Students Learn To Do Intimate Exams

“Part of my left breast has been painful for awhile.”

“Can you lie down so that I can examine you?”

It sounds like a snippet of conversation between doctor and patient. But the doctor, in this recent exchange at the University of Florida campus, was actually an engineering doctoral student — and the patient a “mixed reality human” composed of a life-sized computer avatar on a flat screen and a mannequin with a prosthetic breast.

Intimate procedures such as breast exams, while a routine and critical part of medical care, are notoriously tough to teach. Medical students practice on disembodied prosthetics but have limited opportunities to practice exams on real people — especially patients who have an abnormality. In a collaboration with the Augusta, Ga.-based Medical College of Georgia and three other universities, UF engineers have crafted a solution: a hybrid computer/mannequin that helps train students not only how to correctly perform a breast exam — but also how to talk to, and glean information from, the patient during the procedure.

The project is important because correct examinations and good doctor-patient communication are critical to successful medical treatment, said Benjamin Lok, a UF assistant professor of computer and information sciences and engineering who heads the effort.

“Studies have shown that communication skills are actually a better predictor of outcome than medical skills,” Lok said. With the virtual patient, “all of a sudden, students have to not only practice their technique, but they also have to work on their empathy.”

The mixed reality human, named Amanda Jones, “talks” to students, and they respond via a computer speech and voice recognition system tailored by doctoral student Aaron Kotranza, Lok and others on the team. Her physical form — a mannequin — is immobile, but her virtual representation, created by the engineers, moves and speaks from a large flat screen above her physical body. Students can also view Jones through a head-mounted display.

The interaction is unscripted, but it follows a typical pattern for a woman’s visit and examination — with both verbal and tactile challenges for the medical students.

The student must tease out Jones’ medical history, listen to her concerns and respond to her questions. Just as in a real exam, this interaction occurs simultaneously with the physical examination. For that, the student must use the correct palpitating technique and apply the proper pressure. Sensors within the prosthetic breast — developed by Dr. Carla Pugh at Northwestern University — provide pressure information depicted by colors on the virtual breast, guiding students in the exams. The engineers can program the system to include or exclude an abnormality — and the attendant conversation.

It sounds awkward, and to be sure, the speech recognition element has its hiccups.

But especially for students reared in an era of sophisticated three-dimensional video games, the system turns out to be surprisingly convincing. The researchers have tested it on about 100 medical students so far, all from the Medical College of Georgia, where co-principal investigator Dr. D. Scott Lind is based. One of their most consistent and prominent findings: Students do not hesitate to express empathy to Jones.

“We have found that they will try to comfort the virtual human,” Kotranza said. “They’ll often touch the mannequin in order to comfort her.”

A pilot study has concluded that students who practiced with a mixed realty human improved their communication skills and their technical abilities, but more trials are needed to determine whether those skills persist once the students examine real patients.

That said, it seems obvious that more practice students get, the better off they will be. Lok said the mixed reality patient is not intended to replace real volunteers - far from it. But students typically have only a handful of opportunities with those volunteers before graduating. The mixed reality patient can add to their training while making it easier for teachers to help students with both their conversational and medical techniques.

“What happens if you find something in a woman’s breast? How do you talk to the patient?” Lok asked. “Students have to somehow build their database of experience.”

While the breast exam research continues, the team also intends to explore other intimate exams. Next in line: prostate exams. Lok and the students already have prosthetics they intend to couple with a virtual male patient similar to the breast exam patient.