Google translates doctor's orders into Spanish and Chinese with few significant errors

But is Translate trustworthy?

Researchers at UC San Francisco say the answer is yes -- with some caveats. After analyzing 100 sets of emergency discharge instructions translated by Google's new machine learning algorithm, which was rolled out in 2017, the researchers concluded it was 92 percent accurate for Spanish and 81 percent accurate for Chinese.

The researchers found that only a small minority of the inaccurate translations -- 2 percent in Spanish and 8 percent in Chinese -- had the potential to cause clinically significant harm. These were mostly due to grammar or typographical errors in the original written English instructions, which someone who could read English would have been able to understand correctly.

"Google Translate is more accurate than a lot of clinicians believe, and I think it's definitely more useful than not providing anything at all," said Elaine Khoong, MD, MS, a UCSF primary care research fellow and first author of the study, published Feb. 25, 2019, in JAMA Internal Medicine. "We cautiously support its use."

The algorithm got into trouble, however, when doctors used colloquial terms, like "skip a meal," a phrase that Google translated into Chinese as "jump over" a meal.

A more troubling mistranslation occurred when the doctor told the patient to "hold the kidney medicine," meaning to stop taking it. The algorithm translated that into Spanish as, "keep the medication" and into Chinese as "keep taking" the medication, mistakes that the research team deemed "life threatening."

Machine translation errors were also linked to the use of medical jargon, and long, complicated sentences, such as, "Please return to the emergency department for worsening abdominal pain, inability to eat or drink due to vomiting, bloody diarrhea, if you pass out or any other concerning symptom."

Google's Chinese translation was, "If you pass, vomiting, bloody diarrhea, unable to eat or drink or any other symptoms; come back to the emergency department for treatment."

The researchers deemed this a clinically significant error, although they said it was not life threatening.

"It confused the machine because the sentence structure was so complicated," Khoong said. "It's certainly an argument for doctors to check themselves when they are writing instructions. We get used to communicating in these very complicated sentences."

The researchers said Google Translate is best used in combination with human interpreters, even if they are not in the room, who can interpret the physician's verbal instructions for the patient, over the phone or via video, while the physician shows the patient the written translation on the computer screen or printout. That way, patients can flag errors or confusing passages as they are listening and reading.

The team also recommended that clinicians give their patients the English version of the written instructions, so English-speaking family members or friends can compare them to the machine translation.

Khoong said patients would be worse off with no written translation at all. She described a non-English speaking patient with a neck injury, who left the emergency department with a cervical collar but did not understand the need to put it on.

The patient later showed up with the collar in one hand, and the English discharge instructions in the other, saying, "I know these are important. Can you tell me what they say?"

Authors are Elaine Khoong, MD, MS; Eric Steinbrook; Cortlyn Brown, MD; and Alicia Fernandez, MD, all of UCSF.

The study was funded by the National Institute of Diabetes and Digestive and Kidney Diseases (K24DK102057) and a National Research Service Award (T32HP19025).