Humans less likely to return to an automated advisor once given bad advice

Andrew Prahl and Lyn M. Van Swol (University of Wisconsin) will present their findings in June at the 66th Annual Conference of the International Communication Association in Fukuoka, Japan. The researchers' experiment asked participants to forecast scheduling for hospital operating rooms, a task they were unfamiliar with. In order to complete this task, participants were given help from either an "advanced computer system" or "a person experienced in operating room management." During the 7th trial of the 14 scheduling trials, participants were given bad advice from either the computer advisor or human advisor.

The researchers found that after participants received bad advice, they rapidly abandoned the computer advisor and did not use the advice on subsequent trials. This "punishment" for giving bad advice was not nearly as strong for the human advisor. It is almost as if people "forgave" the human advisor for making a mistake but did not extend the same feelings of forgiveness to the computer.

Past research has looked at trust of automation through simple tasks like warning systems. Prahl and Van Swol wanted to look at more sophisticated automation that actually makes predictions about future outcomes and compare that to human advice. Unlike previous research, the researchers did not find that humans or algorithmic advisors were generally trusted more, the only significant differences arose after the advisor provided bad advice.

"This has very important implications because time and time again we are seeing humans being replaced by computers in the workplace," said Prahl. "This research suggests that any potential efficiency gains by moving towards automation might be offset because all the automation has to do is err once, and people will rapidly lose trust and stop using it -- this is one of the few studies out there that really show the potential downsides of automation in the workplace."