Adding human touch to unchatty chatbots may lead to bigger letdown

In a study, the researchers found that chatbots that had human features -- such as a human avatar -- but lacked interactivity, disappointed people who used it. However, people responded better to a less-interactive chatbot that did not have humanlike cues, said S. Shyam Sundar, James P. Jimirro Professor of Media Effects, co-director of the Media Effects Research Laboratory and affiliate of Penn State's Institute for CyberScience (ICS).

High interactivity is marked by swift responses that match a user's queries and feature a threaded exchange that can be followed easily, according to Sundar.

"People are pleasantly surprised when a chatbot with low anthropomorphism -- fewer human cues -- has higher interactivity," said Sundar. "But when there are high anthropomorphic visual cues, it may set up your expectations for high interactivity -- and when the chatbot doesn't deliver that -- it may leave you disappointed."

On the other hand, improving interactivity may be more than enough to compensate for a less-humanlike chatbot. Even small changes in the dialogue, like acknowledging what the user said before providing a response, can make the chatbot seem more interactive, said Sundar.

"In the case of the low-humanlike chatbot, if you give the user high interactivity, it's much more appreciated because it provides a sense of dialogue and social presence," said lead author of the study, Eun Go, a former doctoral student at Penn State and currently assistant professor in broadcasting and journalism, Western Illinois University.

Because there is an expectation that people may be leery of interacting with a machine, developers typically add human names to their chatbots -- for example, Apple's Siri -- or program a human-like avatar to appear when the chatbot responds to a user.

The researchers, who published their findings in Computers in Human Behavior, currently online, also found that just mentioning whether a human or a machine is involved -- or, providing an identity cue -- guides how people perceive the interaction.

"Identity cues build expectations," said Eun Go. "When we say that it's going to be a human or chatbot, people immediately start expecting certain things."

Sundar said the findings could help developers improve acceptance of chat technology among users. He added that virtual assistants and chat agents are increasingly used in the home and by businesses because they are convenient for people.

"There's a big push in the industry for chatbots," said Sundar. "They're low-cost and easy-to-use, which makes the technology attractive to companies for use in customer service, online tutoring and even cognitive therapy -- but we also know that chatbots have limitations. For example, their conversation styles are often stilted and impersonal."

Sundar added the study also reinforces the importance of high interactivity, broadly speaking.

"We see this again and again that, in general, high interactivity can compensate for the impersonal nature of low anthropomorphic visual cues," said Sundar. "The bottom line is that people who design these things have to be very strategic about managing user expectations."

The researchers recruited 141 participants through Amazon Mechanical Turk, a crowdsourced site that allows people to get paid to participate in studies. The participants signed up for a specific time slot and reviewed a scenario. They were told that they were shopping for a digital camera as a birthday present for a friend. Then, the participants navigated to an online camera store and were asked to interact with the live chat feature.

The researchers designed eight different conditions by manipulating three factors to test the user's reaction to the chatbot. The first factor is the identity of a chatbot. When the participant engaged in the live chat, a message appeared indicating the users was interacting either with a chatbot or a person. The second factor is the visual representation of a chatbot. In one condition, the chatbot included a humanlike avatar and in another, it simply had a speech bubble. Last, the chatbots featured either high or low interactivity when responding to participants, with the only difference being that a portion of the user's response was repeated in the high condition. In all cases, a human was interacting with the participant.

While this study was carried out online, the researchers said that observing how people interact with chatbots in a laboratory may be one possible step to further this research.