New AI tool guides users away from incendiary language

The research shows promising signs that conversational forecasting methods within the field of natural language processing could prove useful in helping both moderators and users proactively lessen vitriol and maintain healthy, productive debate forums.

"Well-intentioned debaters are just human. In the middle of a heated debate, in a topic you care about a lot, it can be easy to react emotionally and only realize it after the fact," said Jonathan Chang, a doctoral student in the field of computer science. The idea is not to tell users what to say, Chang said, but to encourage users to communicate as they would in-person.

The tool, named ConvoWizard, is a browser extension powered by a deep neural network. That network was trained on mountains of language-based data pulled from the subreddit Change My View, a forum that prioritizes good faith debates on potentially heated subjects related to politics, economics and culture.

When participating Change My View users enable ConvoWizard, the tool can inform them when their conversation is starting to get tense. It can also inform users, in real-time as they are writing their replies, whether their comment is likely to escalate tension. The study suggests that AI-powered feedback can be effective in guiding the user toward language that elevates constructive debate, researchers said.

"ConvoWizard is basically asking, 'If this comment is posted, would this increase or decrease estimated tension in the conversation?' If the comment increases tension, ConvoWizard would give a warning," Chang said. The textbox would turn red, for example. "The tool toes this line of giving feedback without veering into the dangerous territory of telling them to do this or that."

To test ConvoWizard, Cornell researchers collaborated with the Change My View subreddit, where roughly 50 participating forum moderators and members put the tool to use. Findings were positive: 68% felt the tool's estimates of risk were as good as or better than their own intuition, and more than half of participants reported that ConvoWizard warnings stopped them from posting a comment they would have later regretted.

Chang also noted that, prior to using ConvoWizard, participants were asked if they ever posted something they regretted. More than half said yes.

"These findings confirm that, yes, even well-intentioned users can fall into this type of behavior and feel bad about it," he said.

"It's exciting to think about how AI-powered tools like ConvoWizard could enable a completely new paradigm for encouraging high-quality online discussions, by directly empowering the participants in these discussions to use their own intuitions, rather than censoring or constraining them," said Cristian Danescu-Niculescu-Mizil, associate professor of information science.