Picking the brains of strangers helps make sense of online information

This process of distributed sensemaking, they say, could save time and result in a better understanding of the information needed for whatever goal users might have, whether it is planning a vacation, gathering information about a serious disease or trying to decide what product to buy.

The researchers explored the use of digital knowledge maps -- a means of representing the thought processes used to make sense of information gathered from the Web. When participants in the study used a knowledge map that had been created and improved upon by several previous users, they reported that the quality of their own work was better than when they started from scratch or used a newly created knowledge map.

"Collectively, people spend more than 70 billion hours a year trying to make sense of information they have gathered online," said Aniket Kittur, assistant professor in Carnegie Mellon's Human-Computer Interaction Institute. "Yet in most cases, when someone finishes a project, that work is essentially lost, benefitting no one else and perhaps even being forgotten by that person. If we could somehow share those efforts, however, all of us might learn faster."

The research team, including Kristie Fisher and Scott Counts of Microsoft Research, will present their findings May 7 at CHI 2012, the Conference on Human Factors in Computing Systems, in Austin, Texas.

They recruited 21 Microsoft employees for the study, asking them to create knowledge maps on three different subjects, one of which was to be created from scratch, one based on a map previously created by one person and yet another based on a map that had iteratively been modified by four other users. The participants took the least time to generate knowledge maps when they based them on iterated maps and self-reported that the quality and helpfulness of those maps were superior to those developed from scratch or from a map generated by a single person.

In most cases, the organization of the knowledge map, rather than any specific content, was most useful. For instance, two people looking to start a garden might live in different climates or settings, so the types of seeds they might plant could be different, but each would benefit from elements such as "design ideas," "how to" and so on.

Using eye tracking, the researchers showed that as knowledge maps are modified successively by multiple users, new users spend less time looking at specific content elements, shifting a greater balance of their attention to structural elements like labels. "This suggests that distributed sensemaking facilitates the process of 'schema induction,' or forming a mental model of the information being considered," Counts said.

But this structure did not start to emerge until a map had been modified at least once. This would explain why participants favored the iterated maps over the others, Fisher said.

One problem for distributed sensemaking is overcoming the "first iteration hump," the researchers noted. Though the participants favored maps that had been improved by a succession of users, they also favored their maps created from scratch over those created by one other person. One way to get people to use newly created maps instead of making their own might be to pay them to modify another user's map, or to require users to integrate some "first-round" maps before they are given access to the highly iterated ones. It may also be possible to use automated methods to produce maps that look more like maps that have been revised by multiple users.

This research was sponsored by the National Science Foundation, Microsoft and the Center for the Future of Work at Carnegie Mellon University.