What's in a simple line drawing? Quite a lot, our brains say

Researchers found that viewing a "beach" scene depicted in a line drawing activated nearly the same patterns of brain activity in study participants as did viewing an actual color photograph of a beach. The same was true when people viewed line drawings and photographs of other natural scenes including city streets, forests, highways, mountains and offices.

Even when researchers removed up to 75 percent of the pixels in a line drawing, people still did better than chance at determining what the lines represented -- as long as the remaining lines showed the broad contours of the scene.

"Our results suggest that our brains can recreate whole detailed scenes from just a few lines, said Dirk Bernhardt-Walther, lead author of the study and assistant professor of psychology at Ohio State University.

"The representations in our brain for categorizing these scenes seem to be a bit more abstract than some may have thought -- we don't need features such as texture and color to tell a beach from a street scene," he said.

Walther conducted the study with Barry Chai and Li Fei-Fei of Stanford University and Eamon Caddigan and Diane Beck with the University of Illinois. Their results appear in the online early edition of the Proceedings of the National Academy of Sciences.

For the study, 10 participants viewed color photographs and line drawings of six categories of scenes -- beaches, city streets, forests, highways, mountains and offices -- while their brains were scanned using functional magnetic resonance imaging (fMRI).

The fMRI images showed the researchers what was going on in several areas of the participants' brains when they viewed the photos and line drawings. The most significant results occurred in the parahippocomal place area (PPA), an area of the brain that scientists know plays an important role in the encoding and recognition of scenes (rather than faces or objects).

Using the data from when participants viewed the color photos, the researchers trained a software-based decoder to tell what type of scene the participants viewed -- a beach, mountain, etc., -- based on the patterns of brain activity in the PPA revealed in the fMRI.

The decoder was far from perfect, but it did better than chance at predicting what scene a person was viewing in a particular fMRI image.

Most importantly, the decoder could do just as well at predicting which scene a person viewed when it was focused on line drawings as it was on photographs. In fact, the decoder did slightly better -- although not significantly so -- at predicting line drawings compared to photographs in the primary visual cortex.

"We expected that line drawings would be good enough to allow some decoding, but it was surprising that there was no benefit to photographs -- the decoder was no better when it was used on photos than it was on line drawings," Walther said.

Findings showed that when the decoder was trained on photographs, it still did equally well at predicting which scenes people were viewing in line drawings, and vice versa.

"That suggests the brain uses the same information to decode which scene it is viewing when it is presented with line drawings or photos," he said.

In addition, results showed that when the decoder did make errors, it made similar errors in both photographs and line drawings. For example, if the decoder thought people were looking at a photo of a mountain when they were really looking at a photo of a forest, it would make the same mistake when it was analyzing line drawings.

"The patterns of error match incredibly well, so that's an additional piece of evidence that the representations for photos and line drawings are very similar in the brain," Bernhardt-Walther said.

But what is it about line drawings that allow people to recognize what they represent? As part of the study, the researchers removed some of the lines in the line drawings and asked participants if they could still tell what scene was depicted. In some cases, they removed up to 75 percent of the pixels in the drawing.

If the researchers left the long contours in the drawings, which represented global structure -- such as sky, water or sand -- participants could still correctly predict what kind of scene was depicted about 60 percent of the time.

However, when researchers took out these long contours and left only short ones -- representing details like leaves, windows in buildings or individual ridges in a mountainside -- the accuracy of participants went way down.

These findings cast doubt on some models of human visual perception which argue that people need specific information that is found in photographs -- such as color, shading and texture -- to classify a scene.

"Of course, we use the rich sources of information found in a photograph when it is available, but the brain is an opportunist -- it uses what is available," Walther said. "We can get a lot of information from a line drawing."

The results also suggest why line drawings have played such an important role in human history, both as an art form and a way of presenting information simply.

"Imagine the astonishment of early man when he discovered he could draw shapes on a rock wall and it resembled the actual animal he had just killed. Line drawings have been with us since prehistoric times," Walther said.

The research was funded with a grant from the National Institutes of Health.