In The Mind's Eye: How The Brain Makes A Whole Out Of Parts

A team from the university's Zanvyl Krieger Mind/Brain Institute describes the complex but speedy process in detail in a recent issue of the journal Neuron.

The question of how the brain sees, recognizes and understands objects is one of the most intriguing in neuroscience, associate professor and paper co-author Charles E. Connor said.

"This may not even seem like a scientific question to some people, because seeing is so automatic and we are so good at it – far better than the best computer vision systems yet devised," Connor said. "That is because a large part of the human brain is devoted to interpreting objects in our world, so that we have the necessary information for interacting with our environment.

"Vision doesn't happen in the eye," Connor said. "It happens at multiple processing stages in the brain. We study how objects are signaled or encoded by large populations of neurons at higher-level stages in the object-processing part of the brain."

The report, based on recordings of nerve cells in the visual cortex of macaque monkeys, reveals that neurons in the higher-level visual cortex at first respond to a visual stimulus "somewhat indiscriminately," signaling all the individual features within a shape to which they are sensitive. For instance, a particular neuron may respond to objects with either a concave fragment at the top or a convex fragment at the bottom. At this point, the neural signals are ambiguous; the brain doesn't know whether the concavity, the convexity or both are present.

Milliseconds later, however, neurons begin to react exclusively to combinations of shape fragments, rather than to individual fragments. In other words, the brain begins to put the pieces together to form larger sections, in the same way that an artisan might fasten discrete shards of stained glass to create a design.

"Humans do a rough categorization of objects very quickly," Connor said. "For instance, in just a tenth of a second, we can recognize whether something we see is an animal or not. Our results show that this immediate, rough impression probably depends on recognizing just one or more individual parts of what we see. Fine discriminations – such as recognizing individual faces – take longer to happen, and our study suggests that this delay depends upon emerging signals for combinations of shape fragments. In a sense, the brain has to construct an internal representation of an object from disparate pieces."

In the long term, understanding exactly how the brain processes information may lead to neural prostheses – artificial replacements for lost sensory, motor and perhaps even memory and cognitive functions. In the short term, such work is driven by curiosity about one of the fundamental mysteries: how the brain works.

"Our ability to see is one of the great evolutionary accomplishments of the human brain," Connor said. "We still don't know how the visual system accomplishes this marvel of information processing. Such experiments are beginning to reveal how large networks of neurons in the brain extract meaning from the eye image."

Funding for this research was provided by the National Institutes of Health. The article appeared in the Jan. 5 issue of Neuron.