Strategic or random? How the brain chooses

In the study, rats playing a game for a food reward usually acted strategically, but switched to random behavior when they confronted a particularly unpredictable and hard-to-beat competitor. The animals sometimes got stuck in a random-behavior mode, but the researchers, led by Janelia lab head Alla Karpova and postdoctoral fellow Gowan Tervo, found that they could restore normal behavior by manipulating activity in a specific region of the brain. Because the behavior of animals stuck in this random mode bears some resemblance to that of patients affected by a psychological condition called learned helplessness, the findingsmay help explain that condition and suggest strategies for treating it. Karpova, Tervo and their colleagues published their findings in the September 25, 2014, issue of the journal Cell.

The brain excels at integrating information from past experiences to guide decision-making in new situations. But in certain circumstances, random behavior may be preferable. An animal might have the best chance of avoiding a predator if it moves unpredictably, for example. And in a new environment, unrestricted exploration might make more sense than relying on an internal model developed elsewhere. So scientists have long speculated that the brain may have a way to switch off the influence of past experiences so that behavior can proceed randomly, Karpova says. But others disagreed. "They argue that it's inefficient, and that it would be at odds with what some people call one of the most central operating principles of the brain -- to use our past experience and knowledge to optimize behavioral choices," she notes.

Karpova and her colleagues wanted to see if they could create a situation that would force animals to switch into this random mode of behavior. "We tried to create a setting that would push the need to create behavioral variability and unpredictability to its extreme," she says. They did this by placing rats in a competitive setting in which a computer-simulated competitor determined which of two holes in a wall would provide a sugary reward. The virtual competitor, whose sophistication was varied by the experimenters, analyzed the rats' behavior to predict their future choices.

"We thought if we came up with very sophisticated competitors, then the animals would eventually be unable to figure out how to outcompete them, and be forced to either give up or switch into this [random] mode, if such a mode exists," Karpova says. And that's exactly what happened: When faced with a weak competitor, the animals made strategic choices based on the outcomes of previous trials. But when a sophisticated competitor made strong predictions, the rats ignored past experience and made random selections in search of a reward.

Now that they had evidence that the brain could generate both strategic and random behavior, Karpova and her colleagues wanted to know how it switched between modes. Since that switch determines whether or not an animal's internal model of the world influences its behavior, the scientists suspected it might involve a brain region called the anterior cingulate cortex, where that internal model is likely encoded.

They found that they could cause animals to switch between random and strategic behavior by manipulating the level of a stress hormone called norepinephrine in the anterior cingulate cortex. Increasing norepinephrine in the region activated random behavior and suppressed the strategic mode. Inhibiting release of the hormone had the opposite effect.

Karpova's team observed that animals in their experiments sometimes continued to behave randomly, even when such behavior was no longer advantageous. "If all they've experienced is this really sophisticated competitor for several sessions that thwarts their attempts at strategic, model-based counter-prediction, they go into this [random mode], and they can get stuck in it for quite some time after that competitor is gone," she says. This, she says, resembles the condition of learned helplessness, in which strategic decision-making is impaired following an experience in which a person finds they are unable to control their environment.

The scientists could release the animals from this "stuck" state by suppressing the release of norepinephrine in the anterior cingulate cortex. "Just by manipulating a single neuromodulatory input into one brain area, you can dramatically enhance the strategic mode. The effect is strong enough to rescue animals out of the random mode and successfully transform them into strategic decision makers," Karpova says. "We think this might shed light on what has gone wrong in conditions such as learned helplessness, and possibly how we can help alleviate them."

Karpova says that now that her team has uncovered a mechanism that switches the brain between random and strategic behavior, she would like to understand how those behaviors are controlled in more natural settings. "We normally try to use all of our knowledge to think strategically, but sometimes we still need to explore," she says. In most cases, that probably means brief bouts of random behavior during times when we are uncertain that past experience is relevant, followed by a return to more strategic behavior -- a more subtle balance that Karpova intends to investigate at the level of changes in activity in individual neural circuits.