Even after learning the right idea, humans and animals still seem to test other approaches, study suggests

The behavior of "exploring" when one could just be "exploiting," could make sense for at least two reasons, said Mriganka Sur, senior author of the study published Feb. 18 in Current Biology. Just because a task's rules seem set one moment doesn't mean they'll stay that way in this uncertain world, so altering behavior from the optimal condition every so often could help reveal needed adjustments. Moreover, trying new things when you already know what you like is a way of finding out whether there might be something even better out there than the good thing you've got going on right now.

"If the goal is to maximize reward, you should never deviate once you have found the perfect solution, yet you keep exploring," said Sur, Paul and Lilah Newton Professor in The Picower Institute for Learning and Memory and the Department of Brain and Cognitive Sciences at MIT. "Why? It's like food. We all like certain foods, but we still keep trying different foods because you never know, there might be something you could discover."

Predicting timing

Former research technician Tudor Dragoi, now a graduate student at Boston University, led the study in which he and fellow members of the Sur Lab explored how humans and marmosets, a small primate, make predictions about event timing.

Three humans and two marmosets were given a simple task. They'd see an image on a screen for some amount of time -- the amount of time varied from one trial to the next within a limited range -- and they simply had to hit a button (marmosets poked a tablet while humans clicked a mouse) when the image disappeared. Success was defined as reacting as quickly as possible to the image's disappearance without hitting the button too soon. Marmosets received a juice reward on successful trials.

Though marmosets needed more training time than humans, the subjects all settled into the same reasonable pattern of behavior regarding the task. The longer the image stayed on the screen, the faster their reaction time to its disappearance. This behavior follows the "hazard model" of prediction in which, if the image can only last for so long, the longer it's still there, the more likely it must be to disappear very soon. The subjects learned this and overall, with more experience, their reaction times became faster.

But as the experiment continued, Sur and Dragoi's team noticed something surprising was also going on. Mathematical modeling of the reaction time data revealed that both the humans and marmosets were letting the results of the immediate previous trial influence what they did on the next trial, even though they had already learned what to do. If the image was only on the screen briefly in one trial, on the next round subjects would decrease reaction time a bit (presumably expecting a shorter image duration again) whereas if the image lingered, they'd increase reaction time (presumably because they figured they'd have a longer wait).

Those results add to ones from a similar study Sur's lab published in 2023 in which they found that even after mice learned the rules of a different cognitive task, they'd arbitrarily deviate from the winning strategy every so often. In that study, like this one, learning the successful strategy didn't prevent subjects from continuing to test alternatives, even if it meant sacrificing reward.

"The persistence of behavioral changes even after task learning may reflect exploration as a strategy for seeking and setting on an optimal internal model of the environment," the scientists wrote in the new study.

Relevance for autism

The similarity of the human and marmoset behaviors is an important finding as well, Sur said. That's because differences in making predictions about one's environment is posited to be a salient characteristic of autism spectrum disorders. Because marmosets are small, are inherently social, and are more cognitively advanced than mice, work has begun in some labs to establish marmoset autism models, but a key component was establishing that they model autism-related behaviors well. By demonstrating that marmosets model neurotypical human behavior regarding predictions, the study therefore adds weight to the emerging idea that marmosets can indeed provide informative models for autism studies.

In addition to Dragoi and Sur, other authors of the paper are Hiroki Sugihara, Nhat Le, Elie Adam, Jitendra Sharma, Guoping Feng and Robert Desimone.

The Simons Foundation Autism Research Initiative supported the research through the Simons Center for the Social Brain at MIT.