How your brain learns to ride the subway, and why AI developers care

"The idea is basically to understand how humans or animals make long-term decisions," says Jan Balaguer, a PhD student at University of Oxford and member of Google DeepMind. "We're interested in trying to find machine-learning solutions to difficult tasks and real-life problems. Quite often it can be useful to draw inspiration from neuroscience."

Balaguer and his colleagues used a navigational game as a proxy to decode the human brain's decision-making processes. On a virtual subway system analogous to the London Underground, each station stop represented an individual step, while the different colored subway lines represented a higher level of the hierarchy. Twenty-two participants were trained on the game and then given a destination station as a goal, and played while in an fMRI scanner.

The researchers examined whether participants focused more on the subway lines or on the individual stations while navigating in the game. The team found that, generally, brain activity and response time increased with the number of line changes standing between participants and their destinations, rather than with the number of stations themselves. The areas in the brain linked to this type of decision making were the dorsal portion of the medial prefrontal cortex, which is known to support higher cognitive functions such as planning, and the premotor cortex, which is more involved in the execution of real or imaginary movements.

"We show, in a more straightforward and direct manner than previous studies, that there are hierarchical representations reflected in the brain," says Balaguer.

However, there were some parts of the brain that became more active as participants inched closer to achieving their goal, with fewer stations left on a single line: the ventromedial prefrontal cortex and the hippocampus. In previous work, the hippocampus has been shown to react to proximity to a given goal.

Overall, Balaguer says, "We want to see how the human brain implements things like hierarchical structures in order to design more clever algorithms. In machine learning, having a hierarchical representation for decision making might be helpful or harmful depending on whether you choose the right hierarchy to implement in the first place."