'Striosome' neurons in the basal ganglia play a key role in learning

Learning by trial-and-error, which can yield positive or negative consequences, is known as reinforcement learning. Individuals can learn new behaviors in unfamiliar environments by exploring and memorizing sensory cues or actions that lead to good or bad outcomes.

Neuroscientists know that a part of the forebrain called the basal ganglia plays an important role in reinforcement learning. A major part of the basal ganglia, the striatum, is composed of a patchwork of two types of tissue: the striosome and the matrix. However, even though they were discovered three decades ago, the distinct roles that these regions play has remained a mystery.

Now, researchers from the Neural Computation Unit at the Okinawa Institute of Science and Technology Graduate University (OIST) have used the latest optical neural imaging technology to isolate and record the activity of the neurons in the striosome, shedding light on the role of striosomal neurons in reinforcement learning. Their work was recently published in the journal eNeuro.

Neuroscientists have long supposed that the neurons of the striosome are involved in predicting when the outcome of a stimulus will be positive, a process called reward prediction. This is because these neurons connect to neurons in the midbrain that produce an important signaling chemical called dopamine, which regulates reward-motivated behavior in the vertebrate brain.

"Reward prediction is important to our daily lives," says Professor Kenji Doya, leader of the Neural Computation Unit, "for example, when you find your favorite dish on a display or a menu, you can get excited even before actually eating it and make a choice accordingly."

The role of the striosomal neurons in reward prediction had not been confirmed because their organization in the brain makes it very difficult to distinguish them and record their activities. "The striosomal neurons comprise only 15% of the striatum and they are scattered mosaically within it, making it hard to isolate them," explains Tomohiko Yoshizawa, OIST technician and first author of the paper.

The OIST researchers overcame this problem by using a special imaging technique and gene manipulation technology. They worked with transgenic mice which express specific genes only in the striosomal neurons. The genes code for calcium indicators which glow under fluorescent light when the neurons become active. Imaging a deep brain area like the striatum previously required removing a large portion of the brain above that area. However, by using a new endoscopic microscope developed by a spin-off company from Stanford University, which uses a thin glass rod as its lens, the researchers could record the activity of the striosomal neurons in a minimally invasive way. Using this imaging technique with the transgenic mice, they were able to observe striosomal neurons for long periods of time and to measure their activity while the mice performed specific tasks.

In these tasks, the researchers presented the mice with four different odors: banana, lemon, cinnamon and mint. Each odor was linked to a specific outcome -- lots of water (a big reward), a little water (a small reward), an air puff to the face (a negative reward) or nothing. The mice started licking the water spout as they anticipated a water reward from specific odors, even before water was delivered.

The researchers showed that, as mice learned the task, the striosomal neurons fired more actively in response to odor cues associated with water rewards, meaning they were involved in anticipating the outcome of the stimulus. These activities were proportional to the expected consequence -- anticipation of a larger reward triggered a greater response in the neuron.

By measuring the activities of striosomal neurons every day for several weeks, the researchers showed that these brain cells are most active in early stages of learning, after about one week, or during late stages, after about two weeks. This means that the predictive activities of striosomal neurons are learning-stage specific.

The striosomal neurons were also active when the water or air puff was actually presented. This means that, in addition to transmitting signals about expected rewards, they also transmit information about the actual rewards.

"We can learn a lot from studying mice because the mosaic design of the striosomes and the matrix is the same in both the mouse brain and the human brain," says Yoshizawa.

Understanding the specific role of striosomal neurons could one day help researchers to diagnose and treat disorders caused by signaling problems in this region of the brain, such as Huntingdon's disease.