Why a whiff of cats or rats is scary: Chemical signal from predators sparks fear in mice

The research was published in the journal Cell on May 14, 2010.

"We're interested in how the brain can be hardwired to respond to chemical signals and how this can lead to complex behaviors," said Scripps Research Associate Professor Lisa Stowers. "Our latest research helps shed light on how this brain circuits work."

Mice -- even those that have never before encountered other species -- will act fearfully when exposed to the odor of many different kinds of predators, including cats, rats, snakes, ferrets, weasels, and foxes . This is the first time that scientists have been able to identify the distinct chemical signal evoking this response and understand how mice are able to detect this signal.

The Smell of Fear

Identifying the chemical pathway of signals that make their way through the neurological system is not easy. One of the challenges for scientists studying brain circuits is that the brain is constantly changing through learning and memory. How a brain detects and then responds to the scent of a particular food, for instance, evolves as the animal learns about that food.

But certain behaviors tend to be the same each time they are triggered, suggesting a steady pathway through neurological circuits. The Stowers group has focused its research program on understanding a number of these steady pathways involving smell -- specifically, chemical cues that are released into the air, secreted from glands, or excreted in urine and picked up by members of the same and different species. Two years ago, the group published research identifying the pheromone that provokes male-male aggression in mice. The scientists hope eventually to compare and contrast different chemical signals to uncover the general principals of brain response.

In the current study, Stowers and her colleagues investigated the fearful response that mice were known to have to odors emitted by predators.

"It's really interesting behavior," said Stowers. "These mice have been inbred in the lab since the 1930s. For hundreds of generations, they haven't been around any other species. It's really remarkable that they haven't lost this circuit."

To begin, the team addressed the issue of how mice detected these chemical cues from other species. Mice have two sensory organs -- the vomeronasal organ (VNO), which is located above the roof of the mouth in the nasal cavity, and the main olfactory epithelium (MOE), found under the eyeball at the top back portion of the nasal cavity. Which was responsible for picking up the scent?

To answer this question, the team compared the behavior of normal (wildtype) mice with mice with a genetic mutation that left them without functioning vomeronasal organs.

When the regular mice were put in a cage with a cotton ball swabbed with rat, cat, or snake odor, the mice reacted fearfully -- staying away from the cotton ball, repeatedly striking a cautious posture (nose in the air and ready to run), and exhibiting elevated levels of stress hormones. In contrast, the mice without functioning vomeronasal organs appeared to be curious about the cotton balls and demonstrated few signs of fear. In fact, the behavior of the mutant mice was comparable to that of normal mice presented with a cotton ball swabbed with a control odor (such as that from a rabbit or another mouse). From these experiments, the scientists were able to determine that the vomeronasal organ was essential for detecting chemical cues from predators.

But the team didn't stop there. To see just how important the olfactory system was in determining this behavior, the team put a series of mice in a cage with an anesthetized live rat. In contrast to normal mice, those without vomeronasal organs showed few signs of fear.

"We did the experiment with quite a few different animals," said Stowers, "and one of them actually curled up and slept right next to the rat! So even though these mice can touch the rat and see it breathing, without the vomeronasal organ system mice don't respond fearfully."

Imaging experiments corroborated the behavioral findings by showing that specific neurons fired in brains of mice responding fearfully, and did not in those failing to demonstrate these behaviors.

Chemical Signals

Next, the team set out to find the exact chemical that signaled to mice that they should be fearful.

"Nobody in the field was looking for the cues because they thought it would be a [chemical] blend and purifying a blend is really hard," said Stowers. "In insects, a lot of pheromones are known to be a blend of two or three molecules. It was postulated that it would be even more complex in mammals."

But the team proceeded anyway, confident in the fact that Scripps Research -- with its top-ranked chemistry faculty -- was an ideal place to confront such challenges.

Using rat urine as a starting point, step by step the team marched its way toward isolating the chemical signal. First, the researchers divided the urine into low and high molecular weight compounds, finding that only the high molecular weight portion was able to evoke the fearful response from mice. Then, the team traced the activity to a specific peak on the mass spectrometry analysis, which pointed to a single protein, rather than a blend. To verify the results, the team made a recombinant version of the protein of interest, which by itself was able to elicit the fearful behavior in mice and activate corresponding neurons in the mouse brain.

Surprisingly, the protein the scientists identified belonged to the major urinary protein (MUP) family. This was a member of the same family of proteins the Stowers lab had previously identified as provoking male-male aggression in mice.

"I was shocked when we found another MUP," said Stowers. "I thought, 'We've looked at two behaviors and we're getting the same thing! That can't be right!' Yet the data was just so convincing…"

To verify and extend their findings, the team moved on to analyzing the odor emitted by cats. Working with cat saliva, the team found that a key protein in cat saliva -- Feld4, the cat version of MUP -- provoked the same fearful response from the mice and activated defensive neurons in the mouse brain. In further experiments, the scientists showed that predator MUPs and mouse MUPs activate different subsets of neurons in the mouse brain.

"Although I was at first hesitant that we had found the same molecule again, evolutionarily it is actually quite exciting," said Stowers. "The order of the MUP receptors suggests that mice were first able to detect MUPs from other species, then evolved the ability to detect signals from other mice."

The first authors of the paper are Fabio Papes and Darren Logan of Scripps Research. The research was supported by the National Institute on Deafness and Other Communication Disorders of the National Institutes of Health, the Skaggs Foundation, and a young investigator grant from the Sao Paulo Research Association (FAPESP).