Howler monkey study examines mechanisms of new species formation

How do new species emerge in nature? One common but overly simplified version of the story goes like this: A population of animals or plants becomes geographically isolated -- by a river that changes course or a mountain range that rises up, for example -- and the two separated groups accumulate genetic differences over time as they adapt to their environments in isolation.

Eventually, the DNA of the two groups is so different that the two populations are considered distinct species. Voilà, speciation has occurred.

In reality, the process is much more complex than that. While geographic isolation can start the speciation process, evolutionary biologists believe that other forces -- including various forms of natural selection -- can help to complete it.

The new U-M study provides rare empirical evidence that multiple forms of natural selection, including a contentious one called reinforcement, are helping to complete the speciation process in a natural howler monkey "hybrid zone," a place where the two species coexist and occasionally interbreed in a process called hybridization.

The study is scheduled for online publication Dec. 22 in the journal Molecular Ecology. In the paper, the researchers use the primate hybrid zone to identify parts of the genome that are likely to contain genes underlying speciation and to test for signals of the selection forces that shaped them.

"We observed patterns in the genetic data suggesting that hybridization is playing a direct role in completing the speciation process by enhancing genetic differences between species," said U-M doctoral candidate Marcella Baiz, the study's first author. The other authors are Liliana Cortés-Ortiz and Priscilla Tucker of the U-M Department of Ecology and Evolutionary Biology.

"We found a signal for multiple forms of natural selection driving species differences, including reinforcement, a process that has been highly debated," Baiz said. "This result is particularly notable because empirical evidence for reinforcement is extremely rare, especially genetic evidence."

The two species at the center of the study, mantled howler monkeys and black howler monkeys, diverged about 3 million years ago and lived apart until relatively recently when they came into contact again -- perhaps within the last 10,000 years -- in a roughly 12-mile-wide hybrid zone in the southeastern Mexican state of Tabasco.

A species was once defined as a group of actually or potentially interbreeding individuals that are reproductively isolated from other such groups. The concept of reproductive isolation is key to that definition and means that despite any hybridization, true species maintain their uniqueness.

However, the modern view of what a species is does not require full reproductive isolation, and hybridization has been discovered to be quite common in nature.

At the howler monkey hybrid zone in Mexico where U-M's Cortés-Ortiz and her colleagues have worked for about two decades, analysis of DNA samples has confirmed that black and mantled howler monkeys interbreed and produce hybrid offspring. The fact that hybridization is occurring between the two groups means that reproductive isolation is incomplete.

Evolutionary biologists believe that various natural selection pressures can help complete the process by strengthening barriers to gene flow between two groups, pushing them toward full reproductive isolation.

And because natural selection favors organisms that successfully reproduce over those that don't, it is biased against hybrids, which sometimes die before reproducing or are simply incapable of reproducing.

Natural selection tries to block the formation of these "unfit" hybrids. One way to do that is to gradually increase the genetic differences between two groups of organisms -- in this case black and mantled howler monkeys -- so that it's more difficult for them to mate and to produce hybrid offspring.

While working to thwart the formation of hybrids in this way, natural selection strengthens reproductive isolation by increasing genetic differences. This process is called reinforcement; while the idea has been around for more than a century, empirical evidence to support it is scarce.

To test for the presence of reinforcement, Baiz and her colleagues compared the DNA of black and mantled howler monkeys living the Tabasco hybrid zone to the DNA of black and mantled howler monkeys living far from the hybrid zone.

If reinforcement is working to thwart hybridization and to strengthen reproductive isolation, then the genetic differences between the two species in the hybrid zone should be greater than the genetic differences between monkeys of these two species living on either side of the hybrid zone.

And that's exactly what Baiz and her colleagues found when they compared genetic markers that are at or near genes likely associated with reproductive isolation.

"Speciation is a complex process that can be driven by direct and indirect mechanisms that interact to maintain and strengthen the process, and this study is one of the few natural examples that documents this," Baiz said.