New method to address deep-seated biases in science

Awareness of female birdsong is growing worldwide, thanks in part to a breakthrough paper by Karan Odom, Ph.D. '16, biological sciences, but it's still understood as a trait found primarily in tropical birds. Evangeline Rose, a current Ph.D. student in the same lab and first author on a new paper in Animal Behavior, wanted to look at song in a temperate species.

During Rose's fieldwork, "I was finding that the females were singing, to me, what sounded just like male songs," she says. "So we started thinking about equality, and equivalence, and how to test for it." On the advice of her advisor, Kevin Omland, professor of biological sciences, she reached out to Thomas Mathew, professor of statistics, who has expertise in statistical equivalence.

Challenging a paradigm

Working together, the team modified a statistical method used in generic drug testing to meet their needs for ecology and animal behavior studies. The existing test helps determine whether generic and brand name drugs are "statistically equivalent," meaning they are similar enough to be prescribed safely for the same purpose. The new modification will allow scientists in other fields to test for equivalence. Before, researchers could only report they did not find a significant difference -- a very different statement than saying two things are conclusively equivalent.

"We're really hoping this new method is going to address some issues with what kinds of data get published," Rose says. "The most important thing about being a good scientist is to be unbiased. And the whole tradition of testing for difference really leads to incredible biases in scientists," Omland says. He adds, "There's a whole realm of things in nature that we find interesting and important because of their similarity."

For example, in addition to similarities in songs between the sexes in birds, researchers could use the new test to ask if two species use the same type of habitat, respond the same way to predators, or consume the same food sources. Answers to those questions could fill long-standing knowledge gaps, or even inform conservation efforts.

"This test is really broadly applicable," says Rose, "and we're hoping to introduce it more to the ecology and evolution field."

A new approach

One advantage of the new method is it accounts for unequal sample sizes. In a medical study, researchers can carefully control the size of treatment and control groups. In other fields, from ecology, to engineering, to agriculture, that's often not possible. The new test also allows researchers to determine the equivalence of several traits simultaneously, Mathew explains. For example, in this study, the authors found that the male and female birds' songs were statistically equivalent across five different characteristics, such as duration of each song and the range of pitches the birds produced.

Rather than testing whether two things are exactly equal, the team was looking for a way to determine if two things were "close enough," given a defined allowable margin of difference. Because of that added layer, "There are additional challenges here," Mathew says.

"Even though this methodology is out there, it hasn't been applied -- even in statistics -- with this kind of data. That's why I was very excited when they brought this project to me," Mathew says. Rose adds, "It ended up being a really great partnership to look at these questions that hadn't been asked before for female song, and we also ended up modifying this test in a really cool, new way."

Changing science

As research on similarities grows, there is also a growing drive to remove the bias against publishing studies that do not find a significant difference, often termed a "negative result." This paper "is part of an amazing drumbeat that's building up in the scientific community," Omland says. "There's a broader problem with the scientific method that's being increasingly acknowledged, and the test we've developed can at least play a small role, and I hope a big role, in addressing it."

Rose, who plans to next investigate the function of female bluebird songs, says she will carry these new techniques with her as she moves through her research career. "I think in the future, I'll be thinking about how equivalence can change the questions we're asking, and I'll always keep in mind that we have extra tools in the toolkit."