Will the real unemployment rate please stand up?

The research, published by the National Bureau of Economic Research, finds that the true unemployment rate may be higher or lower than recent reports indicate. In fact, the authors write that published unemployment rates have gradually become more difficult to interpret over time, especially in the last two decades. The researchers cite survey design changes as a likely culprit, in large part because the changes corresponded with an increase in nonresponse rates by Americans.

"It is potentially a huge issue," said Alan Krueger, the Bendheim Professor in Economics and Public Policy at the Wilson School and former chairman of President Barack Obama's Council of Economic Advisers. "But our results do suggest several important avenues for future research and improvements in the data."

To calculate the unemployment rate, the U.S. Census Bureau and U.S. Bureau of Labor Statistics administer the Current Population Survey (CPS), which collects extensive demographic data to better understand labor market fluctuations and economic conditions. The CPS consists of a sample of nearly 60,000 households in all U.S. states and the District of Columbia. Households are surveyed monthly for four consecutive months, left alone for eight months and then surveyed again monthly during the final four months. In any given month, there are eight "rotation groups," depending on how many months the households have been in the survey so far. Each rotation group is designed to be representative of the population. The official unemployment rate is a weighted average of the eight groups.

Krueger and his collaborators -- Alexandre Mas, professor of economics and public affairs at Princeton's Wilson School, and Xiaotong Niu, an analyst at the Congressional Budget Office -- found that in the first half of 2014, the unemployment rate among people in the first month of being interviewed was 7.5 percent. However, for those in the final month of being interviewed, it was only 6.1 percent. Because the Bureau of Labor Statistics weights the first interview more heavily, the official unemployment rate for this period was 6.5 percent.

This disparity between interview groups, known technically as "rotation group bias," isn't new. But Krueger's paper is the first since 1975 to explore the growing magnitude and evolution of such discrepancies. This new paper shows that rotation-group bias has doubled since 1994, when the CPS underwent a major redesign.

"It is unclear which rotation group provides the most accurate measure of the unemployment rate," said Krueger. "The unemployment rate for each rotation group tends to rise or fall together over time."

Two decades ago, the CPS moved from pen-and-paper questionnaires to computer-assisted phone interviews. They also changed the language and logic behind some of the questions. Shortly after the redesign, CPS began seeing a large rise in nonresponse rates.

Interestingly, the study shows that this same kind of bias isn't found when looking at data from the United Kingdom and Canada, which also use rotating groups to measure their unemployment rates. This finding suggests several avenues for future research including examining the details of the surveys themselves, like the nature of the questionnaire, the interviewing methods and the survey response rates. All, Krueger said, play a pivotal role in shaping the unemployment figure outcomes.