The case for economics -- by the numbers

The study examines 140,000 economics papers published over a 45-year span, from 1970 to 2015, tallying the "extramural" citations that economics papers received in 16 other academic fields -- ranging from other social sciences such as sociology to medicine and public health. In seven of those fields, economics is the social science most likely to be cited, and it is virtually tied for first in citations in another two disciplines.

In psychology journals, for instance, citations of economics papers have more than doubled since 2000. Public health papers now cite economics work twice as often as they did 10 years ago, and citations of economics research in fields from operations research to computer science have risen sharply as well.

While citations of economics papers in the field of finance have risen slightly in the last two decades, that rate of growth is no higher than it is in many other fields, and the overall interaction between economics and finance has not changed much. That suggests economics has not been unusually oriented toward finance issues -- as some critics have claimed since the banking-sector crash of 2007-2008. And the study's authors contend that as economics becomes more empirical, it is less dogmatic.

"If you ask me, economics has never been better," says Josh Angrist, an MIT economist who led the study. "It's never been more useful. It's never been more scientific and more evidence-based."

Indeed, the proportion of economics papers based on empirical work -- as opposed to theory or methodology -- cited in top journals within the field has risen by roughly 20 percentage points since 1990.

The paper, "Inside Job or Deep Impact? Extramural Citations and the Influence of Economic Scholarship," appears in this month's issue of the Journal of Economic Literature.

The co-authors are Angrist, who is the Ford Professor of Economics in MIT Department of Economics; Pierre Azoulay, the International Programs Professor of Management at the MIT Sloan School of Management; Glenn Ellison, the Gregory K. Palm Professor Economics and associate head of the Department of Economics; Ryan Hill, a doctoral candidate in MIT's Department of Economics; and Susan Feng Lu, an associate professor of management in Purdue University's Krannert School of Management.

Taking critics seriously

As Angrist acknowledges, one impetus for the study was the wave of criticism the economics profession has faced over the last decade, after the banking crisis and the "Great Recession" of 2008-2009, which included the finance-sector crash of 2008. The paper's title alludes to the film "Inside Job" -- whose thesis holds that, as Angrist puts it, "economics scholarship as an academic enterprise was captured somehow by finance, and that academic economists should therefore be blamed for the Great Recession."

To conduct the study, the researchers used the Web of Science, a comprehensive bibliographic database, to examine citations between 1970 and 2015. The scholars developed machine-learning techniques to classify economics papers into subfields (such as macroeconomics or industrial organization) and by research "style" -- meaning whether papers are primarily concerned with economic theory, empirical analysis, or econometric methods.

"We did a lot of fine-tuning of that," says Hill, noting that for a study of this size, a machine-learning approach is a necessity.

The study also details the relationship between economics and four additional social science disciplines: anthropology, political science, psychology, and sociology. Among these, political science has overtaken sociology as the discipline most engaged with economics. Psychology papers now cite economics research about as often as they cite works of sociology.

The new intellectual connectivity between economics and psychology appears to be a product of the growth of behavioral economics, which examines the irrational, short-sighted financial decision-making of individuals -- a different paradigm than the assumptions about rational decision-making found in neoclassical economics. During the study's entire time period, one of the economics papers cited most often by other disciplines is the classic article "Prospect Theory: An Analysis of Decision under Risk," by behavioral economists Daniel Kahneman and Amos Tversky.

Beyond the social sciences, other academic disciplines for which the researchers studied the influence of economics include four classic business fields -- accounting, finance, management, and marketing -- as well as computer science, mathematics, medicine, operations research, physics, public health, and statistics.

The researchers believe these "extramural" citations of economics are a good indicator of economics' scientific value and relevance.

"Economics is getting more citations from computer science and sociology, political science, and psychology, but we also see fields like public health and medicine starting to cite economics papers," Angrist says. "The empirical share of the economics publication output is growing. That's a fairly marked change. But even more dramatic is the proportion of citations that flow to empirical work."

Ellison emphasizes that because other disciplines are citing empirical economics more often, it shows that the growth of empirical research in economics is not just a self-reinforcing change, in which scholars chase trendy ideas. Instead, he notes, economists are producing broadly useful empirical research.

"Political scientists would feel totally free to ignore what economists were writing if what economists were writing today wasn't of interest to them," Ellison says. "But we've had this big shift in what we do, and other disciplines are showing their interest."

It may also be that the empirical methods used in economics now more closely match those in other disciplines as well.

"What's new is that economics is producing more accessible empirical work," Hill says. "Our methods are becoming more similar ... through randomized controlled trials, lab experiments, and other experimental approaches."

But as the scholars note, there are exceptions to the general pattern in which greater empiricism in economics corresponds to greater interest from other fields. Computer science and operations research papers, which increasingly cite economists' research, are mostly interested in the theory side of economics. And the growing overlap between psychology and economics involves a mix of theory and data-driven work.

In a big country

Angrist says he hopes the paper will help journalists and the general public appreciate how varied economics research is.

"To talk about economics is sort of like talking about [the United States of] America," Angrist says. "America is a big, diverse country, and economics scholarship is a big, diverse enterprise, with many fields."

He adds: "I think economics is incredibly eclectic."

Ellison emphasizes this point as well, observing that the sheer breadth of the discipline gives economics the ability to have an impact in so many other fields.

"It really seems to be the diversity of economics that makes it do well in influencing other fields," Ellison says. "Operations research, computer science, and psychology are paying a lot of attention to economic theory. Sociologists are paying a lot of attention to labor economics, marketing and management are paying attention to industrial organization, statisticians are paying attention to econometrics, and the public health people are paying attention to health economics. Just about everything in economics is influential somewhere."

For his part, Angrist notes that he is a biased observer: He is a dedicated empiricist and a leading practitioner of research that uses quasiexperimental methods. His studies leverage circumstances in which, say, policy changes random assignments in civic life allow researchers to study two otherwise similar groups of people separated by one thing, such as access to health care.

Angrist was also a graduate-school advisor of Esther Duflo PhD '99, who won the Nobel Prize in Economics last fall, along with MIT's Abhijit Banerjee -- and Duflo thanked Angrist at their Nobel press conference, citing his methodological influence on her work. Duflo and Banerjee, as co-founders of MIT's Abdul Latif Jameel Poverty Action Lab (J-PAL), are advocates of using field experiments in economics, which is still another way of producing empirical results with policy implications.

"More and more of our empirical work is worth paying attention to, and people do increasingly pay attention to it," Angrist says. "At the same time, economists are much less inward-looking than they used to be."