Economist group argues for scientific experimentation in environmental policymaking

In a new paper in the  journal Science, the economists say more frequent use of up-front experiments would result in more effective environmental policymaking in areas ranging from pollution control to timber harvesting across the world.

"Although formal experimentation is a cornerstone of science and is increasingly embedded in nonenvironmental social programs, it is virtually absent in environmental programs," the researchers wrote. "Strengthening the culture of experimentation in the environmental community will require changes in norms and incentives."

The paper acknowledges that scientists and practitioners can legitimately argue about how much time and effort should be given to experiments in environmental policy, but it contends that the current allocation of roughly zero percent is suboptimal.

The paper was produced by The Teton Group, an initiative led by Professor Todd Cherry, the John S. Bugas Chair in UW's Department of Economics. The prominent group of economists meets every fall in Wyoming to discuss critical ideas that impact environmental policy and economic development. Members include colleagues from UW and scholars in behavioral environmental policy from Carnegie Mellon University, Johns Hopkins University, Purdue University, the University of Texas-Austin, the University of Wisconsin-Madison and several key European universities. The group of UW economists include Todd Cherry, Jacob Hochard, Stephen Newbold, Jason Shogren, Linda Thunström and Klaas van 't veld.

"Guesswork is expensive, so we need to apply tools that reduce uncertainty about what works and what doesn't," Cherry says. "Lessons learned can improve current and future policy."

According to the new paper, environmental scientists and practitioners typically rely on field experience, case studies and retrospective evaluations of programs that were not designed to generate evidence about cause and effect. The result can be ineffective or even counterproductive programs.

"To help strengthen inferences about cause and effect, environmental organizations could rely more on formal experimentation within their programs, which would leverage the power of science while maintaining a 'learning by doing' approach," the economists wrote.

For example, an environmental agency that wants to learn how best to encourage industry to comply with environmental regulations might -- instead of implementing a single change in auditing practices across all polluting facilities -- randomly vary implementation of two auditing practices and contrast how facilities respond.

"By creating deliberate variation in how programs are implemented, program administrators can more easily learn about the features that make programs effective," the researchers wrote.

The paper notes that two agencies that regulate environmental practices -- the U.S. Environmental Protection Agency and the U.S. Department of Agriculture -- have embedded formal experimentation in their environmental programs fewer than six times in the past 30 years. In Europe, the practice is even less frequent. The same goes for nongovernmental organizations.

"Although environmental actors engage in thousands of informal 'experiments' every year (such as pilot programs), these are not controlled or designed to test the implicit hypotheses that justify the implementation of current programs or understand how to make these programs more effective," the economists wrote. "Formal experimentation in environmental programs is absent because science typically stops when implementation starts."

The researchers acknowledge ethical concerns about environmental experimentation -- which could expose people or other species to different, untested programs. But such concern "arises from a presumption that those exposed to a program, or a specific version of it, are sure to benefit from it," the economists wrote.

"That assumption, however, is not necessarily true. The effects of many environmental programs are uncertain," they wrote. "… Even programs that do not directly harm the environment or people may simply be ineffective."

Instead of implementing broad changes and then evaluating results later, agencies and organizations would better serve their constituents by being required to provide evidence before making changes, the researchers say. Such a requirement could be implemented for U.S. federal agencies through a new presidential executive order.

"Is there strong empirical evidence that the proposed action is the best option? If not, then the agency would be required to embed experimentation into the program with the intent of quantifying environmental and social impacts and understanding the mechanisms through which those impacts arise," the economists wrote.

The paper acknowledges that experimentation may not be justified or optimal in all environmental policymaking. But it should be used more than it is at present, the economists conclude.