Which COVID-19 models should we use to make policy decisions?

During a disease outbreak, many research groups independently generate models, for example projecting how the disease will spread, which groups will be impacted most severely, or how implementing a particular management action might affect these dynamics. These models help inform public health policy for managing the outbreak.

"While most models have strong scientific underpinnings, they often differ greatly in their projections and policy recommendation," said Katriona Shea, professor of biology and Alumni Professor in the Biological Sciences, Penn State. "This means that policymakers are forced to rely on consensus when it appears, or on a single trusted source of advice, without confidence that their decisions will be the best possible."

At the onset of an outbreak, particularly for a new disease, a large amount of information is often unavailable or unknown, and researchers must make decisions about how to incorporate this uncertainty into their models, leading to differing projections. For the COVID-19 outbreak, for example, uncertainty is present in a wide range of areas, from infection rate to details of transmission to the capacity of health care systems. The designers of each model bring their own perspective and approach to address these uncertainties.

"In order to improve modeling and analysis of epidemic disease, it is essential to develop protocols that deliberately generate and evaluate valuable individual ideas from across the modeling community," said Michael Runge, a research ecologist at the U.S. Geological Survey's Patuxent Wildlife Research Center who specializes in decision analysis for wildlife management. "We have identified best practices that allow the synthesis and evaluation of input from multiple modeling groups in an efficient and timely manner."

In the three-part process, multiple research groups first create models for specified management scenarios, for example, addressing how caseload would be affected if social isolation measures were lifted this summer, or how the duration of the outbreak would change if students return to school in the fall. The research groups work independently during this step to encourage a wide range of ideas without prematurely conforming to a certain way of thinking. Then, the modeling groups formally discuss their models with each other -- an important addition to previous multiple model methods -- which allows them to examine why their models might disagree. Finally, the groups work independently again to refine their models, based on the insights from the discussion and comparison stage.

After group discussion and individual model refinement, the models are combined into an overall projection for each management strategy, which can be used to help guide risk analysis and policy deliberation. At this stage, methods from the field of decision analysis can allow the decision maker, for example a public health agency, to understand the merits of different management options in the face of the existing uncertainty.

Additionally, the combined results can help identify which uncertainty -- what pieces of missing information -- are most critical to learn about in order to improve models and thus improve decision making, providing a way to prioritize research directions.

"This process allows us to embrace uncertainty, rather than hastening to a premature consensus that could derail or deflect management efforts," said Shea. "The process encourages a healthy conversation between scientists and decision makers, enabling policy agencies to more effectively achieve their management goals."

Even after initial decisions are made, the process can continue as new information about the outbreak and management becomes available. This "adaptive management" strategy can allow researchers to refine their models and make new predictions as the outbreak progresses. For COVID-19, this process might inform how and when isolation and travel bans are lifted, and if these or other measures might be necessary again in the future.

The research team plans to implement this process immediately for COVID-19. By taking advantage of the many research groups already producing models for the current outbreak, the strategy should be easy to implement while producing more robust results from the existing process. The team will share results with the U.S. Centers for Disease Control and Prevention as they are generated.

"We hope this process actively feeds into policy for the COVID-19 response in the United States," said Shea. "It also provides a framework for future outbreak settings, including emerging diseases and agricultural pest species, and management of endemic infectious diseases, including vaccination strategies and disease surveillance."

In addition to Shea and Runge, the research team includes David Pannell at the University of Western Australia, William Probert at the University of Oxford in the United Kingdom, Shou-Li Li at Lanzhou University in China, Michael Tildesley at the University of Warwick in the United Kingdom, and Matthew Ferrari at Penn State.

This work was supported by the National Science Foundation and the Penn State Huck Institutes of the Life Sciences though the Coronavirus Research Seed Fund.