What are the models telling us and how have they changed over time?

Question: I’m reading so much these days about projected COVID-19 deaths, especially with this weekend’s widely reported statements made by Dr. Birx on “Fox News Sunday” that “our projections have always been between 100,000 and 240,000 American lives lost…” What are the models telling us and how have they changed over time?

Answer: We’ve talked a bit about modeling in our Q&As of 4/5 and 4/28. The CDC now has a helpful resource page describing the various models, which was just updated on 1 May. I’ve taken a couple of figures from CDC’s resource pages to help answer your question. [Related aside: The IHME model is not listed on the page, but it has been listed in previous estimates. I’m not sure why it’s missing now, but it could be that the conditions that IHME were assuming — stringent national social distancing measures maintain — are no longer in effect nationally.]

As you can see in Figure 1, the many models do vary to some degree, with several estimating that deaths will exceed 100,000 by late-May and others estimating that the rate of increase will slow with deaths in the range of 70,000. The ensemble estimate, which summarizes estimates from the individual models, suggests about 90,000 deaths by late-May (right side of Figure 1). Much of this variation across models is due to: a) the assumptions that the modelers use to make their estimates (ex: CU-20 estimates only a 20% reduction in social contacts while CU-40 estimates a 40% reduction); and b) the various scientific methodologies the models use to make the predictions. If you want to further explore national and state-level estimates, this tool allows you to do so (it’s pretty fascinating!).

When it comes to how these estimates have changed over time, we can look at previous forecasts and see. For example, Figure 2 shows the cumulative death estimates as of April 13th across multiple models. Eyeballing it, the CU-30 estimate for May 1 seems to track the best with the actual number of deaths as of May 1 (60K-65K deaths — exact number varies depending on which source you use). Furthermore, if you wanted to see how IHME estimates have changed over time, this tool makes it very easy to visualize. Check it out!

It’s to be expected that models will change over time as scientists learn more and as modelers adapt their assumptions based on new data and learning. As Ed Yong so eloquently wrote last week in The Atlantic, “If measuring the present is hard, predicting the future is even harder. The mathematical models that have guided the world’s pandemic responses have been often portrayed as crystal balls. That is not their purpose. They instead describe a range of possibilities, and help scientists and policy makers to simulate what might happen pending different courses of action. Models reveal many possible fates, and allow us to choose one. And while distant projections are necessarily blurry, the path ahead is not unknowable.”

Figure 1. April 27th Forecast of Cumulative COVID-19 Deaths Nationally over the Next Month (from CDC)

Image for post

Figure 2. April 13th Forecast of Cumulative COVID-19 Deaths Nationally over the Next Month (from CDC)

Image for post