Question: I’ve been hearing a lot about a couple of models decision-makers are using to forecast hospital utilization needs. I saw that UPenn’s CHIME tool shows that the apex for DC’s COVID cases would come around mid-June/July, but University of Washington’s IHME tool projects DC’s peak to be April 15th or so. I also saw that the DC government is favoring CHIME because it accounts for the fact that not everybody is going to practice social distancing. Would you discuss the pros/cons of each model or talk more broadly about what we should consider before taking these projections as gospel?
Answer: Dang, good question. First off, models are only as good as the assumptions and data that go into them. And in the context of COVID-19, we don’t have much data and our assumptions are still changing as we’re learning more about the new virus all the time. Threading this needle of highlighting health system gaps for program and policy decision-makers — as all of these models do — with reminding folks that the estimates are *not* to be taken as gospel — which none of these models should — is challenging to say the least. On this point, I recommend a couple of articles from The Atlantic and Fivethirtyeight.
All that said, I was curious about these models too, so I read the methodology of each (see here and here). I find the IHME model far more difficult to understand than the CHIME tool (note: I generally find IHME models difficult to understand…. and I was heartened to read that it’s not just me!) Enclosed is a short table I made describing the two models and highlighting some of their differences as I see them.
I’m not in a position to say which is better. But there are a few things that stand out. First, IHME’s model assumes that our social distancing measures will have the same impact on the spread of the virus (R0) as Wuhan’s drastic measures did. This is a lofty assumption and will likely downplay the risk of ongoing transmission, especially in states that have not implemented social distancing measures (I just read today that Georgia reopened its beaches!). By making this assumption, the model estimates that COVID-19 hospitalizations across the US will peak at some point in April. Given what we know of the slow roll-out of social distancing measures in large swaths of the country, especially the south, this estimate seems optimistic. Meanwhile, Susceptible, Infected, Recovered (SIR) models like the one used by CHIME are often limited by assuming homogeneous risk across a population, which can inflate the number of expected cases. The CHIME model partially overcomes this limitation by allowing users to modify the inputs based on local expectations/data. That said, the population susceptible assumption is still a challenge for this model, and may result in overestimates of health system needs.
Personally, I prefer the ability CHIME gives its users to modify the inputs/expectations, especially with regard to social distancing effectiveness. If you’re looking for estimates of deaths, however, of these two models, you can only get estimated deaths from IHME. And if you don’t have a good idea of the inputs for your region, you might prefer the IHME dashboard, which doesn’t require any user input. Finally, if you were wondering how CHIME compares to the oft-cited Imperial College model, check out this post CHIME modelers wrote on this very question (upshot== models preform similarly). Regardless of the model we choose to use, I think the main message is the same: COVID-19 cases will outstrip our hospital capacity, so we need to take measures now to prepare as best as possible for the worst and hope for the best.
Table 1. IHME and CHIME Comparison
