I continue to be surprised by the often-quoted Institute for Health Metrics and Evaluation, and yesterday I was triggered by a note in the news, specific to Washington State, about “Hopeful” and “Sharing Ventilators” with other states. Then, of course I saw a quote about IHME data showed that the peak was past.
The forecast I downloaded from them yesterday showed, for Washington State:
- Peak Date of the epidemic was in the past — 27 March
- Peak Fatalities for the epidemic were only 24
However, reality was already quite different:
- Peak Date so far was 6-April, 10 days later
- Peak Fatalities so far were 39
- 5 Days Wrong, and around 50% off?
Part of it is a problem with having small numbers: a 5 fatality-miss can look like more than 20% error, because Washington’s daily fatalities are very flat.
However?
Would you like your State to relinquish medical supplies or cancel orders based on optimistic forecasting? I would have grave concerns. California, for instance is sharing ventilators, and IHME projects the peak of fatalities for California around 15th of April.
Fortunately, California is apparently not using IHME, and is seeing, as have I, peaking later in May:
Given that coronavirus cases are not expected to peak until May, under current estimates, California could afford to lend the medical devices to parts of the country where they are in seriously short supply, Newsom said.
So, IHME isn’t asleep, and they shared another projection, from apparently 12:30am this morning, which I combined with current data (7 April) from the New York Times:
Today is the peak, and it’s all downhill for fatalities in the State. Hopeful, and I’d love that to be the case for Washington State, but I suspect in a day or two, it will revise up (as it already has), and then up again, and again.
Here’s the crux of the problem:
My perception is that IHME keeps re-forecasting using actual data (good), but is not revising their assumptions around maximum fatalities and inflections in their model. As flattening works, you would assume peak dates stretch out, and that the peak flattens, which is what happens in other models. Gavin Newsom calls it “Flatten and Stretch,” (sounds like something to do at the gym). Strangely, IHME shows peaking and compressing, at least for Washington State, sharper peaking and dates pulling forward instead of pushing out.
Something is quite wrong there.
I will continue to watch, but as I’ve said in multiple posts, what they publish at this point is easy to refute (compared to actual data), and can provide a dangerous sense of optimism. I’m saddened that it continues to be widely communicated.
Stay Calm. Critical decisions need a clear head.
Stay Sane. Incessant COVID noise from deranged people will derange you.
Stay Home. There’s no safety in numbers, for the time being.
Disclaimer: I’m not a statistician, nor am I an epidemiologist or physician. I do have substantial training in science and mathematics. My assumptions are based on publicly available data, and interpreted without domain-specific expertise to temper my judgements.