What national COVID-19 modelling can tell us — and what it can't

Forecasting the future is never easy. When the forecast involves a pandemic's effects on the world's population and its economy, the stakes couldn't be higher.

That's why the COVID-19 modelling the federal government is presenting this morning needs to be read with an understanding of what these models can — and can't — tell us.

The projections that governments across the country are relying on now are imperfect but still important, because they allow those governments to assess their capacity to handle the spread of the virus, explain the reasons behind restrictive preventative measures and prepare for the future.

The number of COVID-19 cases and related deaths around the world continues to grow. In Canada, we're still talking about relatively low numbers — about 20,000 confirmed cases and 500 deaths — but the experience of other countries shows us that where we are today is not where we will be tomorrow.

The short-term projections recently published by Ontario, Quebec and Alberta offered stark descriptions of what that could mean for Canada. Ontario estimates the number of deaths in the province by the end of this month could be 1,600. Quebec puts their lowball estimate of deaths by that point at just under 1,300 while Alberta estimates between 400 and 3,100 deaths by the end of the summer.

Extrapolating the Ontario, Alberta and Saskatchewan long-term models to the entire country — the three had similar numbers on a per capita basis — suggests that without any restrictive measures in place, the total toll from the coronavirus pandemic in Canada would be about 14 million infections and nearly 300,000 deaths.

That's about the same as the total number of deaths in Canada every year from all causes.

These are the kinds of numbers that can concentrate the mind. Premiers Jason Kenney of Alberta and Doug Ford of Ontario have used them to justify the measures their governments have put in place — and to alarm people into taking those measures seriously.

But they also come with a lot of important caveats.

There are significant challenges involved in modelling the future that apply to all situations, from weather patterns to elections to pandemics. Models use existing data to make reasonable estimates of what could happen going forward, taking into account the relationship between different factors and how one thing affects another.

In the case of the COVID-19 pandemic, that means tracking the mortality rate of the disease, how quickly it can spread and what impact physical distancing and other measures have had on that spread — among many other factors.

Inconsistent data, uncertain forecasts

There is a limited amount of data with which to work. Models rely on how the virus has spread in this country, how it has spread in other countries and what has happened in past outbreaks of similar diseases.

But the numbers that do exist are inconsistent. The federal government needs to aggregate information from provincial health systems that compile it in different ways. For example: both Ontario and Saskatchewan have projected a best-case scenario death toll of about 3,000 people in their provinces — even though Ontario has 12.5 times the population of Saskatchewan.

This inconsistency extends to the data coming in from other countries. Those numbers are further complicated by the many differences between societies and the measures each country has taken to slow the spread within their borders (though Canada does benefit from being earlier in the cycle of the outbreak, giving it an opportunity to learn from other countries' experiences).

Each of these complications adds another level of uncertainty to the projections and increases the risk that the assumptions made by any model turn out to be wrong.

That potential grows greater the further into the future a model extends. Speaking in French on Monday, Horacio Arruda, Quebec's director of public health, said any forecasts extending beyond April 30 amount to "astrology."

Projections of what will happen in the next week or two should be fairly accurate. It's harder to predict what the situation will look like in May. It's even harder to know what to expect over the summer. It's like a game of "telephone" — errors pile up the longer the game goes on.

The first case of COVID-19 anywhere can be traced back to late last fall. That means no country has grappled with this virus for more than six months. So telling people where we'll be this winter, or next year, can amount to little more than an educated guess; the patterns of past viral outbreaks might not apply to this one.

Uncertainty not a reason for doubt

Every model, no matter what it's trying to forecast, comes with a great deal of uncertainty. It can't be avoided. Indeed, there is every reason to doubt a model that has too much confidence in its forecasts.

So when you see wide ranges in forecasted outcomes, don't regard that as a flaw in the model. It simply shows that there's a lot we still have to learn and the choices we make will have an impact on those outcomes.

"Every prediction, every model shows us a great variability, depending on the actions that we take as individuals, as a society," Prime Minister Justin Trudeau said Monday in French.

It's very possible that the final death toll will be very different from the projections. If that happens, it won't mean that the models were badly designed. While much depends on the assumptions the models make about the lethality of the virus, much also will depend on how effective the physical distancing measures are, and how closely people follow them.

"These numbers are not a done deal," Kenney said in a broadcast to Albertans on Tuesday. "I want, instead, for Albertans to see them as a challenge, perhaps the greatest challenge of our generation. Those numbers are not inevitable. How this actually plays out … all of that depends on us and our choices."

'Better for me to have a forecast that is not perfect'

When the pandemic is behind us, there will be no awards given for the most accurate forecasts.

On Monday, Arruda said his experts were telling him "'don't announce anything, Horacio, they will cut your head off if you don't have the right numbers.'" The health officials who presented Quebec's forecasts on Tuesday were cautious and would make no projections beyond the end of April.

There is certainly a risk involved in making projections in the midst of a pandemic. But whether the projections turn out to be accurate later matters less than whether the models made reasonable assumptions with the best information available — because governments are using them to inform their decisions.

The modelling released by British Columbia provides a practical example of this. It was focused not on future cases or fatalities, but rather on how many beds in intensive care units would be needed under various scenarios. That allowed the government to determine the likelihood that its health care system would be overburdened and decide what needs to be done in advance to prevent that from happening.

This sort of modelling gives governments the information they need to assess their stock of medical equipment, such as ventilators and masks. It provides an estimate of how long the economy will need to be shut down and how much financial support people will need.

There's no way of coming to these kinds of conclusions without making some assumptions about what is likely to happen next.

"Better for me to have a forecast that is not perfect than not to have any forecast," Quebec Premier François Legault said on Monday.

It's not a prediction — it's a reference point

"[Models] always describe a range of possibilities and these possibilities are for planning purposes," Theresa Tam, Canada's chief public health officer, said on Saturday. "They're not actual crystal balls or real numbers."

What goes into these models is supremely important and they need to be constantly updated. As more information becomes available, the forecasts will become more accurate. The numbers will fluctuate both because of new insights into COVID-19 and the actions taken by individuals.

The results act as a reference point for future decision-making, helping policymakers determine the efficacy of the measures they've put in place and decide when they can be safely ended. Those determinations might differ from one region or province to the next — and how lifting pandemic measures in one jurisdiction will affect its neighbours is yet another consideration for the models to incorporate. This is complicated stuff.

The numbers can be scary — particularly the worst-case scenarios — but they explain why governments have been willing to ask for tremendous sacrifices from Canadians. They're still just estimates. The future can be forecast but it is not yet written.