Canada's pollsters nailed the outcome of the federal election - mostly

Politicians and pundits love nothing more than to say the polls can't be trusted. After all, the numbers frequently get in the way of talking points or convenient narratives.

But in the 2019 federal election, the polls did pretty well — and where they didn't, they offer clues about what happened on Monday.

Roughly 120 national surveys and dozens of regional and riding polls were published during the 40-day campaign. Twelve different pollsters released national survey results in the last week of the election. All of them were relatively close to the final outcome.

The polls captured the close race between the Liberals and Conservatives from start to finish, the steep rise of the Bloc Québécois, the growth of the New Democrats and slumping support for the Greens.

How those numbers broke down at the regional level suggested how difficult it would be for any party to secure a majority of seats, and pointed to the Liberals as the party in the best position to emerge at the head of a minority government.

The Conservatives finished the night with 34.4 per cent of ballots cast, edging out the Liberals' 33.1 per cent. The New Democrats took 15.9 per cent of the vote, followed by the Bloc at 7.7 per cent, the Greens at 6.5 per cent and the People's Party at 1.6 per cent.

Those 12 pollsters pegged the Conservatives at between 30 and 33 per cent, the Liberals at between 29 and 34 per cent, the NDP at between 14 and 21 per cent, the Greens at between six and nine per cent and the People's Party at between one and four per cent. In Quebec, most pollsters were close in predicting the 32.5 per cent the Bloc captured.

Considering the margins of error for all of these polls, that's almost as good as it gets.

In fact, every pollster but one had an average error of less than three percentage points for the Conservatives, Liberals and NDP. Four pollsters (Léger, Abacus Data, Mainstreet Research and Ipsos), along with the CBC's Poll Tracker, predicted all three parties' results within three points.

The final polls by Léger and Abacus had the lowest total error, at just under six percentage points. Mainstreet and Ipsos each had a total error of eight points. These were the four pollsters whose total error was lower than the Poll Tracker's, which had a total error of nine points. The other eight pollsters had total error ranging from 10 to 14 points.

Both the Conservatives and Liberals were largely under-estimated in the polls — every pollster undershot the Conservatives by between one and four points, while 10 of 12 did the same for the Liberals. The New Democrats and (to a lesser extent) the Greens were over-estimated across the board.

There are some reasonable explanations for those small errors. Strategic voting might have played into the hands of the Liberals, peeling off a few NDP and Green voters at the ballot box. Polls suggested supporters of these parties were the least committed and the most likely to flip to the Liberals.

The Conservatives might have seen a bump at the ballot box due to turnout; their supporters are disproportionately older, and older people are more likely to vote. Their large and unexpected numbers in Western Canada might also have played a significant role.

Conservatives beat their polls in Western Canada

At the regional level, the aggregate performance of the polls — as measured by the Poll Tracker — was largely on the mark. But there were some big errors in Alberta and the Prairies.

The Conservatives beat their aggregate polls in Saskatchewan and Manitoba by 9.5 percentage points and in Alberta by 8.5 points — a significant miss that might not have delivered many more seats to the party but would have been worth nearly two percentage points at the national level. That alone explains much of the low-balling of Conservative support in the polls.

The party was only under-estimated by two points in Atlantic Canada and Quebec, and in Ontario by only one. The Liberals were over-estimated in Western Canada but slightly under-estimated in Ontario, Quebec and Atlantic Canada — which explains in part why the Liberals did better in the seat count than was widely expected.

It's possible that polls were not measuring all of the dissatisfaction with the Liberals in Western Canada. It's also possible that many progressive voters in this region did not consider it worthwhile to cast a ballot. In Alberta and Saskatchewan, for instance, the Conservatives captured a majority of the vote in 43 of 48 ridings.

How the seat projection did

There was certainly a big audience for all of the polls published over the course of the campaign — the Poll Tracker received millions of visits over that time. The Poll Tracker's average projection was close to the mark but, like the polls that made it up, it under-estimated both the Conservatives and the Liberals to the benefit of the NDP, Greens and People's Party.

The seat results all fell well within each party's estimated seat ranges of the Poll Tracker. The Liberals ended the night with 157 seats, against 121 for the Conservatives, 32 for the Bloc, 24 for the NDP, three for the Greens and one Independent.

The precise projection, at 137 Liberals, 124 Conservatives, 39 Bloc, 35 NDP and one seat apiece for the Greens, People's Party and Independents was not as close as it could have been — the Bloc and NDP were too high and the Liberals too low. Other projections, like those put out by EKOS Research and Mainstreet Research in particular, were closer.

In all, 87 per cent of ridings were correctly called and 98 per cent of the winners were identified within the 95 per cent confidence interval (94 per cent within the 68 per cent interval).

The ghosts of MPs past

The biggest source of error was the seats that were called for the NDP but actually ended up going to the Liberals — this alone was responsible for a net eight-seat miss in the national seat projection.

The list of those misses is instructive, as they include seats like Halifax and Sackville–Preston–Chezzetcook in Nova Scotia, Acadie–Bathurst in New Brunswick and Toronto–Danforth, Nickel Belt, Thunder Bay–Rainy River and Hamilton East–Stoney Creek in Ontario.

These are all seats where the NDP had a history of electoral success until 2015. Their under-performance in these ridings suggests that the loss of long-time MPs had a particularly significant impact on support for the NDP in these ridings; without these names on the ballot, the NDP brand was not strong enough to win the seat.

Liberal wins in ridings that were projected to go Conservative were largely around Kitchener-Waterloo and seats like Burlington, St. Catharines and Whitby in the Golden Horseshoe — suggesting that the Conservatives have lost some underlying strength here.

A less efficient Bloc vote than projected explains much of the rest — a few Liberal and Conservative incumbents proved more resilient than expected.

As always, there will be some calibrating of the seat projection model in the wake of the results — particularly when it comes to projecting ridings where a party's history of support might be tied more closely to departed MPs or candidates than to a party's brand.

But with the exception of a single day, the Poll Tracker always showed the Liberals in a better position to win more seats than the Conservatives — even when they were trailing in the popular vote — and short of a majority government. In this campaign, at least, you could trust the polls.