Why the polls called it, but seat projections missed

Most of the public opinion polls conducted at the end of the federal election campaign came very close to predicting the result. The election was a success story for pollsters. Yet the magnitude of the Liberals' majority victory came as a surprise to many. Why?

Going into election night, the expectation was that the Liberals would win enough seats to form a minority government. A majority government was a slim possibility, perhaps just as likely as a Conservative plurality. Instead, the Liberals won 14 more seats than required for a majority government.

• How new voters tipped the scales for Liberals
• Election Pollcast: The view from the campaigns' final days
• Analysis: Momentum carries Liberals past majority threshold

That this came as a surprise when the polls were on the mark was largely because of the estimations of seat projections like that of the CBC's Poll Tracker. The Liberals were projected to win between 124 and 161 seats, the Conservatives between 100 and 139 seats, the New Democrats between 51 and 90 seats, and the Bloc Québécois between one and 12 seats.

Instead, the Liberals won 184 seats, the Conservatives 99, the NDP 44, and the Bloc 10.

The results for the three main parties fell within the maximum and minimum ranges (topping out at 185 for the Liberals and bottoming out at 77 for the Conservatives and 27 for the NDP), but those are designed to take into account major errors in the polls. The polls had no such errors.

The Poll Tracker was not the only seat projection to miss the mark. Seat projections from various sources put the Liberals in minority territory, most suggesting the Liberals would come up 30 seats or more short of the mark.

There are good reasons for this. Despite reaching the traditional threshold for a majority government in the popular vote, the Liberals should not have been able to win a majority government based on the 2011 distribution of their support. Instead of growing proportionately or predictably, the Liberals won new votes in unexpected (and largely unpredictable) places. And that tells us quite a bit about why the Liberals managed to win the election so handily.

Winning in all the right places

(On mobile? View the full map of results here.)

Seat projection models are designed to translate national and regional support numbers into seats. If they work properly, then plugging the actual results of an election into them should produce seat totals very close to the actual seat counts.

• Final tally: How the parties did on election night

But when the actual numbers are plugged into the ThreeHundredEight.com seat projection model behind the CBC Poll Tracker, the results do not end up so close to the mark: 154 seats for the Liberals (or a range of between 127 to 170), 120 for the Conservatives (105 to 147), 56 for the New Democrats (37 to 75), and seven for the Bloc (one to 14). Had those numbers been the final projection of the campaign, only a little more consideration would have been given to the potential for a Liberal majority government.

This projection would have under-estimated the Liberals by three seats in British Columbia, one in the Prairies, 14 in Ontario, seven in Quebec and five in Atlantic Canada.

Notably, it would have over-estimated the Conservatives by five in British Columbia and 10 in Ontario, while giving the New Democrats too few seats in British Columbia and the Prairies but too many in Ontario and, in particular, Quebec. 

The results in Ontario, British Columbia, and Quebec are illustrative. The gap of just under 10 points between the Liberals and Conservatives in Ontario should not have been wide enough to deliver 80 seats to the Liberals, 47 more than the Conservatives won. Based on the party's vote inefficiency in recent elections, the gap would have been expected to be between 30 to 40 seats.

(On mobile? View the full map of B.C. results here.)

Despite finishing second in the vote count in British Columbia, the Conservatives would not have been expected to finish third in the seat count in the province, outpaced by both the Liberals (who traditionally do not have geographically widespread support in the province) and the NDP. 

The New Democrats seriously under-performed in Quebec, winning just 16 seats when 19 to 40 would have been expected at the levels of support they had. The Liberals would have been projected to win between 27 and 37 seats, instead of 40.

New voters in new places

This is not to simply point out the errors in the model. Seat projections can act as a sort of baseline, against which real results can be compared. As the model (like others) assumes a relatively uniform increase of support for the party in each region of the country, the actual result provides hints at the kinds of gains the Liberals made in this election campaign.

In Ontario, the Liberals were particularly effective in attracting voters who did not cast a ballot in 2011. Turnout overall was significantly higher in this election than in 2011, and in about two-thirds of Liberal gains in Ontario the party owed its victory to these new (or returning) voters. In other words, their vote efficiency in the province changed because their voting base changed.

This phenomenon was even stronger in British Columbia: four-fifths of Liberal gains can be attributed to new voters. On average, the Liberals earned just more than 16,000 new votes in their B.C. gains over their 2011 performance in the same ridings. The Conservatives lost an average of 3,300 votes in these ridings, and the New Democrats just over 800 votes.

Liberal gains in British Columbia thus came at the expense of the Conservatives rather than the NDP, which, despite dropping in the popular vote, gained seats. This is the kind of very localized shift that a seat projection model based on province wide trends will always struggle with.

(On mobile? View the map of Quebec results here.)

It was a different story in Quebec, where the Liberals won virtually all of their new seats at the expense of the NDP. Roughly three-fifths of Liberal gains in the province can be attributed to the votes they took away from other parties, rather than new voters casting a ballot.

While on average the Liberals won about 15,000 new votes in each of the seats they gained in the province, the NDP lost almost 9,000 votes in each of these seats — in many cases far more than was needed to tip the balance in favour of the Liberals.

Nevertheless, the ability of the Liberals to win votes off the island of Montreal among francophones was something that could not be tracked by province wide polling alone. That is what made the difference.

Next time will not be the same

What can be done to avoid these kinds of errors in the future?

Perhaps looking further back at past election results to find bases of support that exist but may not have voted in the previous election is one solution. 

But with the Liberals pledging that the 2015 election will have been the last to be decided by the first-past-the-post system, any attempts to solve the problems of projecting federal elections may prove irrelevant. If Canada moves towards a version of proportional representation, depending on the specifics, the onus may only be on the polls to get the call right. If Canada instead moves towards a version of ranked ballots, then entirely new means of projecting the outcome will need to be devised.

Thankfully (and unexpectedly), there is plenty of time to figure that out before the next election is held in 2019.