Finance Canada hiring pollsters to gauge Canadians' gut reactions to budget

Dozens of Canadians will be providing minute-by-minute feedback for each statement in Finance Minister Bill Morneau's upcoming budget speech, in a relatively novel effort by the Liberal government designed to help craft its budget sales job.

In cities outside Ottawa, carefully selected Canadians will each be given an electronic dial to register in real time their approval or disapproval of each measure that Morneau announces, on a scale of zero to 100. 

The so-called dial-testing technology, quietly rolled out for a trial run in the 2016 budget, is being expanded for the next budget, expected next month.

Feelings are just as important as facts- David Valentin, executive vice-president, Mainstreet Research

"The research was a successful tool to help develop communications products following the budget day announcement," says a Finance Canada document authorizing this year's plan, obtained by CBC News under the Access to Information Act.

"The research will help to assess and confirm the development of communications following budget announcement in 2017."

Versions of the technology, which registers observers' visceral, instant reactions to debates and other live events, date back to the 1930s.

Sometimes called "continual response," "real-time response," "moment-to-moment research," or just "dial-testing," the technology first gained widespread attention in 1987 when CNN began using it for political debates, televising the changing results in an often mesmerizing graphic.

Used in mock trials

Dial testing has since remained a standard feature of political debates, including Canada's Conservative leadership forums where a non-televised, non-public version was rolled out to monitor debates in Moncton, N.B., Quebec City and Saskatoon.

The computerized technology has been commonly used to fine-tune consumer-product commercials. Law firms have also used dial-testing for mock trials, to help prepare legal strategies for real arguments before an actual judge or jury.

Last year, Finance Canada paid the AC Nielsen Co. of Canada almost $61,000 to do dial-testing of the March 22 budget speech, with French-speaking focus groups in Montreal and English-speaking in Mississauga, Ont.

"To measure reactions during the speech, participants were given a 'dial' that ranges from 0 to 100 and were encouraged to turn it 'up' when [they] heard something that resonated with them, and 'down' when they heard something that did not resonate or was not seen as credible," says the Nielsen report, which used a proprietary technology called "Perception Analyzers."

Participants were paid $100 each for three hours.

Among Morneau's instant winning statements was a promise to end boil-water advisories on First Nation reserves, Nielsen reported. The dial readings dipped, on the other hand, at the minister's mention of carbon-pricing and expanded intake of Syrian refugees.

Dial-testing has critics

A spokesman for Finance Canada said to date, no contract has been signed with a pollster. "We are currently working with the contracting authority at Public Services and Procurement Canada to identify a contractor and no final decisions have been made," Paul Duchesne said in an email.

No date has been announced yet for the 2017 budget. Last year, the contract with Nielsen was signed just one week before the March 22 budget speech. This year's plan is to expand the testing to three cities.

Dial-testing has its critics, who raise questions about whether the pressure for instant results produces reactions, opinions and thoughts that may not endure, among other objections.

"The instrument is very poor for capturing opinions or judgments that form over longer time slices," says Gordon Mitchell, a communications professor at the University of Pittsburgh who has studied dial-testing.

"Something a viewer is listening to might be appealing in one particular aspect but their net opinion ... might take a longer time to coalesce. And there is no real way to capture that on the dial meter." 

The executive vice-president of Mainstreet Research, the Ottawa-based firm that did dial-testing of the Conservative leadership debates, says focus groups, facilitators and goals must be carefully chosen, and there must be proper follow-up with participants afterward.

David Valentin adds that immediate reactions from the gut can be revealing.

"Feelings are just as important as facts," he said from Montreal. "And I think the last U.S. presidential elections are really a great example of that."

"The idea that you believe something is much more important, despite the fact of whether it might be true or not.

"There is no such thing … as a wrong feeling. All feelings, whether we like them or not, are valid."

Finance Canada is not the first federal department to use dial-testing. Health Canada commissioned the technology in 2008-2009 to test the reaction of young Canadians to anti-drug television advertisements.

Follow @DeanBeeby on Twitter