Man who filled out federal government's MAID survey 68 times questions its validity

Ken wanted to know how far he could push the virtual envelope — so he filled out a federal government survey on medical assistance in dying 68 times.

Ken, whose name the CBC isn't publishing because he fears threats or harassment, decided last month to see whether he could fill out the online MAID survey more than once.

• 'These are complex issues': Federal government's survey on medically assisted death raises concerns

"I wanted to see if there was due diligence in collecting the data," said Ken, an Ontarian with a sociology background.

"A standard for me as a sociologist is one survey for one customer, so to speak. As soon as you have somebody doing it multiple times, you now skew the data."

The federal government says 300,000 people filled out the MAID survey from Jan. 13-27, as part of its public opinion process for a new medical assistance in dying law it aims to table by the end of the month.

The Liberal government is working on rewriting the law after the Superior Court of Quebec found that limiting medical assistance in dying to only those who face a "reasonably foreseeable" natural death is unconstitutional. That ruling is set to come into effect on March 11.

Ken took screen shots of 50 of the 68 surveys he filled out, with dates and times, as proof.

He even filled the survey out on Jan. 28, a day after it was supposed to close, he said.

"I wanted to see how far this could go without being shut down," he said. 

"I was very careful to fill out the radio buttons the same, although I asked critical questions in the boxes where they had them. Among them: 'Why am I being able to fill this out multiple times? And how do you expect that data to be correct?'"

He didn't get a response, he said. 

He even tweeted to politicians that there was a problem with the survey.

Other people also have told CBC they were able to fill out the questionnaire more than once.

Curtis Brown with Probe Research in Winnipeg doesn't think that's unusual.

There are two kinds of surveys, Brown said.

"A public consultation survey like this one is designed to get people's feedback and it is quite open and people have a lot of opportunities to weigh in," he said.

"That means they may respond more than once if they really feel passionate about a particular issue."

The other kind of survey is statistically valid, questioning a number of people and making sure that group reflects the larger population.

"So it is weighted by the right number of men and women, people of certain age groups, socio-economic backgrounds, and from that you can extrapolate how the whole population would feel if everyone had an opportunity to be interviewed," Brown said.

The Justice Department said the questionnaire was more accurately a consultation and not a survey with a representative sample of Canadians.

A summary of the results will be made available once the feedback has been analyzed, Justice Department officials said.

They will include a section detailing the measures taken to ensure the integrity of the data as well as an evaluation of those methods, department officials said.

Ken questions whether 300,000 Canadians did in fact respond, or if the number was a lot lower because people could participate multiple times.

• Canadians want expanded access to medical assistance in dying, says Lametti

Ken thinks the survey should be thrown out and the data shouldn't be used.

Brown wouldn't go that far.

"My words would be take it with a grain of salt. Take it with the idea that the 300,000 people may not be fully 300,000 people," Brown said.

It's also not representative of the entire population, he said — just those who took time to respond.

The Justice Department said in a written statement that when multiple responses were received from the same IP address, they were reviewed to determine whether they were from a large organization where multiple people might respond, or repeated submissions of the same information in an attempt to affect the results.

"These instances were not common enough to have an impact on overall statistics," the statement said.

It's not clear if those with multiple responses from the same IP address were deleted before the final number of survey responses was tallied.

Lobbyists' opinions differ

National organizations for and against MAID have different opinions about people being able to fill out the survey more than once.

Jim Cowan, chair of the board for the non-profit organization Dying with Dignity Canada, which supports MAID, expressed concern but said it doesn't invalidate the results .

"I am concerned people could respond more than once. I believe if you are going to do polling and having consultations, you want to make sure they are as active and reflective as they can be," Cowan said.

"If a few people did vote more than once, that doesn't affect the validity of the overall result."

Alex Schadenberg, executive director of the non-profit Euthanasia Prevention Coalition, which opposes MAID, disagrees with Cowan.

"The data in this survey was unreliable because it did not control the number of times a person could respond," he said.

"On top of that, quite a few of our supporters refused to fill it out because when they opened up the questionnaire, they felt a lot of the questions were not written in a neutral manner."

Brown said while he believes the consultation survey is valid, more work needs to be done.

"I would hope and I would expect that the government of Canada would also gather insights from Canadians through a statistically valid survey on this issue, and the results be released to the public as part of the dialogue they have with the public to know what Canadians are thinking," Brown said.