No money to include the North in next international survey of literacy and numeracy

The next international survey of adult literacy and numeracy will include every province in Canada, but not the three territories. 

The Programme for the International Assessment of Adult Competencies (PIAAC) is administered by the international Organisation for Economic Co-operation and Development (OECD). According to the OECD website, the survey measures "key cognitive and workplace skills" in more than 40 countries.

Literacy and numeracy are widely accepted as predictors of financial success and engagement in a democracy. And in Canada, nearly half of adults struggle with literacy. 

The first PIAAC survey was completed in 2012, and it included the three territories. The second is underway now and is expected to be published in 2024. It will track improvement in adult literacy and numeracy, or its lack, since the first survey. According to the 2012 survey, both the N.W.T. and Nunavut lagged far behind world-wide OECD and Canadian average literacy and numeracy scores.

But to those holding the purse strings, the cost of administering the in-person survey in the North — where remote communities are often separated by roadless expanses of terrain — had become too high.

Anna Maiorino, a communications officer with Statistics Canada, said in an email that it cost approximately $3 million to include the North in the 2012 survey with the bulk of that covered by Ottawa. Costs to include the North in the next survey had risen to about $5 million. 

You're going to have a void where there wasn't one before.​​​- Alison Howard, executive director, ABC Literacy Canada

"All partners — the federal government, provinces and territories, and Statistics Canada — were aware of the estimated cost for the territories," Maiorino said. "For the second cycle of PIAAC, there was not sufficient funding to include the territories in the main survey."

Maiorino also cited concern around "response burden" in the North following recent federal census surveys. 

Alison Howard, the executive director of ABC Life Literacy Canada, a national organization dedicated to developing and delivering literacy programing, said that's unfortunate because the second round of the survey will provide a continuity of data that the North will miss out on.

"We can see, relatively speaking, whether initiatives and decisions in Canada are making a difference over time," Howard said. "And we can look to see what other countries are doing to see if their initiatives are having a greater impact, if there are lessons learned from those experiences."

Not having that data for the North means northern policy makers can't use it to adjust policy or spending on literacy in the North.

"You're going to have a void where there wasn't one before," Howard said.

Alternatives 

Maiorino said other ways to include data from the North — apart from the main survey — were being considered, "including the use of the online version of the PIAAC, Education and Skills Online."

But the online version of the survey is no replacement for the full, in-person survey.

"Although comparable in terms of outcomes, Education and Skills Online (ESO) does not replace the Survey of Adult Skills (PIAAC) for the population as a whole," wrote Claudia Tamassia, the PIAAC senior manager at the OECD.

She said the online survey is "best viewed as a complement for specific groups within the PIAAC population," and that the PIAAC survey "remains more comprehensive and better suited to inform policies by focusing on the overall population."

Advantages of the full survey include cross-country comparability and standards around data collection methods. The in-person interview — a standard adopted by all participant countries — also helps adults of low-literacy or technical ability participate in the survey. For example, interviewers help participants understand the questions, and can help record their answers to survey questions.

Highest level of education or the occupation for employed persons ... are not valid or reliable proxies for PIAAC literacy or numeracy scores.- Andreas Schleicher, OECD Education Directorate

Online surveys have other problems: an online assessment could self-select for those who already have the competency and literacy — not to mention the internet connectivity — to complete an online survey; and an online survey can't tell you about someone not able to complete an online survey, or why that person might be unable to complete it.

Tamassia, the survey project manager, said online surveys — including its own — cannot avoid these complications, "which is why the PIAAC standards require countries to conduct in-person interviews."

There are other proxies for measuring literacy. For example, a jurisdiction could use the highest academic grade achieved as a measure of literacy and numeracy.

But Andreas Schleicher, OECD education directorate, said those proxies are not a good replacement for the survey.  "While background variables, such as the highest level of education or the occupation for employed persons, do correlate with PIAAC literacy and numeracy skills, they are not valid or reliable proxies for PIAAC literacy or numeracy scores," he wrote in an email.

Howard, with ABC Life Literacy, said the problem is that education standards and resources can't be assumed equivalent across all regions of Canada. And indeed in much of the North, it is not unusual for high school graduates to require make-up studies to qualify for admission to post-secondary institutions.

Most recent data preferred

The N.W.T. 2022 budget contained at least $3.6M for literacy programing in that territory. The NWT Literacy Council received $397,000 of that funding. When asked for comment, a spokesperson for the council said in an email that the council "values the data that has come from PIACC surveys and we would be in favour of having up-to-date data." Further comments were referred to the territorial government. 

In an emailed statement, a spokesperson for the Department of Education said, "While the [territorial government] would have liked to participate in the survey, it was not possible due to fiscal constraints."

A spokesperson with the Yukon government said they had an issue with the survey's standard sample size. According to the spokesperson, "It is ineffective in small populations over large areas such as the territories and the margin for error becomes unacceptably high."

Tomassia acknowledged that appropriate sample sizes are essential, but "challenging in remote areas."

Howard, the director of ABC Life Literacy did not express concern about sample sizes. She was enthusiastic about what the data will deliver, and how educators will use it, even if the North will not be able to fully participate in the survey.

"It's something that the sector and everyone involved is very eagerly looking forward to because it's such a key piece of information for our work and to know how we can make a difference going forward," she said.

"Everyone's waiting to see what the results are going to show — if they're going to show major or minor changes to the skills levels in Canada to see where we can focus our efforts, where we're going to have the greatest impact on people's lives and work."