Why everyone needs to chill out about month-to-month job numbers from StatsCan

Every month, Statistics Canada puts out new numbers on employment, and every month the data is politically weaponized.

"Nearly 22,000 full-time jobs were lost last month alone," United Conservative Party Leader Jason Kenney tweeted in August, along with an orange-tinted image of a job-seeker's resume and the accusation that Alberta's governing NDP are "out of touch."

One month later, the numbers showed a huge job gain, and Kenney's guns fell silent. This time, the data gave ammunition to Premier Rachel Notley, and it was her chance to to return fire.

"Alberta led the country in growth last month," she tweeted in September, along with a chart showing an ostensible increase of 16,000 jobs and the assertion that "things are looking up."

Politicians, of course, will put the best spin on the numbers that they can. And partisans will play up the data that suits their interests while downplaying the data that doesn't.

The rest of us are left to wonder: What's really going on?

There's a lot of genuine misunderstanding out there, and understandably so. This stuff is complicated, even without the spin.

What we know — and what we don't

The fact of the matter is that we don't actually know how many jobs were lost or added in any given month. We simply can't measure things that precisely.

What do we have is an estimate. It's known as the labour force survey, and it's not perfect. But the good thing about estimates is that, the more of them you have, the better picture you get of reality.

This is why statisticians and economists say a single month's worth of employment data should be taken with a grain of salt. Ideally, they say, we should be looking at several months in a row, at least, to average out margins of error and get a better sense of long-term trends. And, to get an even better picture, the labour force survey should be used in combination with other measures.

Fair enough. But, in the real world, the daily news cycle and the pace of politics means we can't simply wait for several months to talk about the numbers.

So, in the meantime, what can we say about the latest jobs figures from StatsCan?

We have to talk in terms of probability.

Are we feeling lucky?

As we saw from the duelling politicians' tweets, large monthly swings are common in the employment data.

And the latest swing was up.

Way up.

There were 23,700 more people employed in Alberta in November compared with October, according to the most recent labour force survey results. Canada over all added 94,100 net jobs for its largest monthly increase since March 2012.

• Alberta unemployment falls by full percentage point to 6.3%

But as we've also seen, we don't know if 23,700 new jobs were actually created. In fact, the actual number is almost certainly not 23,700. It's somewhere above or below that.

How much above or below? This is where the fine print gets important.

Buried in the footnotes of the Statistics Canada data release are the details on methodology, which tell us the estimate comes with a "standard error" of 9,900 jobs.

If you took Statistics 101 in university, you might recognize this term and what it means.

For the rest of us, there's Trevor Tombe to explain.

Statistics refresher

The University of Calgary economist says thinking — and talking — in terms of probability always gets tricky.

But here's one way he likes to look at the employment numbers.

Let's say — for the sake of the explanation — that the actual employment change was zero. That, in reality, there was no job gain and no job loss in November.

If that were the case, then there would be a 0.8-per-cent chance the labour force survey would show an employment gain of 23,700 or more.

If you ran the same survey a million times, Tombe says, the results would be spread out in the bell-like shape of the blue curve in the chart below. The higher the blue line, the more often you'd get that result.

As you can see, very few survey results would deviate by +23,700 or more from the actual change in jobs. Those outlier results are represented by that tiny pink area below the blue line on the far right of the chart.

In other words, it's possible, but extremely unlikely. It's roughly equivalent to flipping a coin seven times in a row and having it come up tails every time.

Of course, in reality, we don't know what the actual change in employment was.

What we do know is that the labour force survey result we got for November showed an increase of 23,700 people with jobs.

And that one number, alone, has a lot of limitations.

How StatsCan says to interpret the data

Back to that "standard error" of 9,900.

As Statistics Canada explains in another footnote on the labour force survey: "Using this method, the true value will fall within one standard error of the estimate approximately 68 per cent of the time, and within two standard errors approximately 95 per cent of the time."

You've probably heard the 95-per-cent figure more often when it comes to surveys. That's the "19 times out of 20" disclaimer that you typically see attached to political polling.

But in its own analyses, Statistics Canada tends to focus on results that are "statistically significant at the 68-per-cent confidence level." This provides some valuable insights into what's likely happening with the short-term employment situation but, of course, it's not a rock-solid description of what's actually happening.

That's why Vincent Hardy, an analyst with the labour-data division of Statistics Canada, says it's important to look at more than just one month worth of data.

"It may be interesting to look at specific periods, but we also recommend our users not only look at a specific point in time but tend to focus on long-term trends," he told CBC News.

It's also important to look at other measures that complement the labour force survey, he says, such as the Survey of Employment Payroll and Hours, which uses a more accurate method for counting jobs but comes with its own limitations, such as excluding self-employed workers. (You can read much more on the differences between the two measures here.)

So what do the long-term trends in the data show?

Years of data, instead of months

Overall, says Tombe, there's no question that employment has been trending upward, in general, for the past two years.

There have been ups and downs over periods of months but, overall, the number of people working in Alberta has been on the rise, since bottoming out in mid-2016 amid the last recession.

Here's what the last five years look like, at a glance.

It can be frustrating, to some, that the monthly numbers aren't more precise. That there's no easy answer to the question: How many jobs did we gain (or lose) last month?

But Tombe says that's simply the practical reality of trying to figure out what's happening with something as complicated and fast-moving as a provincial economy.

"It's impossible to know what millions of people are doing at any given moment in time," he says.

"There's imprecision in everything."

But, he adds, that doesn't mean we should throw out the whole lot. The data may not be perfect, but it's still useful.

"And we face that in our day-to-day lives all the time," he says.

"If you look on Google Maps and ask it how long it's going to take you to get to that appointment downtown, and it says 15 minutes, of course it's wrong. It'll be off by a minute or two because you hit a red light. That doesn't mean you throw out Google Maps and never use it for anything. It still provides very valuable insight."

Similarly, the month-to-month employment estimates tell us something, even if they don't individually give us the full picture. Over time, a clearer image will emerge.

In the meantime, it's important to remember the limitations that each monthly release comes with, and not get too excited (or despondent) about a single number.

• Read more articles by CBC Calgary, like us on Facebook for updates and subscribe to our CBC Calgary newsletter for the day's news at a glance