Why A.I. companies are paying random strangers to take videos of themselves doing mundane tasks

Tom Simonite's first paid gig working for an A.I. startup firm took place in the baby food section of a San Francisco grocery store.

While a friend stood by and filmed him, Simonite reached out and grabbed a pack of broccoli-flavoured snack puffs off the shelf.

"I approached the shelf and kind of reached out my hand, grabbed these snack puffs — they were broccoli and apple flavour — and took them down and put them in my cart, and that was it," he recalls.

I opened a freezer and pretended to take out some ice cream. I pushed a cart from the left to the right.- Tom Simonite, senior editor at Wired magazine

Simonite isn't planning to quit his day job as a senior editor at Wired magazine any time soon, but he figures he could make as much as $11.75 an hour doing odd jobs like these — all in the name of artificial intelligence research.

In the decades to come, machine learning technologies are expected to displace millions of workers around the world.

But in the shorter term, at least one unusual new occupation has sprung up as a result of A.I. research.

In an effort to build the data sets they need to make their algorithms smarter, researchers are recruiting people around the world to film themselves doing basic tasks, like pulling baby food off a shelf.

"You could sum it up by saying that A.I. software isn't really that smart," Simonite says. "You have to teach it to understand things in the world."

Odd jobs for A.I.

The short, seconds-long clips Simonite filmed in the grocery store are part of a broader field of work known as 'crowd acting,' a term coined by a German-Canadian startup called 20 Billion Neurons.

Pulling broccoli puffs off the shelf was just one of the tasks Simonite took on.

"I opened a freezer and pretended to take out some ice cream. I pushed a cart from the left to the right. I pushed a cart from the right to the left."

"After we got ten short clips like that we scurried away, and nobody noticed us."

As Simonite tells Day 6 host Brent Bambury, A.I. startups hope to use the clips to teach algorithms how to identify human tasks and behaviours, just like they identify objects in still photographs.

"The idea is that it will learn to recognize when people are taking items off shelves and putting them in carts," Simonite says.

But what, exactly, can an algorithm learn from a video of someone taking food off a shelf?

"In the case of me sneaking around a supermarket, the idea is that in the future we might want to have robots that can watch what we're doing out in a store and maybe help us," he explains.

"Or maybe there will be a camera system that needs to understand the difference between someone shopping and shoplifting, for example. Things like that."

Crowdsourcing data

A.I. startups like 20 Billion Neurons recruit 'crowd actors' through crowdsourcing sites like Amazon Mechanical Turk, a website that allows companies to post tasks online.

"If you go on there, you'll see thousands and thousands of jobs put on there by companies that need menial digital work to be performed."

On the Mechanical Turk platform, workers can browse through thousands of odd jobs, many related to artificial intelligence research, and pick the ones that appeal to them most.

It could tell when you're looking sleepy, if you're a driver, and that could save your life.- Tom Simonite

In addition to filming their own clips, workers can sign up to review other people's tasks.

Simonite was able to review plenty of clips that had been uploaded by people all over the world, including many in India.

One of the tasks he reviewed involved people sitting in cars and pretending to fall asleep or read a book.

"I guess the idea is that if a car could tell what you're doing, it could turn the light on when you open your book, for example. That might be helpful if you're a passenger."

"It could tell when you're looking sleepy, if you're a driver, and that could save your life."

Occupation versus exploitation

Simonite estimates that he made about $3.50 for the ten clips he filmed in the supermarket — roughly equivalent to a $4.50 an hour wage, once the time spent uploading the clips and walking around the store was taken into account.

It seems like the way the market is set up is kind of tilted toward the companies looking for the job, and maybe tilted against the people who are looking for work.- Tom Simonite

Some crowd actors are able to make more, he says.

"I spoke to a man called Casey who lives in Tennessee, and he told me I've been doing this acting job in the supermarket all wrong."

According to Simonite, crowd actors like Casey camp out in Walmart for hours at a time, working in pairs to film clips in large quantities and uploading them to the site.

"He calculated that the two of them earned around $11.75 an hour — which he said is a lot better than people are earning in fast food restaurants where he lives."

But that's not the case for the majority of workers on Amazon Mechanical Turk. Earlier this year, researchers at Carnegie Mellon University pegged the median average wage of 'Turkers' at just two dollars an hour.

"Overall, the pay is not very good," Simonite says.

"It seems like the way the market is set up is kind of tilted toward the companies looking for the job, and maybe tilted against the people who are looking for work."

Simonite saw other signs of troubling activity on the site while reviewing clips from workers around the world.

"In a few, I saw children doing the job," he says. "So, you know, a young boy in school uniform sat there pretending to fall asleep."

Amazon requires its workers to be 18 years old before they do the work. But Simonite says he had no way to report the clips for child labour.

"This was frustrating, because my only options were to say: 'Yes, he did the job' or 'No, he did not do the job.'"

"But the company told me they have a separate process that would screen those out, and I guess they are incentivized to do that, not least because Amazon would presumably kick them off if they were using the platform too much in the wrong way."

A shortcut to automation?

Simonite says there's no question that A.I. startups will be able to use these data sets to teach their algorithms to identify complex tasks.

"I don't think there's a question about whether this actually works," he says. "What we haven't seen is broad use of that capability in the real world."

Simonite sees some irony in the fact that A.I. researchers are paying such a low wage while developing technology that could one day replace quality jobs.

"In order to make these software systems, which at the moment, you know, are subject to a lot of hype and are held up as this gleaming pinnacle of technology, you need a lot of people doing the grunt work of clicking to say, 'this is a cat; this is a dog; this is me taking down a can of baby food.'"

On the other hand, he sees plenty of applications for the technology that won't threaten jobs at all.

"No one is paying someone to turn the light on in their car, or to wake them up when they look drowsy behind the wheel," he says. "There are lots of benefits here that seem pretty clear-cut."

To hear the full interview with Tom Simonite, download our podcast or click the 'Listen' button at the top of this page.