Facial recognition tech is allowing stores to reward customers

At first, the footage from the camera mounted over the grocery store entrance in Philadelphia looks as mundane as you'd expect; a steady stream of nondescript customers flows in.

But as each shopper walks by, a circle appears around their face on the monitor we're watching almost 4,000 kilometres away.

The shoppers don't know it, but a computer is scanning their faces and comparing their features to those of known shoplifters. This facial recognition software is being used in stores across North America.

• Face-reading tech could make shopping more convenient — and creepier
• Facial recognition tool screens travellers for border officers

"The science being applied here is machine vision, and the system is trained to recognize differences in human faces," says Joe Rosenkrantz, CEO of the Los Angeles-area company FaceFirst. He walks toward a camera mounted in one corner of the conference room.

"My face is being tracked," he says. "Every image of my face is being run through the system and tested against a watch list." 

Facial recognition usually uses one of two techniques. In one, the camera tracks the face, finds key points such as the eyes and the nose, and calculates the distances between those points. The other method analyzes regions on the face, comparing contours and textures, essentially turning the face into a unique topographical map.

"All this happens at over a million comparisons per second, so we're able to match people against a very large database into the millions," Rosenkrantz says. "It wouldn't be possible without today's computing power."

At the moment, facial recognition technology is used mostly for security. But its developers are selling it to retail stores, which would be able to recognize customers, not just shoplifters.

Pierre Racz, CEO of Montreal's Genetec, attends a Las Vegas trade show to demonstrate his company's facial recognition software. At the Genetec booth, his colleague takes my picture and enters my name into the database.When I walk past the camera, the computer instantly flags me and displays my name.

Racz says companies use this technology to analyze shoppers' behaviour: where they go, what grabs their attention.

"It lets them understand the demographics of the shopper, the dwell time of the shopper," Racz says. "So is the shopper interested in a given display, and is this shopper a male or a female?"

Racz says you don't have to worry about your privacy, because the computer will be able to recognize you, but no one else will.

"We store two copies of the video," Racz says, "one which is encrypted, and which is only available if the chief privacy officer wants to decrypt that video. And the other copy is blurred."

He points to a split screen on a monitor that's showing footage from a camera mounted at the company's booth.

"The left-hand side, the video from inside the booth, that is unblurred, and on the right hand side you can see that it's blurred," Racz says.

"So the people who actually do the shopping are unrecognizable by the standard clerks. However, their behaviour is recognizable. So if they're congregating and waiting for a clerk's attention, well, someone who's looking at those cameras would be able to figure it out.

"All technology can be used for good or for evil. But actually we're using these cryptographic techniques so the public can have their security cake and eat it too."

• MasterCard unveils more details about 'selfie pay'
• After Paris, there will be no stopping the surveillance state

Racz won't list his clients, but says facial recognition is being used by major Canadian retail and coffee chains.

Research in the UK. suggests 30 per cent of retail stores are experimenting with facial recognition. Geoffrey White of the Public Interest Advocacy Centre in Ottawa says that's probably true in Canada as well.

"There are no signs in stores saying, 'We use facial recognition technology,'" White says. "I think if Canadians were aware of that it's becoming a common practice, I think there will be some concerns."

Many companies are developing customer reward programs that allow shoppers to submit their pictures in exchange for deals when they walk into the store and their faces are recognized. Customers can opt in or opt out.

But according to Alvaro Bedoya, executive director of Georgetown University's Center on Privacy and Technology, there are no regulations covering the use of this technology.

"Imagine a pharmacy tracking your purchases, every single purchase you make, even the ones you make in cash, or imagine a stranger pointing a smart phone even being able to identify you by name and pull up your online dating profile from across the room without speaking to you," says Bedoya. "This is what facial recognition can do."

During talks to set voluntary rules for facial recognition, Bedoya tried to convince the U.S. Department of Commerce that companies shouldn't be allowed to use this technology without the permission of the people being tracked.

"Not a single company or industry association that was part of these negotiations would accept that simple request," Bedoya says. "Instead, they all denied it, they all refused to accept it. and I think that's just beyond the pale."

So is all this legal?

"That's the biggest unknown at this point, in terms of the practice in Canada," White says. "And I think it's the area where the privacy regulators need to look closely." 

Your face, he argues, could be considered sensitive personal information. 

"And then there is a question about what the level of informed consent is for the collection of that personal information," White says.

"It's also combining your face with your identity with information about you as you move through time and space. It's a question of what other information sources are being combined with your face, where are they coming from or where it's been shared. And who knows what about you and your habits, and who is doing what with that information? That's the real question for Canadians."

Designers of facial recognition systems like Rosenkrantz say it's just another way to reward loyal customers.

"Imagine a world where you opted in a particular website to a store that you love and the concept is that the next time you return, you receive a text message with a coupon offering you a discount that you would normally not be eligible to receive," Rosenkrantz says.

"So you have a particular class of shoppers who want a higher level of service, desire higher discounts and are willing to opt into a program like that."  

Is the corresponding loss of privacy worth it?

"I think so," Rosenkrantz says. "I think the benefit of society is making the world a smaller place. It's kind of like living in a small town where everybody knows your name."