Surveillance capitalism: Who is watching us online — and why?
Pulling back the digital curtain on the power players trading in 'human futures'
*Originally published on November 10, 2020. This is Part 2 of the six-part 2020 Massey Lecture series Reset: Reclaiming the Internet for Civil Society. Find the rest of the series here.
We know we're being watched.
In the age of social media, it's not a mystery that much of what we see online is tailored to our beliefs and desires. Our searches and interactions influence what shows up on our screens. Getting engaged? Check out these rings. Working from home? Here's the best leisurewear. Feeling stressed? Try this meditation app.
The ads that personalize our internet browsing are obvious examples of how "attention merchants" vie for our data, but the more insidious actors are the ones we don't see. And unfortunately, our personal information is up for grabs with them as well.
Once upon a time, the global economy was built primarily on the trade of tangible natural resources. Materials such as coal, iron, and natural gas were extracted from the earth and sold to build and power everyday life, from factories to transportation.
Today, the economy is driven equally by raw materials, but the most powerful — and valuable — resources have changed. This new economy is built on our attention span and driven by what we watch, what we search for, and the treasure trove of data compiled by an ever-expanding network of sensors all around us. Our behaviour — and our psychology — is up for sale.
It is what Shoshana Zuboff, professor emerita at Harvard Business School, calls "surveillance capitalism," wherein social media companies monitor, archive, analyze, and market as much personal information as they can siphon from those who use their platforms in order to extract value.
"A mere 20 years ago, this would have been the stuff of science fiction," Citizen Lab founder Ron Deibert said in his second Massey Lecture, The Market for Our Minds. "Now, it's becoming an everyday reality."
So who exactly is watching us? And why is our online behaviour so valuable to them?
Who's watching you
Early into the social media era, whistleblower revelations indicated that the government was tracking us more closely than we might have imagined.
Edward Snowden's National Security Agency leaks, as Deibert notes, "showed that government spy agencies had been quietly thriving in the shadows on an epic scale."
But Snowden's exposés revealed far more than just what the government was up to.
By 2013, Deibert said, "commercial data collection efforts dwarfed what any spy agency could do alone, even one as well-resourced as the NSA and its estimated $11 billion [US]."
As Snowden issued warnings about widespread surveillance, social media platforms were emerging as the defining feature of the internet age. Designed to be addictive, these supposedly "free" platforms exploded in scale, popularity and power in the span of less than a decade.
You are the product, not the customer
The oft-repeated adage bears repeating: if something is free, you are the product.
Indeed, the primary customers of "free" services like Google, Facebook and Instagram are not us, their users. Instead, Deibert says, "the real customers are other businesses that are interested in predictions of human behaviour generated by the social media platforms and the data analytics machinery that surrounds them."
"We are simply a means to a larger, commercial end," he said in his Massey lecture.
And that commercial end is substantial, putting tech companies comfortably among the top earners on earth, even though they don't produce any actual content themselves.
"Our likes, emotions, relationships, and thoughts have become their property," Deibert said.
As a result, it's been argued that Facebook, Google and Amazon know us better than we know ourselves. But what does that claim mean? Who? The data analysts? The content moderators who sort through our posts? Mark Zuckerberg? Jeff Bezos?
These very titans of tech argue if you've got nothing to hide, you've got nothing to fear. Google's ex-CEO Eric Schmidt famously stated, "If you have something that you don't want anyone to know, maybe you shouldn't be doing it in the first place." Facebook boss Zuckerberg has argued the age of privacy is over.
Tailored ads may seem harmless. In fact, it's admittedly quite convenient to have ads pop up about the very thing you're searching for while you're looking for it.
But what happens when those same factors cause you to look for something before you might have consciously sought that something out? Or even manipulate you into seeking out something you otherwise might not have, as is often the case with recommendation engines that send us clicking down endless rabbit holes?
Through "predictive signaling," Deibert explains, our personal experiences are turned into commodities, wherein an individual's accountability, or trustworthiness, or likeliness to do something, buy something, or buy into an ideal can be predicted based on a collection of data correlations. When it comes to social media companies, your digital footprint is their fodder.
Trading in 'human futures'
Zuboff, the Harvard Business School professor, calls the trade of this data "human futures." The danger of this marketplace of human psychology is not the conveniently tailored online experiences we see — the vacation ads that pop up when you start contemplating a winter getaway, or the lure of buying cute baby clothes online just as you're considering getting pregnant. Rather, it is what happens when that information is used to manipulate and shape our preferences, choices, and behaviours.
"They don't want to just get to know us. They want to use that knowledge to change us," said Tamsin Shaw, a professor of political theory and philosophy at New York University. "All advertising changes our behaviour. It creates a demand for something, which in turn incites you to buy something. And it might do that in ways that we don't even notice."
And companies selling goods and services aren't the only ones interested in being able to target and sway the public.
As a result, "these systems that claim to capture our reality are actually shaping our reality and our understanding of the world," said Meredith Whittaker, a distinguished research scientist at New York University. She is the co-founder and co-director of the AI Now Institute, a research centre dedicated to understanding the social implications of artificial intelligence.
"There are people who are well-funded and organized who are backing the dissemination of … fascist and extremist content."
The tyranny of convenience
Media scholar Tim Wu calls this dynamic the "tyranny of convenience," where the risk lies in social media users being manipulated to push ideologies. In an open market of human futures, human psychology is up for sale and public opinion can be swayed by those willing to pay, as was the case in the Cambridge Analytica scandal.
Indeed, while targeting tools are used to pair advertising messages with the most receptive consumers, the ability to target receptive audiences can be used just as well by those who have "nothing to sell but ideas and conspiracy theories and racism and white supremacism," says John Naughton, the author of From Gutenberg to Zuckerberg: What You Really Need to Know About the Internet.
As to the question of who is intent on tracking everything we say and do, it's not those who aim to sell us lipstick or vacations that we need to be truly wary of, says Deibert. It's those who are trading in human futures "to undermine public accountability, spread social division, and foster chaos."
The impact of all this manipulation is profound. How we interact with these platforms, says Daniel Deudney, a professor of political science and international relations at Johns Hopkins University, "is going to determine the viability of liberal democracy and ultimately perhaps even autonomous human consciousness."
