Last week Mark Zuckerberg came under attack from Roger McNamee, his former adviser. McNamee, who has criticised the Facebook CEO before, lambasted Zuckerberg and his corporation for its relentless pursuit of user data with increasingly illicit and destructive methods.
“To feed its AI and algorithms, Facebook gathered data anywhere it could. Before long, Facebook was spying on everyone, including people who do not use Facebook,” McNamee wrote. These operations, he said, were honed to manipulate user engagement with practices that were eventually commandeered by bad actors to infiltrate the national consciousness and disfigure political discourse.
McNamee’s analysis is a dangerous category error. While both the Russian government and the plutocrat Robert Mercer, part owner of the now defunct Cambridge Analytica and donor to Donald Trump’s presidential campaign, learnt how to massage the complex secretive machine that Facebook built, these operations and the digital apparatus through which they function do not begin or end with Facebook. Instead, they are key elements in a new economic logic that I call “surveillance capitalism”.
Such practices were invented at Google, travelled to Facebook, engulfed Silicon Valley and have since spread through every economic sector. It would be a grave mistake to assume this is merely a Facebook phenomenon. Regulate Facebook. Break it up. Demand a change of leadership. Surveillance capitalism will not skip a beat. On the contrary, it will quickly fill the void, camouflaged by a new cast of characters and a fresh, trendy glossary of euphemisms.
I began studying the digital shift in 1978, focusing on the workplace. By the time my first book, In the Age of the Smart Machine, was published in 1988, I understood that the path to the digital future would be fraught with conflicts over who had access to new knowledge, who had the authority to decide and who had the power to enforce that authority.
By now, these entangled dilemmas have surged beyond our workplaces to flood every aspect of our lives. Information and communications technologies are more widespread than electricity, reaching three billion of the world’s seven billion people. Their roots run through the necessities of daily life, mediating nearly every form of social participation.
It soon became clear to me that surveillance capitalism diverged from many norms and practices that define the history of capitalism, especially the history of market democracy. Something startling and unprecedented had emerged and its consequences will shape the moral and political milieu of 21st-century society and the values of our information civilisation.
Surveillance capitalism was invented in the teeth of the dot.com bust, when a fledgling company called Google decided to try and boost ad revenue by using its exclusive access to largely ignored data logs — the “digital exhaust” left over from users’ online search and browsing. The data would be analysed for predictive patterns that could match ads and users. Google would both repurpose the “surplus” behavioural data and develop methods to aggressively seek new sources of it.
According to its own scientists’ accounts, Google’s new methods were prized for their ability to find data that users had opted to keep private and to infer extensive personal information that users did not provide. These operations were designed to bypass user awareness and, therefore, eliminate any possible “friction”. In other words, from the very start Google’s breakthrough depended upon a one-way mirror: surveillance. The new methods were invented and deployed from 2001 to 2004 and held in strict secrecy. Only when Google went public in 2004 did the world learn that on the strength of these new operations Google’s revenues had increased by 3,590 per cent.
This shift in the use of surplus behavioural data was a historic turning point. Google had found a game-changing, zero-cost asset that could be diverted from service improvement towards a genuine commercial exchange. Surveillance capitalism soon migrated to Facebook and rose to become the default model for capital accumulation in Silicon Valley, embraced by every start-up and app.
It was rationalised as a quid pro quo for free services but is no more limited to that context than mass production was limited to the fabrication of the Model T. It is now present across a wide range of sectors, including insurance, retail, healthcare, finance, entertainment, education and more. Capitalism is literally shifting under our gaze.
Surveillance capitalism soon migrated to Facebook and rose to become the default model for capital accumulation
Perhaps the most poignant illustration of this is found in the birthplace of mass production: the Ford Motor Company. A hundred years ago, pioneer capitalists such as Henry Ford bent to the task, shaping a new century of mass consumption. Ford understood that farmers and shopkeepers wanted automobiles too, but at a price they could afford. In his world, customers and workers were linked in a cycle of production and sales that combined low-cost goods with consumption-worthy wages immortalised by Ford’s Five-Dollar Day.
In November 2018, Jim Hackett, Ford’s chief executive, pointed to a new paradigm for car manufacturers. He told an interviewer: “The case I would make is that we have as much data in the future coming from vehicles, or from users in those vehicles, or from cities talking to those vehicles, as the other competitors [such as Tesla] that you and I would be talking about . . . My belief is: we have 100 million people in vehicles today, that are sitting in Ford blue-oval vehicles. That’s the case for monetising opportunity versus an upstart who maybe has, I don’t know, what, they got 120,000 or 200,000 vehicles in place now. Just compare the two stacks: which one would you like to have the data from?”
Once customers are reinvented as data sources, it’s easy for Hackett to imagine the next step in which the data that streams from cars in real time is combined with Ford’s financing data, where, he says: “We already know . . . what people make . . . we know where they work; we know if they’re married. We know how long they’ve lived in their house.” He concludes: “And that’s the leverage we’ve got here with the data.” As one industry analyst puts it, Ford “could make a fortune monetising data. They won’t need engineers, factories or dealers to do it. It’s almost pure profit.”
This is where we live now — a world in which nearly every product or service that begins with the word “smart” or “personalised”, every internet-enabled device or vehicle, every “digital assistant” — each is a supply-chain interface for the unobstructed flow of behavioural data.
It has long been understood that capitalism evolves by claiming things that exist outside of the market dynamic and turning them into market commodities for sale and purchase. Surveillance capitalism extends this pattern by declaring private human experience as free raw material that can be computed and fashioned into behavioural predictions for production and exchange.
In this logic, surveillance capitalism poaches our behaviour for surplus and leaves behind all the meaning lodged in our bodies, our brains and our beating hearts. You are not “the product” but rather the abandoned carcass. The “product” derives from the surplus data ripped from your life.
In these new supply chains we may find signs of the people you share our life with, your tears, the clench of his jaw in anger, the secrets your children share with their dolls, our breakfast conversations and sleep habits, the decibel levels in my living room, the thinning treads on her running shoes, your hesitation as you survey the sweaters laid out in the shop and the exclamation marks that follow a Facebook post, once composed in innocence and hope. Nothing is exempt, from “smart” vodka bottles to internet-enabled rectal thermometers, as products and services from every sector join the competition for surveillance revenues.
These are siphoned from your daily life in ways that are designed to keep you ignorant. In the US, breathing machines used by people who suffer from sleep apnoea secretly funnel data to the beleaguered sleeper’s health insurer, often to enable the company to refuse payment. Some cell phone apps record your location as often as every two seconds for sale to third parties.
In July 2017, iRobot’s autonomous vacuum cleaner, Roomba, made headlines when the company’s CEO, Colin Angle, told Reuters about its data-based business strategy for the “smart home”, stating that its stock price increased after its proposal to share free floor plans of customers’ homes, scraped from the machine’s new mapping capabilities.
At a certain point, surveillance capitalists discovered behaviour modification: digitally mediated real-time interventions that nudge consumers in the direction of desirable outcomes. As one data scientist explained to me: “We can engineer the context around a particular behaviour and force change that way . . . We are learning how to write the music, and then we let the music make them dance.”
Examples include the gentle herding of Pokémon Go players to eat, drink and purchase in the restaurants, bars and fast-food joints that pay to play in its behavioural futures markets, or the ruthless expropriation of surplus from Facebook profiles for detailed “psychological insights”, which, according to a 2017 internal company report, allow advertisers to pinpoint the exact moment when a teenager needs a “confidence boost” and is, therefore, most vulnerable to a specific configuration of advertising nudges and cues.
Surveillance capitalists produce deeply anti-democratic asymmetries of knowledge and the power that accrues to knowledge. They know everything about us, while their operations are designed to be unknowable to us. They predict our futures and configure our behaviour, but for the sake of others’ goals and financial gain. This power to know and modify human behaviour is unprecedented.
Often confused with “totalitarianism” and feared as Big Brother, it is a new species of modern power that I call “instrumentarianism”. Instrumentarian power can know and modify the behaviour of individuals, groups and populations in the service of surveillance capital. The Cambridge Analytica scandal revealed how, with the right knowhow, these methods of instrumentarian power can pivot to political objectives. But make no mistake, every tactic employed by Cambridge Analytica was part of surveillance capitalism’s routine operations of behavioural influence.
It did not have to be this way. In the year 2000, computer scientists and engineers collaborated on a project called the Aware Home. They envisioned a “human-home symbiosis” in which animate and inanimate processes would be captured by a network of “context aware sensors” embedded in a house and by wearable computers used by its inhabitants.
The system was designed as a simple closed loop controlled entirely by the occupants. Because the house would be “constantly monitoring the occupants’ whereabouts and activities . . . even tracing its inhabitants’ medical conditions”, the team concluded “there is a clear need to give the occupants knowledge and control of the distribution of this information.” All the information was to be stored on the occupants’ wearable computers “to ensure the privacy of an individual’s information”.
Surveillance capitalists want us to believe that their trajectory towards the digital future is inevitable. But that is not the case
Now fast forward to 2017, when two University of London scholars published a detailed analysis of a single “smart home” device, the Google-owned Nest thermostat. They determined that were one to enter into the Nest ecosystem of connected devices and apps, each with their own burdensome terms of service for third-party data sharing, the purchase of a single Nest thermostat would entail the need to review nearly a thousand so-called “contracts”. Should the customer refuse to agree to Nest’s stipulations, the terms of service indicate that the functionality and security of the thermostat may be deeply compromised, no longer supported by the necessary updates meant to ensure its reliability and safety.
Surveillance capitalists want us to believe that their trajectory towards the digital future is inevitable. But that is not the case. Today we might mourn the innocence of the Aware Home but, like a message in a bottle from a bygone age, it tells us something important. Once we were the subjects of our lives, now we are its objects. The Aware Home is a record of what we have lost and what we must now find again: the rights to know and decide who knows about our lives and our futures. Such rights have been and remain the only possible foundations for human freedom and a functional democratic society.
In late October 2018 Tim Cook, the chief executive of Apple, stood in the European Parliament and decried the “data industrial complex” with its “stockpiles of personal data” that serve only to “enrich the companies that collect them”. “This is surveillance,” he stressed, and it allows companies “to know you better than you may know yourself . . . This crisis is real. It is not imagined, or exaggerated, or crazy. And those of us who believe in technology’s potential for good must not shrink from this moment.”
Not surprisingly, some are cynical about the depth of Cook’s commitment to “not shrink from this moment”. They see a marketing campaign aimed at distancing the company from the shadow that has fallen over the tech sector. Others cite Apple’s own inconsistencies during the past decade, including an iPhone that defaults to Google Search, the storage of user data in Chinese servers, a lack of transparency regarding many of Apple’s data collection practices and protections, and many other contradictions. The FT’s own Tim Bradshaw and Mehreen Khan note that it is easier for Apple to take a strong stand on privacy when the company’s revenues depend on the sale of its devices, not targeted advertising.
These are each important criticisms, but for many reading Cook’s remarks that day, they provoked a profound sense of hope — that somebody in the industry finally had the courage to speak out. In 46 of the most prominent 48 opinion surveys administered in the US and Europe between 2008 and 2017, substantial majorities supported measures for enhanced privacy and user control over personal data. (Two early surveys were less conclusive, because so many participants said they did not understand how or what personal information was being gathered.)
An important 2009 survey found that when people were informed of the ways that companies gather data for targeted online ads, more than 73 per cent rejected such advertising. A substantial 2015 survey found 91 per cent of respondents disagreed that the collection of personal information “without my knowing” was a fair trade-off for a price discount.
People are crying out for an alternative path to the digital future, one that will fulfil our needs without compromising our privacy
Tech firms typically dismiss these results, pointing to users’ actual behaviour and the spectacular revenues it produces as justification for the status quo. Recall former Google CEO Eric Schmidt’s infamous 2009 brush-off: “If you have something that you don’t want anyone to know, maybe you shouldn’t be doing it in the first place.” Experts call the gap between attitudes and behaviour “the privacy paradox”, but it’s really not a paradox. It is the predictable consequence of the pitched battle between supply and demand, expressed in the difference between what surveillance capitalism imposes upon us and what we really want.
Today’s historic gap between supply and demand is a call to action for business leaders who have the foresight and conviction to stand against the tide. Resisting surveillance capitalism is not simply “the right thing to do”. Just about everyone connected to the internet is crying out for an alternative path to the digital future, one that will fulfil our needs without compromising our privacy, usurping our decisions and diminishing our autonomy. Given the forces at play, the person with the best chance of forging an alternative would need considerable commercial and political heft behind them. Which could be where Tim Cook comes in.
A 2017 study of stock market returns concluded that Apple had generated more profit for investors than any other American company in the 20th century. If Cook is willing to truly act on his words in Brussels, Apple could drive an alternative path towards the digital future, reuniting capitalism with the people it should serve. Cook has said that supposed conflicts between privacy-versus-profits or privacy-versus-innovation are false choices. In fact, the historical pattern suggests that today’s disjunctures between supply and demand signal a dramatic opportunity for a qualitative leap forward in the evolution of capitalism.
We have stood at this kind of precipice before. “We’ve stumbled along for a while, trying to run a new civilisation in old ways, but we’ve got to start to make this world over.” It was 1912 when Thomas Edison laid out his vision for a new industrial civilisation in a letter to Henry Ford. Edison worried that industrialism’s potential to serve the progress of humanity would be thwarted by the power of the robber barons and the monopolist economics that ruled their kingdoms. He decried the status quo marked by the “wastefulness” and “cruelty” of US capitalism.
Both Edison and Ford understood that the modern industrial civilisation for which they harboured such hope was hurtling towards a future marked by misery for the many and prosperity for the few. They also understood that the moral life of industrial civilisation would be shaped by the practices of capitalism.
Everything would have to be reinvented: new technologies, yes, but these would have to reflect new ways of fulfilling people’s needs; a new economic model that could turn those new practices into profit; and a new social contract that could sustain it all. The citizens, consumers, executives, workers, lawmakers, jurists, scholars, journalists, managers and public officials who undertook this effort stepped into unknown territory.
Our time demands this kind of creative leap into the unknown that can bend the trajectory of the digital future back towards people. Without a courageous and creative response, surveillance capitalism will continue to fill the void. If surveillance capitalism is to be interrupted, tamed, even outlawed, we will need new laws, regulations, and forms of collective action tailored to specific mechanisms.
For these reasons and a thousand more, I call on Tim Cook and other business leaders to take up the challenge of carving a new road home. You will not be lonely. Those of us who live in the gap between home and exile will not shrink from this moment. The fight for a human future belongs to all of us.
Shoshana Zuboff is the Charles Edward Wilson Professor Emerita at Harvard Business School. “The Age of Surveillance Capitalism: The Fight for the Future at the New Frontier of Power” (£25) is published on January 31 by Profile Books
Follow @FTMag on Twitter to find out about our latest stories first. Subscribe to FT Life on YouTube for the latest FT Weekend videos
Letter in response to this article:
Copyright The Financial Times Limited . All rights reserved. Please don't copy articles from FT.com and redistribute by email or post to the web.