«Wire Design - Collage of city pedestrians and surveillance cameras. Recording Everything: Digital Storage as an Enabler of Authoritarian Governments ...»
December 14, 2011
Wire Design - Collage of city pedestrians and surveillance cameras.
Recording Everything: Digital Storage as an
Enabler of Authoritarian Governments
W ithin the next few years an important threshold will be crossed: For the
first time ever, it will become technologically and financially feasible for authoritarian governments to record nearly everything that is said or done within their borders – every phone conversation, electronic message, social media interaction, the movements of nearly every person and vehicle, and video from every street corner. Governments with a history of using all of the tools at their disposal to track and monitor their citizens will undoubtedly make full use of this capability once it becomes available.
The Arab Spring of 2011, which saw regimes toppled by protesters organized via Twitter and Facebook, was heralded in much of the world as signifying a new era in which information technology alters the balance of power in favor of the repressed. However, within the world’s many remaining authoritarian regimes it was undoubtedly viewed very differently. For those governments, the Arab Spring John Villasenor is a likely underscored the perils of failing to exercise sufficient control of digital nonresident senior fellow in Governance communications and highlighted the need to redouble their efforts to increase the Studies and in the monitoring of their citizenry.
Center for Technology Technology trends are making such monitoring easier to perform. While the Innovation at Brookings. He is also domestic surveillance programs of countries including Syria, Iran, China, Burma, professor of electrical and Libya under Gadhafi have been extensively reported, the evolving role of engineering at the digital storage in facilitating truly pervasive surveillance is less widely recognized.
University of California, Plummeting digital storage costs will soon make it possible for authoritarian Los Angeles.
regimes to not only monitor known dissidents, but to also store the complete set of digital data associated with everyone within their borders. The
Recording Everything: Digital Storage as an Enabler of Authoritarian Governments 1 Introduction When Moammar Gadhafi’s forces lost control of Tripoli in August 2011, the Libyan state surveillance apparatus was exposed for ordinary Libyans and the rest of the world to see. As described in an August 30, 2011 Wall Street Journal article, 1 companies found to have supplied communications interception and monitoring gear to Libya include French company Amesys, the Chinese telecommunications giant ZTE, and a small South African firm called VASTech. This equipment enabled Libya’s state security apparatus to capture and archive “30 to 40 million minutes” 2 of telephone conversations every month and to regularly read e-mails exchanged among activists.
The Gadhafi regime was unusual among dictatorships only in that its internal spying activities were so thoroughly unmasked, not that they were occurring.
There is ample evidence that other authoritarian regimes are embracing the extensive use of surveillance technology to track their own citizens as well. For example, the Syrian government under President Bashar al-Assad was reportedly working to install an interception system 3 built by Italian surveillance company Area SpA, which appears to have in turn acquired and incorporated equipment and software from California-based NetApp and Blue Coat Systems, German company Utimaco, and France-based Qosmos. 4 Evidence that Blue Coat surveillance products are being used by the Burmese government has also been uncovered. 5 The apparent presence of U.S.-built surveillance technology in countries such as Syria and Burma has led to Congressional calls for an investigation into NetApp and Blue Coat for possible American export control violations. Both companies have denied prior knowledge that their equipment was being sold inappropriately. 6 Even when all sellers and resellers of American-built surveillance equipment follow export control regulations, governments of export-restricted countries have other avenues for building a domestic spying capability. One option is to acquire equipment such as video cameras that can fall outside the scope of American export control laws, but that can nonetheless be used for surveillance. This approach is being used by the local government in the inland Chinese city of Chongqing, which is installing a massive network of video cameras using equipment supplied in part by Cisco. 7 While the “Peaceful Chongqing” project is portrayed as an anti-crime initiative, the data it collects will clearly be of value to authorities interested in monitoring and suppressing unapproved demonstrations as well.
There is also an enormous amount of non-American surveillance technology on the international market, some made in countries that have much laxer export control standards than the United States. The Iranian government, which is subject to extensive U.S. export control restrictions, has reportedly purchased electronic communications tracking technology from by Chinese telecommunications giant Huawei. 8 Countries can also perform surveillance using homegrown technologies.
Recording Everything: Digital Storage as an Enabler of Authoritarian Governments 2 In July, officials in Beijing began requiring businesses including hotels, restaurants, and cafes to install Wi-Fi monitoring software developed by a Shanghai company under a contract from the Chinese government. 9 As these examples illustrate, authoritarian governments can acquire the components of a surveillance infrastructure from any number of sources across the globe or at home. In practice, therefore, an important limiting factor in the ability of these governments to employ technology in the service of their domestic spying programs is the state of technology itself. Storage technology is particularly important in this respect because it supplies the literal memory of a vast domestic surveillance apparatus.
How Much Storage Does it Take to Record Everything?
There is nothing new or noteworthy about the observation that storage costs are declining exponentially. The thresholds that get crossed along the way, however, correlate to broad shifts in the power and impact of information technology. In 1984, the commercial availability of consumer-grade 10-megabyte hard disks made it possible, as observed by the New York Times, to store “the equivalent of around 2,500 typed pages,” removing the need “to plow through half a dozen file cabinets.” By the end of the next decade, devices that could hold the digital equivalent of millions of pages of text were considered unremarkable, but storage of digital music was still a challenge. Most MP3 players of that era, noted a 1999 New York Times article, could “store no more than 64 minutes of high-quality music.” 10 Today’s pocket-sized Apple iPod Classic can store “up to 40,000 songs, 200 hours of video, or 25,000 photos.” 11 When that much information can be held in the palm of a hand, the prospect that an authoritarian government could archive the entire life of a nation no longer seems impossible. Declining storage costs will make such monitoring not only possible, but likely.
Figure 1 shows the inflation-adjusted retail cost of hard disk storage on a dollars per gigabyte basis since 1980. 12 Over the past three decades, storage costs have declined by a factor of 10 approximately every 4 years, 13 reducing the pergigabyte cost from approximately $85,000 (in 2011 dollars) in mid-1984 14 to about five cents today. 15 In other words, storage costs have dropped by a factor of well over one million since 1984. Not surprisingly, that fundamentally changes the scale of what can be stored.
Recording Everything: Digital Storage as an Enabler of Authoritarian Governments 3 So what, exactly would it take to store everything? The answer depends in part on the nature of the information. Location data is far less voluminous than audio from phone calls, which in turn requires much less storage than video.
Location data, which is readily obtained from mobile phones, Wi-Fi connections, and GPS receivers, can already easily be archived. It takes fewer than 75 bits (ones and zeros) to pinpoint a person’s location anywhere on the earth to an accuracy of about 15 feet. 16 The information identifying the location of each of one million people to that accuracy at 5-minute intervals, 24 hours a day for a full year could easily be stored in 1,000 gigabytes, which would cost slightly over $50 at today’s prices. For 50 million people, the cost would be under $3000.
The audio for all of the telephone calls made by a single person over the course of one year could be stored using roughly 3.3 gigabytes. 17 On a per capita basis, the cost to store all phone calls will fall from about 17 cents per person per year today to under 2 cents in 2015. For a country like Syria, which has a population of 15 million people over the age of 14, 18 the current cost to purchase storage sufficient to hold one year’s worth of phone calls for the entire country would be about $2.5 million 19 – a high number but certainly not beyond governmental reach. If historical cost trends continue, the annual cost in 2011 dollars to purchase enough storage for Syria’s government to record all calls made in that country will fall to about $250,000 by 2016 and to about $25,000 by 2020. Iran has an over-age-14 population of 59 million, 20 so the corresponding cost to the Iranian government to record all calls in Iran would be about four times higher than in Syria. Cost will soon be no object for internal security services wishing to store everything said on a telephone in Syria, Iran, or even in a much more populous nation such as China.
Video surveillance can generate far more data than audio surveillance. The storage requirements for video depend on factors including the number of cameras used, the resolution of the images and the number of image frames per second Recording Everything: Digital Storage as an Enabler of Authoritarian Governments 4 captured. At a relatively slow five frames per second, a fairly high-resolution 21 traditional video surveillance camera might generate 22 about 1 megabit per second of data. By contrast, full-motion, true “high definition” 23 video at 30 frames per second can produce anywhere from about 2 to 5 or more megabits per second. 24 Cities around the world are increasingly deploying extensive camera systems to capture vehicle license plate numbers. As of early 2011, there were over 4000 such cameras in England and Wales providing continuous license plate data for traffic for cities including London, Birmingham, and Manchester. 25 Washington D.C. has a network of 73 license plate cameras. 26 New York uses a combination of over 100 cameras mounted at fixed roadside locations and an additional 130 cameras affixed to police cars. 27 Plate reading cameras are also being used in Canada, Australia, and India.
The deployments listed above are occurring in countries with robust ongoing debates regarding the balance of privacy and security. Privacy concerns have led to limitations on the length of time that plate data is retained – 72 hours, for example, in the case of Toronto 28 and three years for the system in Washington. 29 In authoritarian countries, however, there is essentially no open privacy debate, and governments have little incentive not to build permanent archives of license plate tracking information.
Over the course of a full year, a system of 1,000 roadside license plate reading cameras each producing 1 megabit per second would generate image data that could be held in storage costing about $200,000. The resulting database of license plate numbers (as opposed to the images used to obtain the numbers) could be stored for a small fraction of this cost.
Memory costs do not become a major obstacle to video surveillance unless the system is truly massive. Even then, the obstacle will only be temporary. The Chinese “Peaceful Chongqing” project will utilize up to 500,000 video cameras to blanket a city with a population of 12 million, 30 corresponding to one video camera for every 24 people. If each camera in that system were to produce an average of 3 megabits per second, 31 the corresponding annual cost to purchase storage to contain this data would be about $300 million dollars – an amount that would today be prohibitive. As a result, in the near term, operators of video surveillance deployments on the scale of the Peaceful Chongqing project will have to make choices. They can opt, for example, to store high-resolution data for only a limited amount of time. Or, they can permanently archive data from all of the cameras but at a lower image quality, resolution, or frame rate.
By the latter half of the decade, storage cost trends will make the need to make such choices obsolete. By 2020 the cost to store, in high resolution, all of the video acquired by the Chongqing network will drop to a much more practical $3 million per year. On a per capita basis this corresponds to about 25 cents per person per year, an amount that could easily be budgeted or even extracted from the population being monitored through a euphemistically worded “public safety tax.” In the longer term it is also possible that authoritarian regimes will choose to Recording Everything: Digital Storage as an Enabler of Authoritarian Governments 5 augment and eventually supplant their own storage facilities by renting storage from cloud-based storage providers. At large volumes, the current monthly cost to rent storage is roughly equivalent to the amount that would be needed to purchase it outright. 32 For countries building data archiving systems on a scale where the storage costs are a significant impediment, owning is still far more cost-effective than renting. However, when storage purchase costs decline to the point where they constitute only a small fraction of the overall costs of acquiring and maintaining an archive, rental will become a more attractive option.