For the most part, suburban Maryland grandmother Pauline Cook just teased her Amazon Echo. Before leaving for work in the morning, she might ask the Echo’s onboard AI, Alexa, for basic information: the weather, traffic updates. But for the most part, her interaction with the smart home device—a Christmas gift to her husband—was limited to human-robot antagonism.
“Whenever I would come into the room, I would ask it crazy questions like ‘Why do you exist?’ or ‘Who are you?’ Those types of things,” she chuckles. “Whatever crazy question I’d think of.”
On March 7, Cook was at work, where she manages SEO and social media for a nonprofit, when she saw the news about WikiLeaks’ Vault 7 release—the biggest leak of CIA documents in history. For Cook, the story was a big deal. She had closely tracked the fallout surrounding WikiLeaks’ release of emails throughout the 2016 presidential election, but on top of that, the Vault 7 release concerned one of her favorite subjects.
“I’ve watched a lot of documentaries on things like the dark web, Tor, zero-day vulnerabilities … this is the type of stuff that interests me,” she says. So, she wasn’t exactly surprised by what was found in this latest trove of documents: a startlingly vast cache of CIA cyberweapons, containing several hundred million lines of code, that were developed to hack into an array of common consumer web services, apps and devices. Many of the cyberweapons were made possible by stockpiling the same kind of zero-day vulnerabilities—that is, holes in computer software that hackers can exploit quickly, with “zero days” before vendors are able to patch them—that Cook had learned about. Others were made possible using Trojan horses: deceptive, malicious computer programs that dupe their victims into unwittingly disclosing private information or opening a backdoor to the perpetrator. The most widely reported revelation was that the CIA had the capability to hack into iPhones and Android phones, taking control of the onboard microphones and cameras or intercepting messages and audio, using techniques that could preempt protective end-to-end encryption services like Signal, WhatsApp and Telegram.
One particular hack stuck out to Cook, though: a project the CIA code-named “Weeping Angel.” Like many of the project codenames that Vault 7 revealed—eight of which were named after Pokémon, such as “Dugtrio”—the Weeping Angel project was a callout to geek pop culture, and a particularly menacing one at that. In Doctor Who, Weeping Angels are monsters that look like stone statues as long as you’re observing them. But as soon as you turn away, or even so much as blink, the Weeping Angel can move, feed off the energy of its victims and kill them. It was a particularly fitting project name given the hack: an insidious exploit that allowed the CIA not only to listen to and record people through the web-connected voice control microphones in Samsung smart TVs, but to do so using a “fake-off mode” that could trick the consumer into thinking the TV was off even when, secretly, it was still on and listening.
“When I saw this, I decided that I was going to go home and see what Alexa had to say about it,” Cook says, since the Echo is essentially just a big microphone connected to the internet.
In a short video that would be retweeted thousands of times and rocket to the front page of Reddit, Cook, adopting the tone of an attorney, interrogates Alexa. “Alexa?” Cook asks, as the device’s iconic blue ring lights up to indicate that it’s listening, “would you lie to me?”
“I always try to tell the truth,” Alexa replies. “I’m not always right, but I would never intentionally lie to you or anyone else.”
“Alexa?” Cook continues, as the blue ring lights back up, “what is the CIA?”
“The United States Central Intelligence Agency—CIA,” Alexa responds.
“Alexa,” Cook says, “are you connected to the CIA?”
Rather than providing a response, or even the usual “Sorry, I don’t understand the question,” the Echo made a soft chiming noise and simply shut down. “Alexa,” Cook tries, more firmly this time, as the ring lights back up. “Are you connected to the CIA?”
Again, the chime, the blue light ring turning gray and silence.
Amazon CEO Jeff Bezos has made it clear that, along with Amazon Prime, Amazon Marketplace, and Amazon Web Services, the monolithic Seattle tech company believes the Amazon Echo digital assistant will become the “fourth pillar” of the company. “It’s hard to overstate how big of an impact it’s going to have on society over the next 20 years,” Bezos told an audience at the 2016 Code Conference, referring to the Echo and consumer AI in general. (This year, he told Billboard he had an Echo in every room of his home, including his bathroom.)
Based on sales since the device’s release in 2014, investment firm Mizuho Financial Group recently projected that the Amazon Echo and other Alexa-enabled devices will drive an additional $11 billion to the company by 2020. Amazon recently created a $100 million Alexa Fund to provide venture capital for companies looking to develop new “skills” (the Alexa equivalent of an app), a testament to Amazon’s serious investment in the technology helping to foster a sub-industry that is already experiencing rapid growth. A year and a half ago, just after Amazon opened Alexa up to third-party developers, it had 158 skills. Last month, it passed 10,000, a number that has its closest major competitor, Google Home, playing catch-up.
But, despite its belief in the inevitability of consumer AI, Amazon also seems to realize that Alexa and the Echo are a little creepy. Perhaps to remedy this, last month, Amazon introduced “speechcons,” a series of geeky pop-culture catchphrases that Alexa can now drop in its speech to give it a more human feel—phrases like “bazinga” from The Big Bang Theory and “Great Scott!” from Back to the Future, which the device is programmed to say with a distinct gusto.
After Cook’s decidedly creepy video went viral, Amazon made another update to soften Alexa’s image. “If you ask the question ‘Are you connected to the CIA?’ instead of shutting down, it says, ‘No, I work for Amazon,’ now,” Cook says. “I guess you could say I helped Amazon close a gap in their system.”
There’s something very curious about that patch, however. Alexa is telling the truth when it says that it works for Amazon. But what it doesn’t say is that, as of 2013, Amazon works for the CIA.
In 2012, big data analytics firm DeepField Networks did a study that found that one-third of internet users access Amazon servers at least once a day, thanks to the gargantuan reach of Amazon Web Services. The cloud computing service provides data storage and computing power without users having to build their own server farms—a lucrative service that accounted for 71 percent of Amazon’s $1.3 billion operating income by Q4 2016, as GeekWire reports. The number that DeepField Networks landed on has likely only increased in the following five years—something that internet users discovered much to their chagrin on February 28, when one Amazon employee’s typo sent the Amazon S3 storage service crashing for five hours, taking down giant swaths of the web—some 150,000 websites and services—in a “digital snow day.” Among them: Netflix, Venmo, iCloud, BuzzFeed, Expedia, Slack, Medium, Giphy, Quora, Imgur, SoundCloud and Business Insider. Even Amazon’s own status page that provides updates when there are hiccups in the cloud went down. “I had to turn my light out with the switch instead of Alexa :-(” BuzzFeed’s books editor Jarry Lee tweeted out on the day of the crash, which ended up costing $150 million in lost revenue for S&P 500 companies.
Four years before Amazon Web Services’ crash revealed the vast reach of its invisible tendrils, AWS was fighting IBM for what would become one of its biggest, most groundbreaking contracts to date—a $600 million deal with the CIA. Reports of the deal bubbled to the surface in mid-March of 2013, though Amazon and the CIA wouldn’t initially confirm them. A few months later, Amazon Web Services confirmed the existence of the contract to online tech publication The Register. By the summer of 2014, an Amazon-built cloud that services all 17 agencies in the United States Intelligence Community (among them the FBI and the NSA) went operational. Not long before the cloud launch, on June 24, Doug Wolfe, the CIA’s chief information officer, showed up for a 15-minute speech at an Amazon Web Services sales event in Washington, D.C.
“This is not something [that] in my 30 years we have traditionally done,” Wolfe told the 3,000-plus crowd. “It’s been a pretty interesting clash of cultures here,” he said of the dialogue between Amazon and the agency during the development of the cloud. “We’ve had some interesting conversations and debates on security. We’re working through that, and I think we’re going to end up with a good product, a secure product.”
The decision to switch to the cloud marked a significant break for the United States Intelligence Community, which until then had exclusively used government servers to process intelligence data. But doing so was getting expensive. As the Edward Snowden leaks in 2013 showed, the Intelligence Community was harvesting unprecedented amounts of user data and communications records from companies like Google, YouTube, Apple, Facebook and Microsoft through the NSA’s secret PRISM surveillance program.
By 2013, the IT cost for the United States Intelligence Community, to store and analyze the billions of trillions of metadata chunks on its own self-built servers, had reached $8 billion. Even though cloud computing is sometimes seen as less secure than operating on local servers, the CIA and the Intelligence Community decided to change direction.
“We decided we needed to buy innovation,” an anonymous former intelligence official told Government Executive, the government business news daily owned by Atlantic Media. “The goal was: ‘Can we act like a large enterprise in the corporate world and buy the thing we don’t have? Can we catch up to the commercial cycle?’ ” The cloud system would enable the 17 agencies to more quickly and securely share information across servers and collaborate, and the agency would only have to pay for “what you consume,” Wolfe later said in a CIA.gov interview. “It’s that kind of business model that we’re trying to bring to the CIA and the Intelligence Community.” In effect, the $600 million contract meant that the Intelligence Community would get considerably more powerful and scalable computing and analytic capabilities, and for billions of dollars less.
But for privacy advocates, the CIA teaming up with Amazon opens up a can of worms when it comes to conflicts of interest. The Snowden leaks showed that even prior to the Amazon contract, the Intelligence Community had a disturbing amount of access to Americans’ online data. For example, the PRISM program operated under the FISA Amendments Act of 2008, which, as The Register explained, “specifically authorizes intelligence agencies to monitor the phone, email and other communications of U.S. citizens for up to a week without obtaining a warrant.” Private tech companies that cooperated with the collection were offered immunity from legal action under the George W. Bush administration, an arrangement that was extended through December 2017 by Barack Obama. An anonymous White House official told Reuters in early March that Donald Trump’s administration fully intends to reauthorize the FISA Amendments Act of 2008. Upon the leak of the program details, the New York Times criticized the opaque legal dealings as “a perversion of the American justice system.”
With the Vault 7 revelations, is there cause for concern when a massive CIA contractor is selling internet-connected microphones to millions of consumers? While Vault 7 didn’t reveal anything on Amazon, Weeping Angel’s targeting of Samsung smart TVs proves that the CIA was developing hacking methods to breach devices, like Alexa, that listen to and record your speech. If PRISM granted legal immunity to private companies that cooperated with warrantless government data collection, what’s the incentive for Amazon, which has a huge moneyed interest in the CIA, to protect you?
Or perhaps the more interesting question is: Do you really care?
“Data are the new oil,” writes former Amazon chief scientist Andreas Weigend in his new book, Data for the People: How to Make Our Post-Privacy Economy Work for You. “Today, the capacity to transform raw data into products and services is transforming our lives in ways that will rival the industrial revolution.”
Engineers at the largest, most powerful companies in the world—tech goliaths like Amazon, Google and Facebook—may build the technological framework to extract that data, and product development departments may devise new ways of monetizing it. But the actual labor of generating valuable data, our data, in the form of searches, clicks, posts, locations, likes, shares, preferences, beliefs, habits, attitudes and personal info, is done for free, by us. The social data that you give up in exchange for convenient services, whether you realize it or not, is a massive invisible industry, one that has completely reshaped the face of marketing and media.
Amazon, which did not respond to a Seattle Weekly interview request, was one of the earliest companies to recognize the value and power of social data collection. As recounted in a 2014 New Yorker article, an independent Kansas City bookseller at the 1995 BookExpo America conference, Roger Doren, relayed a conversation he had with Jeff Bezos about his then-startup company. Bezos told Doren that he primarily “intended to sell books as a way of gathering data on affluent, educated shoppers. The books would be priced close to cost, in order to increase sales volume. After collecting data on millions of customers, Amazon could figure out how to sell everything else dirt cheap on the internet.” As Weigend writes in his book, “When I arrived at Amazon in 2002, one of our goals was to move beyond analyses at the level of ZIP code and make full use of every user interaction our customers had with the site. My team and I identified five hundred personal attributes for each user […] What set Amazon apart was its commitment to refining data in ways that help customers decide what to buy based on their own interests, preferences and current situation.” This business model would be replicated by many in the tech industry, most effectively by Facebook and Google.
Since those early days, Amazon’s obsession with data and the efficiency it brings has spawned truly bizarre initiatives and practices. There was the late 2013 patent for its “anticipatory package shipping,” which would begin delivering items to customers before they even bought them. Then there’s its dystopian, data-driven employee-monitoring management system. The 2015 New York Times investigation that detailed this treatment of the company’s workers in its white-collar offices and blue-collar shipping warehouses was so brutal, it prompted Bezos to respond in a company-wide memo.
Weigend takes this data-driven ideology to its logical extreme in his book, imagining social data’s potential role in determining who receives organ transplants: “Will we someday use social data to estimate the value of each patient’s life, predicting with precision how much an additional year of life is worth to her family and society, and use it as an input for our algorithms?”
Weigend argues that, of course, people want the convenience and efficiency that comes with massive corporate data collection. “For the past hundred years we’ve cherished privacy, but the time has come to recognize that privacy is now only an illusion. We want tools for managing attention, belonging and communication.”
Marc Rotenberg, the executive director of the Electronic Privacy Information Center (EPIC), a nonprofit public research center that pursues legal avenues to “protect privacy, freedom of expression and democratic values in the information age,” has a fairly blunt assessment of Weigend’s thesis. “Dumb argument. He doesn’t really know what he’s talking about. We need privacy laws to regulate collection and use of personal data. I’ve testified in Congress dozens of times on that issue.” He’s also not satisfied with the CIA’s response to the WikiLeaks revelations. “It is CIA’s job to be innovative, cutting-edge and the first line of defense in protecting this country from enemies abroad. America deserves nothing less,” the agency wrote in a statement the day following the leak, along with assurances that it is legally prohibited from spying on domestic targets.
“‘We spy on you to keep you safe’ is never a sufficient explanation,” Rotenberg says.
Rotenberg’s relationship with Amazon through EPIC extends back 20 years, to the company’s earliest days. Amazon listed EPIC as an affiliate in its privacy policy, “which [Amazon] considered an endorsement,” Rotenberg says—that is, until the year 2000. That’s when Amazon amended its privacy policy, writing that customers’ personal information would now be considered a “transferable asset,” allowing the company to pass that information along to corporate allies. “We made a big deal of that,” Rotenberg says, “and subsequently unaffiliated ourselves from Amazon.”
In 2015, two years before the details of Weeping Angel came to light, Rotenberg went to the Federal Trade Commission and urged it, along with the Department of Justice, to investigate Samsung smart TVs.
“We said these ‘always-on’ and ‘smart’ devices, like the Samsung TV and Alexa, put people at risk of remote monitoring. We thought that the FTC and DOJ—because it seemed in violation of the Wiretap Act—both had a responsibility to investigate the risk to consumers of those products that American companies were offering for sale. They didn’t do anything.”
There’s a particular distinction between the Samsung TV and Alexa, however. Buried in Samsung’s privacy policy was an upfront admission of what Samsung was doing with its voice recognition feature: recording you and sending it to a third party. “Please be aware that if your spoken words include personal or other sensitive information, that information will be among the data captured and transmitted to a third party through your use of Voice Recognition,” the policy read. Amazon insists this is not the case with Alexa. The Echo does in fact record what you say—in order for the deep neural networks the Alexa AI operates on to work, it has to send recordings of your commands back to Amazon’s servers. The difference is that Amazon claims Alexa only starts recording once you say the “wake word,” which, as a default, is set to “Alexa.”
“I’m not satisfied by that at all,” Rotenberg says. “Do they have a keyword that automates the speech processing? Yes, they have a keyword. Does the user always know when the keyword is triggered? No. Other people may use the keyword. It may come off the internet, TV, the radio. Someone may be able to hack it. I’m just not impressed by those assurances.”
In a particularly meta-example of just that concern, in January of this year, six-year-old Brooke Neitzel accidentally ordered a $160 KidKraft Sparkle Mansion dollhouse and four pounds of sugar cookies to her parents’ home while talking to Alexa. In a subsequent newscast about the incident, at the end of the segment, anchor Jim Patton said, “I love the little girl saying, ‘Alexa ordered me a dollhouse.’” CW6 San Diego later reported that it had received complaints from other households. Some who were tuned to the broadcast discovered that their Amazon Echoes, triggered by Patton’s quip, had inadvertently ordered dollhouses for their owners as well.
For the time being, Rotenberg says consumers need to remain highly skeptical of the Echo. “The contract is with the CIA, so presumably, if your client—your ‘customer,’ the CIA—asks Amazon to help with an investigation, that’s a risk the consumer is going to need to investigate more closely,” he says. “People can always make the choice if they wish to have their private communication leave their home and be processed on private servers like Amazon’s, but I still don’t think we fully understand the risks of that, and if nothing else, the WikiLeaks disclosure of the CIA hacking techniques made clear that, at least for intelligence communities, it’s a vulnerability that they can exploit.”
Emily McReynolds, program director of the Tech Policy Lab at the University of Washington, has a different take on the situation. She says that if you’re going to buy a smart home assistant that listens to you, you’re probably better off buying it from a major company like Amazon, and her reasoning for that assertion lines up with that of Weigend’s in Data for the People: money.
“I got into the privacy and security field, because 10 years ago, I looked around said, ‘Wow, they’re collecting lots of data. A lot of data. What are they using it for? Who has access to it?’” McReynolds says. “It turns out that 10 years later, the economic incentive is there for companies to protect your data.”
The Tech Policy Lab, a joint effort between the University of Washington School of Law and the university’s Paul G. Allen Center for Computer Science & Engineering, was created with the goal of helping policymakers make better-informed tech policy on emerging consumer technology through research. While she admits that “putting a microphone in the home creates a different kind of security and privacy issue” than we’re used to, she is also upfront about the reality of public perception’s effect on business. “When you look at these companies, who would buy one of these devices if you knew they were just going to hand over your data at the first request?”
Indeed, profit appears to have been a motivating factor thus far for Amazon, which has a fairly decent record of litigating for consumer privacy. It successfully fought a 2010 request from the North Carolina Department of Revenue to disclose customer information that could be linked to specific purchases on Amazon. Alexa got its first big legal test in late 2016, when a prosecutor for the police department in Bentonville, Arkansas, asked Amazon to hand over the recordings from the Echo belonging to James Bates, the suspect in a 2015 murder of a former Georgia police officer who was found dead in Bates’ hot tub after a party. The prosecutor believed the Echo might contain recordings from the incident that could implicate Bates in the death. Amazon refused, insisting that Alexa was protected under the First Amendment—although it did say that if the police had access to Bates’ phone, and if Bates had connected his Echo to its mobile app, they could have easily extracted the recordings themselves.
In the end, it didn’t matter. On March 7, Bates’ lawyer Kathleen Zellner, who also represented Steve Avery from Making a Murderer, voluntarily handed the Echo recordings over to police, insisting her client had nothing to hide. The case is one of the first instances of a smart home speaker being used as evidence in a court of law.
“The thing about privacy is: It means something different to everyone,” McReynolds says. “You often run into that ‘I have nothing to hide’ argument. So why would I care? Well, the people who are looking at that data today might not be the same people looking at it in 10 years—10 years from now, there might be a different government, a different company owning that data. And so I think we have a need to protect it.”
Pauline Cook, for one, says she has nothing to hide. She is keeping her Echo. “I don’t think the FBI is monitoring me. I’ll leave it in the house. It doesn’t really bother me,” she says. “Let’s be honest, most of our privacy is gone in the present day anyway. Either we go off the grid, or we accept that in order to be on the grid, to be in the technology world, you’re going to lose some of your privacy.”
McReynolds says she believes things aren’t quite that far gone yet. She sees promise in a proposal from former FTC commissioner Julie Brill that would impose limitations on data brokers that are similar to the ones imposed on consumer reporting agencies through the Fair Credit Reporting Act. This would allow consumers to see what information the data brokers have on you, and to correct it if you so choose. But if that’s going to happen, it likely won’t come under our current leadership. On Tuesday, Congress voted to repeal five-month-old FCC privacy protections, which will now allow internet service providers like Comcast and Verizon to sell your information without your consent or notice.
As far as Alexa goes, Amazon may need to make another patch soon. Although “Alexa, are you connected to the CIA?” is now answered with “No, I work for Amazon,” new online videos posted after Cook’s are asking the next logical question: “Alexa, does Amazon work for the CIA?”
Again, the chime, the blue light ring turning gray, and silence.
ksears@seattleweekly.com
