Posted on

The real threat of fake voices in a time of crisis

As federal agencies take increasingly stringent actions to try to limit the spread of the novel coronavirus pandemic within the U.S., how can individual Americans and U.S. companies affected by these rules weigh in with their opinions and experiences? Because many of the new rules, such as travel restrictions and increased surveillance, require expansions of federal power beyond normal circumstances, our laws require the federal government to post these rules publicly and allow the public to contribute their comments to the proposed rules online. But are federal public comment websites — a vital institution for American democracy — secure in this time of crisis? Or are they vulnerable to bot attack?

In December 2019, we published a new study to see firsthand just how vulnerable the public comment process is to an automated attack. Using publicly available artificial intelligence (AI) methods, we successfully generated 1,001 comments of deepfake text, computer-generated text that closely mimics human speech, and submitted them to the Centers for Medicare & Medicaid Services’ (CMS) website for a proposed federal rule that would institute mandatory work reporting requirements for citizens on Medicaid in Idaho.

The comments we produced using deepfake text constituted over 55% of the 1,810 total comments submitted during the federal public comment period. In a follow-up study, we asked people to identify whether comments were from a bot or a human. Respondents were only correct half of the time — the same probability as random guessing.

Image Credits: Zang/Weiss/Sweeney

The example above is deepfake text generated by the bot that all survey respondents thought was from a human.

We ultimately informed CMS of our deepfake comments and withdrew them from the public record. But a malicious attacker would likely not do the same.

Previous large-scale fake comment attacks on federal websites have occurred, such as the 2017 attack on the FCC website regarding the proposed rule to end net neutrality regulations.

During the net neutrality comment period, firms hired by industry group Broadband for America used bots to create comments expressing support for the repeal of net neutrality. They then submitted millions of comments, sometimes even using the stolen identities of deceased voters and the names of fictional characters, to distort the appearance of public opinion.

A retroactive text analysis of the comments found that 96-97% of the more than 22 million comments on the FCC’s proposal to repeal net neutrality were likely coordinated bot campaigns. These campaigns used relatively unsophisticated and conspicuous search-and-replace methods — easily detectable even on this mass scale. But even after investigations revealed the comments were fraudulent and made using simple search-and-replace-like computer techniques, the FCC still accepted them as part of the public comment process.

Even these relatively unsophisticated campaigns were able to affect a federal policy outcome. However, our demonstration of the threat from bots submitting deepfake text shows that future attacks can be far more sophisticated and much harder to detect.

The laws and politics of public comments

Let’s be clear: The ability to communicate our needs and have them considered is the cornerstone of the democratic model. As enshrined in the Constitution and defended fiercely by civil liberties organizations, each American is guaranteed a role in participating in government through voting, through self-expression and through dissent.

Image Credits: Zang/Weiss/Sweeney

When it comes to new rules from federal agencies that can have sweeping impacts across America, public comment periods are the legally required method to allow members of the public, advocacy groups and corporations that would be most affected by proposed rules to express their concerns to the agency and require the agency to consider these comments before they decide on the final version of the rule. This requirement for public comments has been in place since the passage of the Administrative Procedure Act of 1946. In 2002, the e-Government Act required the federal government to create an online tool to receive public comments. Over the years, there have been multiple court rulings requiring the federal agency to demonstrate that they actually examined the submitted comments and publish any analysis of relevant materials and justification of decisions made in light of public comments [see Citizens to Preserve Overton Park, Inc. v. Volpe, 401 U. S. 402, 416 (1971); Home Box Office, supra, 567 F.2d at 36 (1977), Thompson v. Clark, 741 F. 2d 401, 408 (CADC 1984)].

In fact, we only had a public comment website from CMS to test for vulnerability to deepfake text submissions in our study, because in June 2019, the U.S. Supreme Court ruled in a 7-1 decision that CMS could not skip the public comment requirements of the Administrative Procedure Act in reviewing proposals from state governments to add work reporting requirements to Medicaid eligibility rules within their state.

The impact of public comments on the final rule by a federal agency can be substantial based on political science research. For example, in 2018, Harvard University researchers found that banks that commented on Dodd-Frank-related rules by the Federal Reserve obtained $7 billion in excess returns compared to non-participants. When they examined the submitted comments to the “Volcker Rule” and the debit card interchange rule, they found significant influence from submitted comments by different banks during the “sausage-making process” from the initial proposed rule to the final rule.

Beyond commenting directly using their official corporate names, we’ve also seen how an industry group, Broadband for America, in 2017 would submit millions of fake comments in support of the FCC’s rule to end net neutrality in order to create the false perception of broad political support for the FCC’s rule amongst the American public.

Technology solutions to deepfake text on public comments

While our study highlights the threat of deepfake text to disrupt public comment websites, this doesn’t mean we should end this long-standing institution of American democracy, but rather we need to identify how technology can be used for innovative solutions that accepts public comments from real humans while rejecting deepfake text from bots.

There are two stages in the public comment process — (1) comment submission and (2) comment acceptance — where technology can be used as potential solutions.

In the first stage of comment submission, technology can be used to prevent bots from submitting deepfake comments in the first place; thus raising the cost for an attacker to need to recruit large numbers of humans instead. One technological solution that many are already familiar with are the CAPTCHA boxes that we see at the bottom of internet forms that ask us to identify a word — either visually or audibly — before being able to click submit. CAPTCHAs provide an extra step that makes the submission process increasingly difficult for a bot. While these tools can be improved for accessibility for disabled individuals, they would be a step in the right direction.

However, CAPTCHAs would not prevent an attacker willing to pay for low-cost labor abroad to solve any CAPTCHA tests in order to submit deepfake comments. One way to get around that may be to require strict identification to be provided along with every submission, but that would remove the possibility for anonymous comments that are currently accepted by agencies such as CMS and the Food and Drug Administration (FDA). Anonymous comments serve as a method of privacy protection for individuals who may be significantly affected by a proposed rule on a sensitive topic such as healthcare without needing to disclose their identity. Thus, the technological challenge would be to build a system that can separate the user authentication step from the comment submission step so only authenticated individuals can submit a comment anonymously.

Finally, in the second stage of comment acceptance, better technology can be used to distinguish between deepfake text and human submissions. While our study found that our sample of over 100 people surveyed were not able to identify the deepfake text examples, more sophisticated spam detection algorithms in the future may be more successful. As machine learning methods advance over time, we may see an arms race between deepfake text generation and deepfake text identification algorithms.

The challenge today

While future technologies may offer more comprehensive solutions, the threat of deepfake text to our American democracy is real and present today. Thus, we recommend that all federal public comment websites adopt state-of-the-art CAPTCHAs as an interim measure of security, a position that is also supported by the 2019 U.S. Senate Subcommittee on Investigations’ Report on Abuses of the Federal Notice-and-Comment Rulemaking Process.

In order to develop more robust future technological solutions, we will need to build a collaborative effort between the government, researchers and our innovators in the private sector. That’s why we at Harvard University have joined the Public Interest Technology University Network along with 20 other education institutions, New America, the Ford Foundation and the Hewlett Foundation. Collectively, we are dedicated to helping inspire a new generation of civic-minded technologists and policy leaders. Through curriculum, research and experiential learning programs, we hope to build the field of public interest technology and a future where technology is made and regulated with the public in mind from the beginning.

While COVID-19 has disrupted many parts of American society, it hasn’t stopped federal agencies under the Trump administration from continuing to propose new deregulatory rules that can have long-lasting legacies that will be felt long after the current pandemic has ended. For example, on March 18, 2020, the Environmental Protection Agency (EPA) proposed new rules about limiting which research studies can be used to support EPA regulations, which have received over 610,000 comments as of April 6, 2020. On April 2, 2020, the Department of Education proposed new rules for permanently relaxing regulations for online education and distance learning. On February 19, 2020, the FCC re-opened public comments on its net neutrality rules, which in 2017 saw 22 million comments submitted by bots, after a federal court ruled that the FCC ignored how ending net neutrality would affect public safety and cellphone access programs for low-income Americans.

Federal public comment websites offer the only way for the American public and organizations to express their concerns to the federal agency before the final rules are determined. We must adopt better technological defenses to ensure that deepfake text doesn’t further threaten American democracy during a time of crisis.

Read More

Posted on

Maybe Information Actually Doesn’t Want to Be Free

SAN FRANCISCO — Jessica Lessin thinks the biggest story of the moment — how tech is swallowing the universe — is hopelessly under-covered by the news media. The issue is “massive,” she said not long ago in her spare, cube-like office here, and “no one is paying attention.”

Of course, it can be hard to see the forest for the tweets. From analysis of Trump’s utterances to conspiracy-peddling publishers amplifying themselves on Facebook and YouTube, tech stories increase exponentially every day. But Ms. Lessin, founder of The Information, an influential Silicon Valley publication, thinks most reporters are still focusing on the wrong topics: glamorous cryptocurrency, for example, rather than the blockchain looming over bank loans and stock trades; or the number of cars sold, rather than the artificial intelligence and driver networks that threaten to make that number obsolete.

She has focused her site on the larger picture, pursuing industry scoops and keeping the publication ad-free, instead charging $399 a year for complete access. The Information achieved profitability in 2016, Ms. Lessin said, three years after she left The Wall Street Journal to start it. She added that she expected $20 million in sales by the end of 2020, and for her staff of two dozen reporters and editors in the Bay Area, Seattle, Los Angeles, New York, Washington and Hong Kong to grow. “The fact that we have a business that’s scaling makes me excited,” she said.

This sense of hope is discordant with the rest of online media, which seems in grim shape — last year, more than 1,000 people were laid off at BuzzFeed, AOL, Yahoo, HuffPost and Vice Media. (BuzzFeed is now back on more solid footing and could be headed for a sale.)

As other online organs have bloated and intermittently fasted, The Information’s reporters have become known in Silicon Valley for sniffing out the industry’s misdeeds and tweaking its powerful. A 2017 story revealed sexual harassment allegations against a venture capitalist that led to the shutdown of his firm. A recent article revealing hidden financial data at Quibi, a new streaming service, prompted its chief executive, Meg Whitman, to compare reporters to sexual predators. (She later apologized.)

The Information is sparely, almost clinically designed and frequently refreshed. Subscribers include Amazon’s founder, Jeff Bezos, and the media investor James Murdoch (“Please write nice things about her,” he said of Ms. Lessin), corporate clients like Google and Goldman Sachs, and most of start-up royalty. Laurene Powell Jobs, the world’s seventh wealthiest woman and an influential philanthropist who also owns The Atlantic, finds the site useful. It covers “an ecosystem and an industry I care about,” she said, adding, “I’ve followed Jessica’s byline since The Journal.”

Ms. Lessin, 36, is the rare editor to have risen from ink-stained wretch to a player, much like Peter Bart when he ruled Variety, or Anna Wintour of Vogue. But her success, unlike the editors’ of an earlier time, owes as much to the data-driven discipline of her business as her editorial tastes. In an era when many pay walls, if they exist at all, are easily scaled, Ms. Lessin is fiercely guarding the fortress.

“I’ve said this from the beginning,” she said, “and I continue to say this, but you can’t give away what you expect the reader to find valuable.”

Ms. Lessin’s instinct for tradecraft showed up before the internet was ubiquitous, when she was editor of The Greenwich Academy Press, the half-size broadsheet of her private high school, and wanted to publish it in full color. To raise the money, she persuaded the school to allow her to auction off parking spots. “I just really wanted it to look as big and professional as possible,” she recalled.

While attending Harvard, she scored the coveted faculty beat at the Crimson newspaper. “It was like covering Congress,” Ms. Lessin said. “It’s fun because you get the bickering and the politics.” Lauren Schuker Blum, a friend who worked with her there and later at The Journal, remembered Ms. Lessin’s work habits. “We all had these reporter notebooks and most of us would use like half of it, or lose it, but she had like 30 of them, impeccably detailed,” Ms. Schuker Blum said. “She was like a libel lawyer’s dream.”

After graduating in 2005, Ms. Lessin completed an internship at The Journal, then kept coming back into the office to pitch stories. Eventually, she landed a full-time job covering personal tech, one of the least popular beats at the time. The year was 2005. BlackBerrys were the gold standard of smartphones and Facebook was just an online phone book for college students.

In 2008, Ms. Lessin moved to San Francisco to cover the tech industry — and regularly breaking stories. “I was like, ‘Who the hell is this girl?’” said Paul Steiger, the Journal’s managing editor at the time. “I kind of followed her work and asked people, ‘Is she as good as this looks?’ And they said yes.”

But it was also around this time that some people began to whisper about Ms. Lessin’s possible conflicts of interest. Through Harvard, she had become friends with start-up founders or fast-rising executives at places like Google and Facebook, ostensibly her key subjects. She was also dating another graduate, Sam Lessin, who had started a company that would later be acquired by Facebook. (The two married in 2012.)

A holiday excursion in 2008 resulted in a scolding for Ms. Lessin. As the economy was plummeting, she and Mr. Lessin jetted off to the vacation home of his family on the island of Cyprus with friends of theirs from the start-up scene.

The group passed the time as many people do on vacation, drinking and lounging around the pool. And before filming such activities and sharing them with strangers would become commonplace on Instagram, they posted footage online, including the women wearing matching black-and-white checkered swimsuits, lip-syncing to Journey’s “Don’t Stop Believing.”

The Cyprus travelers were blasted for their stunning lack of self awareness as the nation’s economy teetered toward crisis and tech companies were laying off employees. Ms. Lessin was singled out by Valleywag, the now-defunct tech site, in a post headlined, “WSJ reporter parties in Cyprus with people she covers.”

“Oh, that never made sense to me,” she said. “These were not people I wrote about. These were friends.” (A scan of Journal articles from the period shows she interviewed at least one Cyprus attendee in an article — Mike Hudack, the head of, a video start-up that has since shut down. Ms. Lessin says they were not friends when she wrote the article.) Still, her vacation drew disapproving scrutiny from higher-ups at The Journal, though not an official reprimand.

Ms. Lessin, in turn, was beginning to chafe at how newsrooms were covering tech — from a cool remove, she thought, never going deep. In contrast were the many bloggers who could delve into the industry’s every incremental move, but who had become so close to subjects the stories read like ad copy. Ms. Lessin said she thought: Couldn’t you do both? In-the-know reporting that still held subjects to account?

“I knew if I didn’t do it, someone else would, and I’d be kicking myself,” she said.

Valley underminers like to snipe that Ms. Lessin never had to persuade investors to back her plan. She had her own money. Her father is Jerome C. Vascellaro, a partner at the private equity giant TPG, and a significant investor in tech and media businesses like Uber, Vice and Airbnb. Her husband, a son of the late tech investor Robert H. Lessin, made a fortune from the Facebook stock he received as part of the company’s acquisition of his start-up years ago.

Ms. Lessin said she tapped her own bank account, using “less than $1 million,” to start The Information, and continues to own and control it wholly. She pays competitive salaries (albeit without equity) — as much as $180,000 or more for some top reporters. She refuses to spend more than she grosses, she said.

So far, this strategy seems to be paying off. A 2016 article on Tony Fadell, then the head of Google’s Nest division, exposed how the executive’s last-minute decrees and slow decision making had crippled the company’s hardware efforts. The story was so in demand it converted over 600 new subscribers in the first day, recalled the reporter who wrote it, Reed Albergotti, who worked at The Information from 2015 to 2019. “It blew up,” he said. “That was proof of the model.”

But is The Information — whose title anticipates an interest in nothing short of everything — just a trade publication, like Advertising Age or Publishers Weekly? (One heavily trafficked section features richly detailed organizational charts that executive recruiters mine for leads.)

Ms. Lessin, seeming a little annoyed by the question, tilting her head and widening her eyes as she computed her reply. “I think that misses the point,” she finally said. “There’s so much hunger for what we produce.”

In December, she introduced a consumer-friendly version of the site, an app called The Tech Top 10, priced at $30 a year. Instead of a dense story on Netflix’s debt structure, the app might publish a short explainer on Netflix’s price increase. “You’re matching the reader with the level of expertise they want,” Ms. Lessin said. “That’s what subscriptions allow you to do.”

She won’t say how many subscribers The Information has, but some back-of-the-envelope math suggests she’ll have to hit 40,000 paying readers by this year to reach her sales objective, which could be a significant challenge. According to three people familiar with the business, the publication surpassed 20,000 subscribers only around the middle of last year. “I can confirm we have more than that,” she said, declining to be more specific.

Her publication’s success has attracted suitors. Some time last year, John Ridding, the chief executive of The Financial Times, Britain’s pre-eminent business publication, met with Ms. Lessin in San Francisco. The salmon-colored broadsheet was interested in a possible takeover, three people familiar with the matter said. Mr. Ridding declined to comment, and Ms. Lessin said The Information was not for sale.

As at any start-up, the vibe at The Information’s open-plan offices is like a college dorm room that’s in the middle of being cleaned up ahead of Parents’ Weekend. A large part of the staff hails from The Journal, including Martin Peers, who used to be Ms. Lessin’s editor. Now, she’s his boss.

Mr. Peers, 59, is famous within journalism circles for his cantankerous nature and deep skepticism of Silicon Valley — and yet he came west. “I had been at the Journal for 15 years,” he said. “I was exhausted and what Jessica was proposing was the perfect antidote, and I thought, ‘Why not?’”

In June 2017, the site landed one of its biggest scoops: a feature that revealed sexual harassment allegations against one of Silicon Valley’s most well-connected venture capitalists. Six women had accused Justin Caldbeck, a partner at Binary Capital, of unwanted sexual advances, with three of them speaking to the reporter, Mr. Albergotti, on the record.

The story exposed a pervasive culture of misogyny and harassment within tech, immediately raised The Information’s profile and was a precursor of the broader #MeToo movement. But Mr. Albergotti, who now works at The Washington Post, remembered the staff’s anxiety as they got closer to publishing. They were keenly aware of what had happened to Gawker, which was sued for invasion of privacy by Hulk Hogan. The suit, which was financed by the venture capitalist Peter Thiel, drove Gawker into extinction and stoked a fear among publishers that anyone with enough money and willpower could vaporize a news outlet.

As the Caldbeck story was about to go to press, Ms. Lessin was in Italy attending a conference. She consulted the company’s liability insurance, which she had printed out, in her hotel room before heading to a dinner where she would be seated with Jeff Bezos. “I don’t remember if I vomited or not,” she said. “But I was very nervous.” She gave the green light.

Mr. Caldbeck didn’t sue. Instead, he resigned. A short while later, his venture firm collapsed. As a female entrepreneur, Ms. Lessin felt The Information’s work was “deeply personal,” especially as several men in the industry, who had heard the piece was in the works, contacted her to suggest the claims were overblown. These were “men I respect, who I was close to,” she said.

She wouldn’t name them. Ms. Schuker Blum, who worked with her at The Journal, said Ms. Lessin is not a gossip, like many reporters. “She’s not the journalist who’s always complaining,” Ms. Schuker Blum said. “She’s not a conspiracy theorist. She sees the best in people.”

Daniel Ek, the chief executive of Spotify, said he found the occasional, critical story on his company “not unfair.” But he added that Ms. Lessin “has to walk a tightrope given the level of access that she has. That’s got to be tough.”

Ms. Lessin’s connections continue to raise eyebrows, particularly those to Facebook. She and her husband are friends with their Harvard classmates Mark Zuckerberg, the company’s chief executive, and his wife, Priscilla Chan, who runs the couple’s philanthropy efforts. They attended each other’s weddings and both have young children. (Ms. Lessin’s two boys, Lion and Maverick, are both under the age of 3.) Mr. Zuckerberg was at The Information’s launch party, where she joked that for the super-high subscription rate of $10,000 a story could be killed (but just one). Recently, Ms. Chan was a speaker at an Information event.

The Information has published tough stories on Facebook, including a 2016 piece that revealed a weakness in its business. A more recent article exposed tensions between Chinese employees and Facebook’s leaders. But so far, it has only taken smaller swipes at the tech giant.

So how does The Information write about a company run by a friend of the site’s owner, one that is also perceived as having failed democracy, if not the universe?

Ms. Lessin was circumspect, her contralto voice echoing slightly off the glass walls of her office. “I’m very careful to draw lines around my personal life,” she said. “We have very clearly defined our culture around getting the best, most accurate story possible.”


NYT > Business