• Lab
  • AndroidForMobile Foundation at
    HOME
              
    LATEST STORY
    Bad news from Mashable, BuzzFeed, and Vice shows times are rough for ad-supported digital media
    ABOUT                    SUBSCRIBE
    Aug. 31, 2017, 10:27 a.m.
    Reporting & Production

    With its first Facebook Messenger bot, ProPublica is collecting reader stories about hate speech

    Can a Facebook Messenger bot used to collect stories from readers also double as a new form of storytelling? ProPublica thinks so.

    The big tech companies have long clung to the idea that their platforms are driven solely by the bias-free, objective realities of their algorithms. But of course the truth is more complicated: Behind those algorithms are people, and, when it comes to policing hate speech, the only thing consistent, it seems, is inconsistency.

    Back in June, ProPublica published an in-depth investigation into . The findings, which were were clear, if a little disheartening: Facebook’s rules were being inconsistently applied, creating issues both for people abused without recourse and people whose posts had been taken down with little explanation or room for appeal.

    ProPublica, eager to learn more about these secret algorithms, is using its own technology to help. Earlier this week, the , which is designed to collect stories about people’s experiences with hate speech on Facebook. Using Messenger’s conversational interface, the bot asks users questions such as “Did you experience hate speech” and “Did Facebook delete the post?” Users are also able to share the exact wording of the comments in question and direct screenshots.

    , senior reporter at ProPublica, explained that the bot is designed to help quantify the organization’s reporting on how Facebook is enforcing its rules. All information that readers submit (which is anonymized) is entered into a structured database. This will help ProPublica determine how Facebook handled, say, all content that it considered to be a “call to violence,” and what patterns — if any — there are in its responses.

    “The goal here is to figure out how are these rules actually being implemented,” Angwin said. “What does it mean for people who have written a post that they think is fine, but that someone has flagged as hate speech? Does all of this fit within our conceptions of what hate speech is?”

    The Messenger bot is primarily a way for people to share their stories with ProPublica’s reporters, but it also doubles as “another form of storytelling,” said , an engagement reporter at ProPublica. While there have been some high-profile stories about people that Facebook uses to moderate its platform (), many Facebook users don’t understand the human element of Facebook’s moderation decisions. “We see this as a way to get the reporting back out there and give it to another audience that can make use of it and react to it,” Tobin said.

    The use of Facebook, too, was important to the experiment. While news organizations looking to collect stories from readers en masse have always had the option of using tools from SurveyMonkey or Google Forms, ProPublica wanted to use Facebook for this experiment because it was the best place to collect stories about people’s experiences with the platform. “If you’re looking to talk to a community, it’s more powerful if you can go right where they already are,” said Tobin. On the other hand, the organization realized that some users interested in telling their stories may be uncomfortable using Facebook (or have been banned from the platform outright), so its developers .

    The bot, the first of its kind for ProPublica, came with a handful of new challenges and questions for the organization. Early experimentation on how “chatty” the bot should be resulted in a product that is “friendly” but not too verbose. Another issue, which ProPublica hasn’t yet solved, is promotion. While the organization has promoted the bot on its own site, via email, and through Twitter, “the discoverability factor is still hard” when it comes to Facebook bots, said Tobin. Using Facebook Messenger to communicate directly with a news organization’s reporters is still a new idea for most Facebook users, which is why ProPublica has experimented with changing its to both advertise the bot and illustrate exactly where users need to click on the page to share their stories.

    ProPublica is considering more messaging-based projects this year, with plans to experiment with other chat apps as well as SMS. One of the big open questions is how much any of these experiments will entice readers, both in the initial phases and once the novelty factor of the conversational interface wears off.

    And then there are the trolls. Giving some users the opportunity to share their stories with readers opens up the door for troublemakers to do the same. But so far, ProPublica hasn’t seen much of it: Off the 100 or so responses the organization saw on the first day (many of which Tobin said were “really good”), only a few of them came from people attempting to cause trouble. It’s not clear how much that ratio will change as news of the project reaches more people.

    Facebook, for its part, has taken steps to improve its transparency, at least at the margins. , former public editor of The New York Times, to help it become more open and transparent about its decisions about , fake news, terrorism, (Facebook did not respond to a request for comment about the ProPublica bot in time for publishing.)

    But the company has largely been quiet on the issues that ProPublica has raised in its reporting, which is why Angwin says the work is so vital to raising awareness. “This is a very important issue, and Facebook doesn’t have an appeals process for users affected by it. Their decisions are opaque and not explained. We’re acting as that appeals process.”

    POSTED     Aug. 31, 2017, 10:27 a.m.
    SEE MORE ON Reporting & Production
    SHARE THIS STORY
       
    Show comments  
    Show tags
     
    Join the 45,000 who get the freshest future-of-journalism news in our daily email.
    Bad news from Mashable, BuzzFeed, and Vice shows times are rough for ad-supported digital media
    The rapid growth of Google and Facebook continues to take its toll on digital media companies.
    Asking members to support its journalism (no prizes, no swag), The Guardian raises more reader revenue than ad dollars
    The Guardian revamped its ask and its membership offerings — moving from 12,000 members in the beginning of 2016 to 300,000 today.
    Beating the 404 death knell: Singapore news startups struggle to cover costs and find their footing
    Political news reporting doesn’t seem to be holding up well as a business in the city-state. And it’s even harder when you’re seen as “alternative” media.