What’s Next for the Credibility Coalition

The Credibility Coalition (CredCo) today announces it has received generous grants from Google News Lab, the Facebook Journalism Project, Craig Newmark Philanthropies and additional private donors to expand its work to develop indicators of content credibility on the web and develop training data with these indicators based on real-world news articles.

The grants build on a $50,000 Knight Prototype Fund grant received in 2017 and mark a significant increase in resources to support the Coalition’s efforts to develop a nuanced framework of online content credibility. In particular, the new support will support frameworks and processes for assessing the credibility of analyzed articles. Data generated from these efforts will then be made available to researchers, the public, and major content platforms.

Co-founded by Meedan and Hacks/Hackers, CredCo is a diverse, interdisciplinary community committed to improving information ecosystems and media literacy, designing and testing a rigorous set of credibility indicators through an open and collaborative process. Members of the coalition are interested in complementing efforts to address online misinformation by defining factors that communicate information reliability to readers.

“This support gives CredCo the opportunity to design, test, and improve applied responses to the much discussed and researched challenge of misinformation,” noted Ed Bice, CEO of Meedan.

“We’re thrilled to see industry and philanthropic partners join in on the momentum of our cross-disciplinary grassroots initiative born out of MisinfoCon event in February 2017,” noted Jennifer 8. Lee, a journalist and technologist from Hacks/Hackers, which also organizes MisinfoCon. “The funding will allow us to significantly grow our efforts to tackle misinformation in a scaleable, rigorous and open way.”

Rather than jointly deciding whether an entire article is “credible” or “not credible” or endorsing any single index or evaluation of credibility, the coalition fosters the development of a framework around which credibility can be assessed. The group’s goal is to provide a range of indicators that can be used by a person — or a system — to support assessments of credibility related to online content, sources, and claims. Sample indicators include:

• Title Representativeness — Article titles can be misleading or opaque about the topic, claims, or conclusions of the content.
• Tone — Articles may contain exaggerated claims or emotionally charged sections, especially with expressions of contempt, outrage, spite, or disgust.
• Ad Placements — Aggressive placement of ads and social calls may be an indicator of financially-motivated misinformation, for instance by appearing in pop-up windows, covering up article content, or distracting through additional animation and audio.
• Citation Practices — More credible articles accurately represent any sources that they cite or quote, such as articles, interviews, or other external materials.

To date, some of the Credibility Coalition’s core accomplishments include:

• The group’s recent study, A Structured Response to Misinformation: Defining and Annotating Credibility Indicators in News Articles, analyzed annotations for 16 indicators of credibility on 40 of the most shared climate science and public health related articles. Led by researcher Amy X. Zhang of MIT CSAIL, the paper will be presented in April at the Web Conference, a yearly international conference on the topic of the future directions of the World Wide Web, as part of the conference’s special track on Misinformation, Journalism and Fact Checking.
• The data set from the study, developed by annotators using Check, a collaborative verification platform developed by Meedan, and Public Editor, a crowdsourced content analysis system developed by the Berkeley Institute for Data Science and GoodlyLabs and powered by TextThresher, will be publicly released soon. The data will be made available in raw format and accessible through Hypothesis, a general-purpose W3C standard web annotation platform.
• Members of the coalition have hosted workshops with leading journalists, researchers, and technologists in venues like Columbia University, Northwestern University, and the Mozilla Festival. The group has presented its work at the Knight Commission for Trust, Media and Democracy, MisinfoCon London, News Rewired, and NICAR, among other events.
• The coalition helped establish the new Credible Web Community Group (CWCG), the organization responsible for developing protocols and guidelines for the web.

CredCo will work in conjunction with the CWCG, which aims to produce consensus technical standards for exchanging credibility-related data on the web (in the style of schema.org), with the goal of empowering users to better assess the credibility of what they see on the web. The CWCG will work in cooperation with CredCo to drive practical deployment of elements of the Coalition’s work as it reaches sufficient maturity.

“The Web has never been 100% safe, but the danger of misleading information seems to be increasing,” said Sandro Hawke, a long-time member of the W3C staff at MIT who is helping connect CredCo with the W3C community. “While this isn’t entirely a technical problem, it looks like there are some technical steps we can take, as an industry, to help turn the tide and give people better access to information they can trust.”

The Credibility Coalition will present its work at the International Journalism Festival in Perugia, Italy, and at the Web Conference in Lyon, France, and they will lead workshops in June at MisinfoCon in Boston and upcoming MisinfoCon events in Kyiv, Ukraine, and Washington, DC.

###

About the Credibility Coalition

Co-founded by Meedan and Hacks/Hackers, the Credibility Coalition is a diverse, interdisciplinary community committed to improving our information ecosystems and media literacy through transparent and collaborative exploration.

The Coalition was incubated at the first MisinfoCon in early 2017, held at the Nieman Foundation for Journalism at Harvard and the MIT Media Lab. Its members come from a wide variety of disciplines, from data science to media research to sociology, who join weekly calls, regular Slack conversations and occasional in-person workshops and discussions. They come from the following organizations: AAAS, AppNexus, Associated Press, the Berkeley Institute for Data Science, Climate Feedback, Data & Society, Data for Democracy, Factmata, Global Voices, GoodlyLabs, Hypothesis, Snopes, the New York Times, MIT CSAIL and the University of Michigan’s Center for Social Media Responsibility. A list of founding members can be found at https://credibilitycoalition.org/about/.

For more information, please contact hello@credibilitycoalition.org or visit www.credibilitycoalition.org.