New verification projects should consider core lessons from older and better-established fact-checking styles while adapting for social media ecosystems, Chris Blow writes for First Draft.
Careful design can help make public-interest research projects more effective on social media. Usability, legibility and readability are essential aspects of verification. But specific design guidelines seem under-discussed in the academic and industry literature. For new types of verification projects designed for sharing on social media, the visuals of verification are especially important.
News graphics have special requirements when fact-checking and debunking. A few templates currently exist for newsrooms and independent organizations starting new types of web-based verification and research projects, but many opportunities remain to be explored. Here are a few.
‘You gotta fight fire with fire here’ — Alexis Madrigal
Three years ago this week, in October 2012, Alexis Madrigal with Megan Garber, Chris Heller and Tom Phillips created an article debunking Hurricane Sandy images, a post that was an important experiment in “real-time debunking”.
In his 2015 report, ‘Lies, Damn Lies and Viral Content‘, Craig Silverman quotes Madrigal in describing his motivation as simply, “to sort that shit out”. The result was a simple and effective visual system and design workflow for talking about viral misinformation.
Based on the success of that post and other recent success stories for both professional and independent verification projects, our community is taking lessons about how to talk about misinformation and earn good traffic for it.
The results of fact-checking are mixed. We know the process carries risks. The result on the public misinformation dynamics seems unclear. (See, for example, The Washington Post’s Michelle Amazeen’s work.)
But the opportunity for newsrooms and independent media seems big. Many projects are emerging, experimenting and looking at new methods and bigger audiences. Madrigal’s post on the Atlantic shows this type of public-interest ‘debunk collection’ can be super popular with readers — Silverman quotes Madrigal as saying it was actually his most popular post at the Atlantic.
It’s possible to get this stuff wrong, though. Misinformation is dangerous, and a poorly designed debunk can make the problem even worse. Even before we get to the design issues there are concerns about seeking permission, ethics, payment, and credit, as Eyewitness Media Hub’s Claire Wardle wrote recently.
Misinformation often degrades the public sphere by exploiting stereotypes and social fears. If verification projects present their results in a misleading format, then readers will be even more misinformed about the issue.
Considering our vulnerability to cognitive biases and the illusion of truth, the risk is that our verification efforts could muddy exactly the messages we are seeking to clarify.
Practitioners of various styles of truth-oriented information campaigns can learn from each other. According to the Duke Reporters’ Lab there are now 89 officially recognized organizations around the world checking political claims. These official fact-checking projects can be considered a core of the verification practice. Fact-checking like this has the advantage of being researched by academics — for example Nyhan’s research shows the importance of such projects as a deterrent to political misinformation and that exposure to fact-checks helps people become better informed.
But what specifically makes such visuals work? What makes them fail? So far there is too little investigation. But we can start to formulate heuristics. With a mind toward user experience and usability, we can ask better questions to advance the state of visual debunks: Why does Politifact’s “pants on fire” joke work? Why is the diagonal stamp format so popular? What font is best? What aspect ratio to use?
Considering the importance of fact-checking in public discourse, it makes sense that the usability aspects of such outputs should be taken seriously. They are a key point of contact between fact-checking efforts and readership. And on social media, where visual content often resonates with users, fact-checking needs to take both a visual and verbal form.
Amazeen confirmed that most readers like visual rating scales in fact-checking efforts, in addition to written analysis.
Increased design scrutiny of these assets can perhaps help political fact-checking and verification projects alike develop their voice on social media; we can hopefully prove clearer, more effective more newsroom templates for generating credibility and accountability in public.
Whereas traditional fact-checking relies more on domain-specific expertise, the newer verification work is more diverse and often deals with basic questions about the provenance of eyewitness media.
In addition to a core of ‘traditional’ political fact-checkers, new types of semi-public social verification projects like Bellingcat show that public-interest campaigns can drive engagement and sometimes break huge stories. (Disclosure: I work on Bellingcat’s Checkdesk, a micro-site that lives alongside the Bellingcat.com CMS. It’s a safe space for crowdsourced research.)
First Draft is especially curious about these new channels, which supplement the professional fact-checking elite. For one example of this type of project in the context of politically sensitive, collaborative journalism on a global scale, see our post from last month: How community verification and transparency can drive powerful engagement.
Indeed, independent verification and investigation projects are evolving rapidly in ways that have not yet been systematically studied compared to the traditional fact-checkers. Many workflows are now being refined by practitioners and are overdue for closer focus on the design choices that make them work well in a networked environment.
A proliferation of verification activity in the last two years is leading to some design innovations which are particularly evident in social media. These practitioners operate in public for all kinds of reasons; regardless of their motivations it is important to observe them closely and learn from their innovations.
Print and TV guidelines offer clues
Because there are not yet many clear visual guidelines from debunkers, we suggest that new verification projects generally should look to fact-checking efforts for guidelines on how to debunk material successfully in the public interest.
The principles and idioms of traditional fact-checking can serve as foundations for more adventurous new types of information campaigns. Some of the political fact-checking visual systems like the PolitiFact Truth-O-Meter™, seem especially resonant with readers on a global scale; they should be emulated and taken further by new experiments.
We can find plenty of TV discussion of debunks which hasn’t followed the TV fact-checking guidelines from FlackCheck. The Weather Channel does not follow these guidelines in their debunks of viral imagery, and the result is highly misleading without the audio, or if the visual has been detached from the audio entirely (for example, in an animated GIF).
Such visual systems can be considered as an improving and improvable interface pattern in a media criticism, something that is a known quantity with readers who may not be used to thinking in media-critical ways. Design patterns and anti-patterns identified by the fact-checking literature should be applied to newer verification projects. Nyhan, Amazeen and Silverman’s encouragement of visual systems should be taken seriously.
Bloggers and multinational broadcasters alike should get their act together, lest they pollute the media ecosystem further
Lucas Graves’s research has inspired a lot of useful contemplation and opened many questions about the best practices.
As Politifact’s Bill Adair wrote in August: “Are we writing our fact-checks too long? Too short? Are we using enough data visualizations to help readers? Should we take the time to create more infographics instead of simple charts and tables? What do we need to do to give our fact-checks authority? Are links sufficient? Or should we also include quotes from experts?”
Each of Adair’s questions could launch a Ph.D, but practitioners of new verification projects have a need to figure these things out immediately. An evolving style of verification is needed because misinformation is evolving.
Let’s look at some more examples.
Aesthetics of automation
In some cases, more algorithmic approaches like TrooClick, LazyTruth and TruthGoggles can be developed to semi-automatically intervene in the flow of misinformation. This seems exciting to verification UI designers because it creates new interactive and visual possibilities. TruthTeller is an example of automated fact-checking with a flourish.
Truth Teller uses overlays to dissect video of politicians. After a lead-in pause for “Analyzing Speech” we see a claim and clear “False” stamp. The delay helps set user expectation that the media is under scrutiny. It seems exciting, if it works.
Using print-worthy illustration: El Sabueso
If you want something special, one way to go is to use custom illustration and more expressive categories. The Mexican fact-checking column El Sabueso does this beautifully with an eight-part color scheme and this lovable bloodhound:
— AnimalPolitico.com (@Pajaropolitico) 4 сентября 2015 г.
A DOWNRIGHT GLAMOUROUS animated gif of a bloodhound’s ears flapping is a way to engage the public sphere in media criticism. It gets attention, carries the newsroom brand, and tells readers that this project won’t be some dreary fussiness. (There is a rendering error in the loop, but it feels charming, almost intentional).
Using a time-lapse gif to debunk Twitter bots: PicPedant
Taking another example of creative new forms of visual verification, @PicPedant uses an inventive format of time-lapse overlays to show tweets that have been reused by several spam accounts.
— PicPedant (@PicPedant) 21 октября 2015 г.
A time-lapse is a great way to give a visceral sense that something is wrong.
And by overlaying each of the images to create an animation, the Tweet is very effectively shown to be some kind of spam. An excellent visual technique.
— PicPedant (@PicPedant) 21 октября 2015 г.
Again, by fading between the hoax and original photo, the manipulation becomes clear. But clearer over-labeling or side-by-side placement would be useful, like some kind of stamp.
But perhaps this is going too far. In some cases for entertainment it might be appropriate to recirculate photos that have been filtered for artistic purposes.
Does every Instagram filter need to be spelled out? Every focal-depth and film type? PicPedant is benefitting from the manipulation by debunking it and does not credit the author — would your editor let you publish this kind of piece in your newsroom? We can leave that as an open question — the purpose of this article is not to suggest exactly how to treat your questionable images, but rather to explore what your options are.
Coming back to that simple stamp
In the Hurricane Sandy story, the design execution was effective because it puts that classic stamp on top of the faked images. To improve handling of sensitive media, consider the simplest stamp like this. It can be created in many kinds of image software. Simple as it is, this works well, especially when combined with the side-by-side technique.
The stamp technique is surprisingly simple and strongly in the public interest, incorporating meme-style image text directly into the meme image. A stamp can be applied directly to imagery — a destructive change that permanently flags the imagery as fake.
It’s not an archival technique for preserving something; it’s defensive publishing that creates a totally new asset with the warnings built, in so anyone who takes a screenshot and shares it will inherit that stamp. It’s smart.
The Pope table cloth trick video
Consider the recent fake video of the Pope doing a table cloth trick which was originally on the Ellen Show. When a talk show does jokes like this, it seems harmless enough. But what happens when realistic fake videos like these can be produced in less than a day? How will journalists manage this new type of social media which manipulates reality so directly? What responses are journalists using as hoax hypermedia becomes hyper-realistic?
Matt Novak’s coverage of this hoax on Gizmodo’s Factually project explained the story with a side-by-side comparison of the unedited video with the edited video. A side-by-side view is a great way to present this type of fake. It contains the manipulation and immediately provokes media-critical dialogue about it.
If there is any possibility of a fake being misunderstood, even further manipulation of the image seems warranted on ethical grounds. Side by side is great — but darker gradient overlays can be added to help further obscure the viral imagery being debunked. This approach can help distance a reader from the immediacy of a hoax image, just as a blur is used editorially to cover violent images. The degree of cover needed to contain the confusion of a hoax is variable and ideally should be determined by an editor.
The best stamp designs are ultra high-contrast, appropriate for readers who are multitasking
These are as simple as highway signs. They should reframe a fake and make it’s questionable status immediately obvious. They help us keep a healthy sense of incredulity.
By superimposing image text, we can help ensure that we are talking about a debunked artifact, not reality, and have a conversation about the falsification of media without merely recirculating unverified media.
For broader circulation beyond a media-critical space like this one, a static image with even greater clarity might be appropriate.
The point of such editing is to layer on warnings that make it safer to talk about the video. With GIF software we can also adjust the framerate to make the hoax less realistic. Silverman refers to this as hedging. Media ethicists will disagree about what degree of hedging makes it safe to talk about the video in which contexts, but it’s worth considering.
We could go on, adding more overlays to the video, covering, filtering and hedging until we are certain it’s safe to recirculate for discussion. The final result should be measured by how resilient it is — the question is, how much does this design asset resist misunderstanding, while still facilitating discussion about the hoax?
The purpose here is not to find the perfect aesthetic, but rather to make the point that we have defensive design techniques at our disposal. The final result would probably involve a combination of several techniques, in conversation with an editor.
Other examples of visual debunking
Antipatterns: what not to do
Here are some lessons taken from selected examples of debunk visuals that pose usability problems and could potentially misinform readers on social media:
- Humor can be confusing. Even when using ‘obvious’ Photoshop montages (below), Africa Check risks misinterpretation. This might be appropriate for some non-serious topics, but should raise a red flag for careful editors.
- Don’t use stock imagery. It makes the debunk feel insubstantial, more like the start of an argument than the conclusion. Factcheck.org used a fake doctor in 2013 (below), and it doesn’t present their strong reputation in the best light. In this context, readers seem sensitive to aesthetic clues which appeal to emotion instead of logic.
- Don’t incorporate the message that you are debunking too prominently. For example, the example “Ted Cruz says Iran has a holiday called Death to America Day” has MOSTLY FALSE in small text. Worse, the Politifact Texas logo in the bottom right seems to suggest that this claim is true (their logo always points to the right side of the meter).
- Don’t recirculate hoax meme imagery on social media directly (Below at right). The reason might be surprising — sometimes Twitter shows media in your timeline without your Tweet text. Remember that we are not always in control of how these assets present to end-users — that’s why it’s so important to modify the graphics directly with an embedded debunk stamp. That way, they always contain the debunk.
Watch out for how your images actually display on social media. For example, images on Twitter are not usually shown full size until you tap to engage them. (As of October 2015, they need to have a 2:1 ratio to show fully without tapping on them.) Of all the UI design pitfalls evaluated here, this is probably most common. The result of this bug is pretty serious — it means that people who scroll past your graphics in their timeline will sometimes only see part of the image. This is awkward for all publishers on Twitter, but if you are publishing carefully crafted visual debunks it can be even worse. Removing those few pixels might seriously truncate your message.
As an industry we’re just getting started here, and all of the above-mentioned organisations are leading the way in terms of debunking hoax stories or misrepresented facts. Debunking activity is not limited to professional journalists and debunk visuals don’t have to come at the end of the process. What might debunk visuals look like that try to actually recruit reader input?
For discussion, below are a couple of concepts from our Checkdesk project that incorporate the displacement technique with warning emoji. What do you think?
Summary ideas and questions
If you are engaging in fact-checking, how should you actually present your results?
- Consider using side-by-side fake and real images like Novak did with the Pope video. To safely show video consider a ‘filmstrip’ approach. For multiple videos, consider a matrix like Flackcheck does with campaign videos. This helps distance the reader from misinformation.
- Use a clear stamp of verification status that covers the media in question. Use overlay annotations to indicate which aspects are faked. If you debunk something and don’t clearly show what you are debunking, you’re risking being terminally vague, like the Weather Channel example. Superimpose text directly on top of the image to create a safer composite asset that can be shared.
- Use clear text in your labels. Don’t muddy the facts with a clickbaity text annotations or headlines. As Silverman says: “many news organizations pair an article about a rumor or unverified claim with a headline that declares it to be true. This is a fundamentally dishonest practice.” Link-bait style headlines, which might be an appropriate art for some kinds of entertainment journalism, are inappropriate in this type of effort.
- Design defensively. Learn more about the contexts in which your visuals will be displayed: For example, some views on Twitter only fit images that are 2:1 aspect ratio. Use images that are twice as wide as tall to avoid cropping of important debunk overlay text.
- Consider affordances for collaborative fact-checking. How can we involve more people in media criticism? The recent fact-checking of the G20 Summit demonstrates that there can be important successes through this type of collaboration. Like the American Diabetes post above, in some cases it’s best to reach out to your readers to get feedback from these outputs. Visually, these interactions can be a potent part of a dialogue with an audience.
As Madrigal, Graves, Amazeen, Nyhan, Novak and others have shown, responsible verification projects can be very popular with readers.
“The debunking efforts in the press are in some ways an anti-viral viral content strategy, a maneuver to insert themselves into trending content by examining and verifying viral stories. This brings traffic to debunkers, but it also helps spread the truth about a viral story,” wrote Silverman in his study.
Gawker Editor Max Read wrote, we’re “ankle-deep in smarmy bullshit and fake ‘viral’ garbage.”
If these techniques can work reliably and safely for journalists and earn traffic, it’s like finding a way to reliably convert that bullshit into fuel.
As a community of both journalistic fact-checkers and nontraditional verifiers, we’ve only starting to understand some of the usability implications of these interface and interaction designs.
This article is based on a lightning talk at ONA15. Thanks for feedback and encouragement! The original designs above, where noted, are released under the CC-BY; remixes encouraged.
By Chris Blow, First Draft