Home / Politics & Society / Disinformation, Misinformation, and Fake News Teach-Out / Measuring Impact of Disinformation Campaigns
Measuring Impact of Disinformation Campaigns
Ben Nimmo of Graphika discusses how experts measure if disinformation campaigns made an impact.
<v ->One of the most difficult questions</v> when you’re looking at online influence operations, is whether it actually made a difference? Did the influence operation ultimately have influence? And the reason it’s difficult is that so much of the operation is aimed at changing people’s perceptions, or changing their behavior.
But unless you know the baseline and unless you actually have a very reliable way of judging what people were thinking before and after, you can’t measure the change that went on and you can’t separate it out from the impact of all the other things that are happening at the same time, information operations don’t exist in a vacuum, they’re surrounded by all the real world events and all the other considerations that people go through in their daily lives. One proxy for measuring impact in this way would be to use engagement statistics on social media.
You can look at numbers of shares or of likes, and you can try and judge from that, how many people might’ve seen this content, but the problem is, there’s an online marketing fake engagement, so it’s possible to go online and buy thousands of likes or thousands of shares, which can make a post or a campaign look very effective, where in fact, the only accounts that engage with that content were fake accounts.
And we’ve seen numerous examples of that from influence operations from different countries, for example, there’s a Russian operation called Secondary Infektion, which would post forged documents on hundreds of different social media platforms, but as far as we’ve been able to judge, despite the wealth of content that they put out there, very few real users ever engaged with that content, and some of the users who engaged called it out as Russian propaganda, or there’s a Chinese operation that we call Spamouflage Dragon, which was across YouTube, Twitter and Facebook, posted high volumes of content, generated hundreds of likes and shares and comments from different accounts, but as far as we’ve been able to tell, all those accounts that engaged with the operation, were part of the operation, so as far as we’ve been able to tell, nobody outside that operation actually engaged with the content at all.
So there are problems with using the numbers as basic metrics. But what you can do is ask yourself of the question, where is this story spreading, and who’s picking it up? And that can give you a scale of options for how much influence or how much impact an operation is likely to have. So for example, if a particular story is only spreading in one community on one social platform, it’s not going to have much of a reach to start with, because no matter how tightly woven that community is, the story is not moving beyond it, if it’s only on one platform by definition, you’re limited to users of that platform. And then the next question is who’s picking it up?
Can you see influencers, big hitters on social media? Can you see politicians or journalists? Can you see the mainstream media picking up this story and amplifying it? Because they have a reach which goes beyond social media, and one of the things that we’ve seen in, for example, the Russian information operation against the US in 2016, with the number of times that claims from the Russian troll farm or posts from the Russian troll farm were actually quoted by the mainstream media, when that happens, the post has suddenly got a much greater reach, because it’s not limited to one platform, it’s not limited to one group.
Sometimes we’ve even seen in different contexts, politicians picking up the claims of disinformation operations and publicizing them in their political campaigning, and that’s one where potentially a false claim can actually reach into a legislative initiative or it can come up in a parliamentary debate. So the way to think of influence operations is to look beyond the numbers, and to look for whether there are breakout moments in the operation, does it stay in one community or does it break out into different communities? Does it stay on one platform or does it actually break out onto different social media platforms? Does it break out further into the mainstream media or into political debate?
And these can give you a rule of thumb for how far a particular false story or a piece of disinformation or an influence operation is actually spreading. And that’s important to think of, because it’s a reminder that for all of us, we are potential influencers. If any of us are on social media and we share a full story, we’re effectively giving it the stamp of our own credibility, we’re saying, “Hey, I’ve seen this story and I’m passing it on to you.” That comes with risks, if the story is false, we have unwittingly amplified, a falsehood which could be designed to influence other people.
And it’s a particularly important consideration for high powered influencers who have hundreds of thousands of followers, because they can spread a story so much further. And so it’s really important for all of us on social media, particularly at times of heightened tension and heightened uncertainty, heightened pressure, such as you get around in an election, to really check what we’re looking at and where it’s coming from before sharing it, thinking before you click is always important, but in an election time, it’s even more so.
What impact does it have?
In this step, Ben Nimmo, Director of Investigations for network analysis firm Graphika, discusses how experts measure the impact of disinformation campaigns. He talks about using engagement statistics on social media as a proxy measure and the problems with that approach as an accurate metric. As a disinformation investigator, he asks himself two questions: “where is this story spreading?” and “who is picking it up?” These questions can give you an idea of the impact of a specific campaign.
Discussion: Ben suggests that we “think before we click” and encourages people to understand what it is that they are sharing before amplifying its impact. Reflecting on the activity in the beginning of this Teach-Out, what would you do differently when considering sharing a news article or social media post? How has your thinking evolved? What impact might our social media behaviors have on the way false information flows?
This article is from the online course:
Disinformation, Misinformation, and Fake News Teach-Out
Created by
This article is from the free online
Disinformation, Misinformation, and Fake News Teach-Out
Reach your personal and professional goals
Unlock access to hundreds of expert online courses and degrees from top universities and educators to gain accredited qualifications and professional CV-building certificates.
Join over 18 million learners to launch, switch or build upon your career, all at your own pace, across a wide range of topic areas.
