Home / Business & Management / Data Science / Survival Statistics: Secrets for Demystifying Numbers / What is Margin of Error Polling?
What is Margin of Error Polling?
The Professor introduces three important new concepts/ideas: The first is “margin of error” which you may have heard of but which is a tricky concept to actually understand well. For now, let’s just say that where there is a survey estimate there is also uncertainty about the estimate and the margin-of-error concept represents an attempt to quantify some of this uncertainty.
Hello, again. I’m here with Chris Hanretty– Professor Chris Hanretty– of the Politics and International Relations Department at Royal Holloway. Chris works a lot on polling and elections, and he works closely with the UK polling company Survation. So to get things rolling, Chris, let me ask you a question that you probably get a lot in one form or another. People will often assert that election polls always get it wrong, and why is that? They don’t always get it wrong. I think there’s a natural human tendency to focus on the mistakes.
If you think about your water supply, your internet connection, most of the time it works fine, but you really, really notice it when your internet goes down– because that drives people crazy. And I think it’s the same with election polling. We can all probably recall some big, high-profile mistakes. So in the UK, polling for the 2015 general election and the 2017 general election wasn’t that great. In the US, the 2016 presidential election, most people went to bed at night thinking that Clinton would have won, and we ended up with President Trump. So you’ve mentioned three elections in particular. You mentioned Trump-Clinton, and then UK elections 2015, 2017 as not complete successes.
I mean, would you classify any of those as failures, or perhaps just not as good as we might want, or? Yeah, I think 2015 in the UK probably has to go down as a big failure, just because the figures for the top two parties were comfortably outside the margin of error. 2016 in the US, I think, is a slightly different case, because we’ve got to remember that Clinton was the popular vote winner. And national opinion polls in the States– they’re just trying to predict vote share, not Electoral College votes.
So that, I think, is slightly different, because we use polls to understand how people are going to vote, but then people who go on and do that extra step of election forecasting, they have a second job of saying, OK, if that’s how people are voting nationally, here’s how it works out in terms of particular areas. So 2016 presidential election– I don’t think that’s a huge, huge failure, but people regard it in that way, because the polls pointed to the wrong winner. And a lot of the time, people don’t care about the percentage error or any numbers. They just want the poll to point to a winner. And when a poll doesn’t pick the right winner, they get upset– understandably. Right.
I mean, that actually segues pretty well into the second question I have on this sheet here, which is why is it so hard to get election predictions correct? I mean, it seems like predicting elections is more difficult than just taking temperature of public opinion. What do you think the issues are, there? I think there’s some issues that are common to all elections, and there are some issues which are more problematic in, probably, the UK and the US. So if we think about what we’re doing in an election poll, you’re asking people two questions. Are they going to go out and vote? And if they vote, who are they going to vote for? And those two questions are certainly different.
The question about, are you going to vote? That’s a question asking about your future intention. And those kinds of questions are always difficult. People can say something with a complete straight face, and it turns out not to be right. I can tell you tomorrow I’m going to get up super early, half five, go to the gym before I come to work. I can mean that now. I can be absolutely truthful when I say that to you now– Then the alarm clock goes off. –but tomorrow morning, we have a different story.
And so when we’re asking people whether they’re going to turn out to vote, 80%, 90% of people will say, yep, I’m going to vote tomorrow– or whenever the election is. Now, we don’t see turnout at 80% or 90%, and so we have to make some adjustments. So that’s one tricky issue that you get in election forecasting which you don’t have when you’re just asking people, how much money did you spend on coffee this week? They’re just reporting some piece of information that they have access to. Now, that’s common to all elections. Different issue comes up in the UK and the US where you’ve got two sets of numbers.
You’ve got your votes, and then you’ve got your seats in the UK, or your Electoral College votes in US presidential elections. And that mapping is hard, because you could get the national figure in terms of votes just right, but depending on how those votes are distributed across the country, you might be off with your Seats figure.
Chris Hanretty is a Professor of Politics at Royal Holloway, University of London. His research spans empirical legal studies and studies of public opinion and voting behaviour. He is the winner of the UK Political Studies Association’s Richard Rose Prize (for a “distinctive contribution to the study of British politics”) and the 2018 Philip Leverhulme Prize.
The Professor introduces three important new concepts/ideas:
- The first is “margin of error” which you may have heard of but which is a tricky concept to actually understand well. For now, let’s just say that where there is a survey estimate there is also uncertainty about the estimate and the margin-of-error concept represents an attempt to quantify some of this uncertainty.
- The second is that the answers that people give to a survey might be wrong. Specifically, Chris raises the possibility that people might say that they are going to show up and vote but then not actually do so in practice. This is not an entirely new idea for us, falling under the broad category of “how is that measured?” Still, it’s new enough that it’s worth highlighting.
- The third is that national vote shares may have only a loose connection with election results. The best recent example of this truth is the Trump-Clinton election. We can say that the late polls came out fairly close to national vote shares. But a reasonable critic may say that this doesn’t really matter because US presidential elections are not decided by national vote shares.
Discussion
As of now, how do you understand the concept of margin of error? Please don’t do any research. Just go based on the state of your thinking at the moment.
This article is from the online course:
Survival Statistics: Secrets for Demystifying Numbers
Created by
This article is from the free online
Survival Statistics: Secrets for Demystifying Numbers
Reach your personal and professional goals
Unlock access to hundreds of expert online courses and degrees from top universities and educators to gain accredited qualifications and professional CV-building certificates.
Join over 18 million learners to launch, switch or build upon your career, all at your own pace, across a wide range of topic areas.
