The Top Skills in Pakistan for Employability
Find out the top skills employers in Pakistan are looking for. We take a look …
Home / Business & Management / Data Science / Survival Statistics: Secrets for Demystifying Numbers / The Idea of a Survey
The Idea of a Survey
In this video Professor Michael Spagat asks what is a survey and why do we do them? Then he plays tennis with Serena Williams (in his mind).
What is a survey? And why do we conduct them? To avoid controversy, I’ll answer this question within the context of an innocuous example, abortion. Suppose we want to know the percentage of the US population that supports abortion. Our first idea is simply to ask people what they think. This makes sense, right? But now, two obvious questions jump out at us. One, who do we ask? And two, how do we ask them? One answer to the first question is that we could ask every single person in the whole country. But there’s a glaring problem with this proposal. It would be terribly expensive to do this. It’s much cheaper to just interview some people.
But then we need to be careful about how we pick the people to interview. Interviewing people as they walk out of an abortion clinic will almost surely lead to different results from what we would get if we interview people as they walk out of a church. The group of people we actually interview is called our sample. My point about churches and abortion clinics is just my way to say that we want our sample to be representative of the whole population that we’re trying to understand. Ideally, our sample will be a microcosm of the US. That is, abortion support within the sample should be equally strong as abortion support within the country as a whole.
The question then becomes, how do we get such a representative sample? We’ll return to this issue shortly in the course. Let’s now return to question number two. How do you elicit people’s views about abortion? In fact, the reason I picked the abortion example in the first place is that it’s easy for anyone to think up loaded questions that would push the responses in one direction or another. So I won’t even bother here to propose some loaded questions. Instead, I’ll show you a subtle case where a seemingly innocuous change in an American survey question seems to have a fairly big effect on people’s answers. Question 1, abortion should be legal in almost all cases. Agree or disagree? 28% ticked agree.
Question 2, women should have a legal right to safe and accessible abortion in almost all cases. Agree or disagree? Now, 37% agree. Conclusion, you need to know the exact wordings of questions that are asked in surveys before you can begin to interpret a survey. Always ask to see the questions.
After I recorded this video a veritable volcano blew its stack on social media in a way that is highly relevant to my last point about knowing exact wordings of questions. A YouGov poll asked people about their prospects for winning a tennis point against Serena Williams.The responses to this question can and has been discussed vigorously (to put it mildly) from many angles. You’re all free to express your own views in our discussion forum as long as you follow our rule to be nice to each other (which was not observed very much on social media).Here I make a relatively narrow point that extends my video discussion. Different people can interpret a single question in different ways so you should be careful about assuming that you know what question various survey respondents are actually answering.Here is the YouGov question:
“Do you think if you were playing your very best tennis, you could win a point off Serena Williams?”
12% of men and 3% of women answered “yes”. There’s a rather big gender difference in the responses that’s worth discussing but I won’t do so here. Instead, I draw your attention to YouGov’s interpretation of the finding that they tweeted out, thereby launching the whole firestorm:
“One in eight men (12%) say they could win a point in a game of tennis against 23 time grand slam winner Serena Williams”
I see two main problems with this interpretation.
- First, if “Serena Williams” had been preceded by “23 time grand slam winner” in actual question that YouGov asked then I’m pretty sure that fewer people would have answered “yes”.
- Second, and more importantly, the actual question does not place a limit on the number of chances you get to win a point. Yet YouGov’s interpretation places a very low limit, specifically 4 chances, on your wiggle room. That’s a huge difference. Surely, some people thought something like the following: “if Serena serves 1,000 times eventually she’ll double fault so my best answer is ‘yes’”.
The author of the linked article gives third interpretation.
A match yields considerably fewer chances than 1,000 but far more chances than 4. In any case, the question says nothing about a match.Another wrinkle is that in online discussions some people emphasized the phrase “playing your very best tennis” which is in the actual question wording. In my opinion, this bit is only relevant to a tiny handful of people who are extremely good tennis players. But I don’t really know how other people might interpret and rate the importance of this phrase.“…the research firm ask[ed] British citizens whether they believed that they would be capable of winning a point in a tennis match against Serena Williams.”
Discussion
Speculate on what was actually measured by this question and propose further research that could test your speculations. And, please, be nice to each other!This article is from the online course:
Survival Statistics: Secrets for Demystifying Numbers
Created by
-
The Top Skills in Nigeria for Employability
Find out the top skills employers in Nigeria are looking for. We take a look …
The Top Skills in Canada for Employability
Find out the top skills employers in Canada are looking for. We take a look …
Top 15 courses from Australian universities 2021
On our list of the 15 top online courses from Australian universities, you’ll find some …
This article is from the free online
Survival Statistics: Secrets for Demystifying Numbers
Our purpose is to transform access to education.
We offer a diverse selection of courses from leading universities and cultural institutions from around the world. These are delivered one step at a time, and are accessible on mobile, tablet and desktop, so you can fit learning around your life.
We believe learning should be an enjoyable, social experience, so our courses offer the opportunity to discuss what you’re learning with others as you go, helping you make fresh discoveries and form new ideas.
You can unlock new opportunities with unlimited access to hundreds of online short courses for a year by subscribing to our Unlimited package. Build your knowledge with top universities and organisations.
Learn more about how FutureLearn is transforming access to education
