Learn more about this course.

The Idea of a Survey

In this video Professor Michael Spagat asks what is a survey and why do we do them? Then he plays tennis with Serena Williams (in his mind).

After I recorded this video a veritable volcano blew its stack on social media in a way that is highly relevant to my last point about knowing exact wordings of questions.

A YouGov poll asked people about their prospects for winning a tennis point against Serena Williams.

The responses to this question can and has been discussed vigorously (to put it mildly) from many angles. You’re all free to express your own views in our discussion forum as long as you follow our rule to be nice to each other (which was not observed very much on social media).

Here I make a relatively narrow point that extends my video discussion. Different people can interpret a single question in different ways so you should be careful about assuming that you know what question various survey respondents are actually answering.

Here is the YouGov question:

“Do you think if you were playing your very best tennis, you could win a point off Serena Williams?”

12% of men and 3% of women answered “yes”. There’s a rather big gender difference in the responses that’s worth discussing but I won’t do so here. Instead, I draw your attention to YouGov’s interpretation of the finding that they tweeted out, thereby launching the whole firestorm:

“One in eight men (12%) say they could win a point in a game of tennis against 23 time grand slam winner Serena Williams”

I see two main problems with this interpretation.

First, if “Serena Williams” had been preceded by “23 time grand slam winner” in actual question that YouGov asked then I’m pretty sure that fewer people would have answered “yes”.
Second, and more importantly, the actual question does not place a limit on the number of chances you get to win a point. Yet YouGov’s interpretation places a very low limit, specifically 4 chances, on your wiggle room. That’s a huge difference. Surely, some people thought something like the following: “if Serena serves 1,000 times eventually she’ll double fault so my best answer is ‘yes’”.

The author of the linked article gives third interpretation.

“…the research firm ask[ed] British citizens whether they believed that they would be capable of winning a point in a tennis match against Serena Williams.”

A match yields considerably fewer chances than 1,000 but far more chances than 4. In any case, the question says nothing about a match.

Another wrinkle is that in online discussions some people emphasized the phrase “playing your very best tennis” which is in the actual question wording. In my opinion, this bit is only relevant to a tiny handful of people who are extremely good tennis players. But I don’t really know how other people might interpret and rate the importance of this phrase.

Discussion

Speculate on what was actually measured by this question and propose further research that could test your speculations. And, please, be nice to each other!