Skip main navigation

New offer! Get 30% off one whole year of Unlimited learning. Subscribe for just £249.99 £174.99. New subscribers only. T&Cs apply

Find out more

Potential Pitfalls

In this video Professor Michael Spagat presents some of the main things that can go wrong in surveys. But what can go wrong doesn't have to go wrong.

Think back to steps 2.12 and 2.13 about error margins. Notice that none of the pitfalls covered in the video are even mentioned in these steps on error margins. Error margins and the pitfalls reflect two separate sets of considerations.

Steps 2.12 and 2.13 are about uncertainty caused by the fact that we interview only a sample of the population rather than the entire population. This sampling issue, at least as presented in these steps, doesn’t systematically “bias” our estimates upwards or downwards compared to the true numbers we are trying to estimate. It does, however, inject uncertainty into our estimates.

In contrast, the issues raised in the present video do systematically “bias” results in one direction or another. For example, a sample packed with rich people will tend to lean Tory or more Republican more than the population as a whole does.


What can you add to my list of things that can go wrong with surveys and bias the results in a direction you can identify?

Caution: I’m not suggesting that everything that can go wrong will go wrong. Think of it like this. When you’re planning a family vacation it might be a good idea to first sit down and list all the things that might go wrong. The list might look a little daunting but this doesn’t mean that you should just stay home. In fact, making such a list should increase your chances of making a successful trip.

This article is from the free online

Survival Statistics: Secrets for Demystifying Numbers

Created by
FutureLearn - Learning For Life

Reach your personal and professional goals

Unlock access to hundreds of expert online courses and degrees from top universities and educators to gain accredited qualifications and professional CV-building certificates.

Join over 18 million learners to launch, switch or build upon your career, all at your own pace, across a wide range of topic areas.

Start Learning now