Potential Pitfalls

Think back to steps 2.12 and 2.13 about error margins. Notice that none of the pitfalls covered in the video are even mentioned in these steps on error margins. Error margins and the pitfalls reflect two separate sets of considerations.

Steps 2.12 and 2.13 are about uncertainty caused by the fact that we interview only a sample of the population rather than the entire population. This sampling issue, at least as presented in these steps, doesn’t systematically “bias” our estimates upwards or downwards compared to the true numbers we are trying to estimate. It does, however, inject uncertainty into our estimates.

In contrast, the issues raised in the present video do systematically “bias” results in one direction or another. For example, a sample packed with rich people will tend to lean Tory or more Republican more than the population as a whole does.

Discussion

What can you add to my list of things that can go wrong with surveys and bias the results in a direction you can identify?

Caution: I’m not suggesting that everything that can go wrong will go wrong. Think of it like this. When you’re planning a family vacation it might be a good idea to first sit down and list all the things that might go wrong. The list might look a little daunting but this doesn’t mean that you should just stay home. In fact, making such a list should increase your chances of making a successful trip.