Skip to 0 minutes and 3 seconds When we construct a sampling frame for a given survey, we want to make it representative of the people that we’re trying to survey. And so what that means is, if we want to speak to a representative sample of the electorate, we need to make sure that we have right number of old people, the right number of young people, the right number of men, the right number of women, the right number of old women who live in Buckinghamshire and read The Daily Telegraph, or young men who live in Essex and read The Sun. To keep things representative, we’ve had interviewers on boats to go to the Outer Hebrides to do, for example, house condition surveys.
Skip to 0 minutes and 38 seconds You have to make sure you cover the whole country, so yeah, that absolutely still exists. I think the biggest issue that we come across is sample. So we see a lot of unrepresentative samples, we see a lot of membership organisations doing surveys of their members, or newspapers doing their own polls. And some of those do get reported as, or at least initially when those organisations put them out, some of them do make it unclear that they are just a survey of their members. But often then, once they’ve - it’s someone reporting, someone else reporting it - then that tends to get lost.
Skip to 1 minute and 17 seconds And the biggest frustration, I think, is that all we can say is, well, it wasn’t designed to be representative. An online survey panel is if you like a standing database of respondents who you can go to in a relatively cost effective way to ask people, if you like, market research questions. And it could be about anything from the soap powder that you use, through to the cars that you drive or would like to buy. So the idea is that you’ve got a panel of tens of thousands, often hundreds of thousands of people, you know who they are, so then you could target them quite carefully.
Skip to 1 minute and 51 seconds For example, the classic one might be to the main shopper in the household, and then you send a questionnaire to them and interview them about the specific topic. So it’s quite a flexible way of getting to quite targeted groups of the population. YouGov has a panel of 500,000 people who it has recruited to take part in surveys, and then we contact a specific group from within that total panel to be representative of the country as a whole. Not representative of the panel, not representative of the internet population, but the country as a whole.
Skip to 2 minutes and 23 seconds So we will contact the right number of old people, the right number of young people, et cetera, et cetera, to make sure that the final results are representative. And the internet’s helped because, for example, to go in for students, we start off approaching them online, then we ring them up, and then eventually they might get a call from the interviewer. So that’s a way of managing your costs but also for approaching people in a way that works for them.
Skip to 2 minutes and 45 seconds So one of the features of research these days is there’s much more talk of what we call “multimodal research,” where you perhaps start with one method, and then you bring in others to boost that response rate and make it that bit more representative. Compared to face-to-face or telephone surveys, web surveys tend to be cheaper. Typically, they’re around one tenth of the cost of the equivalent face-to-face survey. They also tend to be quicker. At YouGov, for instance, we do surveys of 2,000 people within 24 hours. Whereas 10-15 years ago, that would have taken a face-to-face agency a couple of weeks, if not a month.
Skip to 3 minutes and 22 seconds They also tend to be more accurate on sensitive subjects, so if you’re asking somebody a subject like, let’s say their sexual preference, then to not have an interviewer there means the people are more honest with their responses. But of course there are disadvantages. The most obvious disadvantage is that not everyone’s on the internet. And so there are a group of people who are, shall we say, sceptical towards new technology and new ideas, and so you’re not factoring that into your survey. So that’s problematic. There’s certainly a much broader range of techniques we can use in our market research these days.
Skip to 3 minutes and 58 seconds So we can show you what we’re doing, for example, in the area of neuroscience and then over in the area of behavioural science, and there’s some absolutely completely groundbreaking things going on. But actually, even in neuroscience and behavioural science, what its often being used is to improve the survey questionnaires and to make them better, and smarter tools of actually gathering the attitudes or the behaviour that you’re interested in.
Hearing from the data collectors
In this video, we hear from representatives of three companies involved in the surveying of the public as they discuss the survey methods they use and the challenges they face.
They explain some of the basic principles of surveying, why web-based surveys are an increasingly important mode of survey data collection and how to read survey results properly.
Simon Atkinson is the Chief Knowledge Officer for Ipsos Mori, having worked for the company for over 12 years. Ipsos Mori is the 2nd largest market research company in the UK, conducting surveys for a wide range of international companies, advertising agencies and newspapers.
Joe Twyman is Head of Political and Social Research for Europe, Middle East and Africa at YouGov. YouGov is an international Internet-based market research and data analytics firm, specialising in market research through online methods
Amy Sippitt is the Research and Impact Manager for Full Fact. Full Fact is an independent fact-checking charity that works primarily within the fields of journalism and politics. Funded by donations and trustees, they pride themselves on an impartial approach and often carry out fact-checks when requested to do so by the public.
© The University of Sheffield