Skip main navigation

Nicholas Jewell on quantifying uncertainty

Professor Spagat interviews Nicholas Jewell about quantifying uncertainty in surveys - some things are not readily quantifiable.
13.3
Well, I think the huge advantage of surveys, as you say it, scientifically and statistically is by conducting them using well-understood techniques of sampling, that is, in fact, what allows you to directly [INAUDIBLE] to a measure of uncertainty, assuming everything else went fine. So you get the uncertainty from the fact that I only sampled 1% of the population. And you understand how you sampled that 1%. That allows you, when you extrapolate up to describing an estimate for the entire population, also to extrapolate a measure of uncertainty based on just the sampling properties.
58.1
You’re absolutely correct, though, that that in itself is a narrow version of uncertainty in that it doesn’t account for things that went a little bit wrong or choices that were made in the midst of taking the survey. How you deal with non-response would be a classic example of that. So the non-respondents don’t– the uncertainty estimates don’t maybe go up or down depending on the level of the response, but how much the estimate may be biassed in terms of how non-respondents might have responded is not accounted for by sampling uncertainty. And of course, yet that fundamentally is part of what we don’t know.
104.3
And so statisticians have gotten more and more sophisticated in trying to deal with things like non-response or missing data with fairly weak assumptions to say there’s not only uncertainty because we’re only sampling a piece of the population, but there’s also uncertainty because the group we sample don’t all respond. And then there’s uncertainty because the group that do respond don’t tell the truth, and so on and so forth. And these errors, of course, promulgate. And part of my job, and part of statisticians’ job, is trying to, as honestly as we can, account for all of these sorts of error, not just a sampling variation.

We have just expended a fair amount of energy learning about the quantification of uncertainty surrounding sample survey-based estimates of violent war deaths. So perhaps some of us are not currently in the best frame of mind to receive Nicholas Jewell’s message that there is still more uncertainty that our methods do not account for. But receive it we must.

You may recall that I made the same point back in step 2.4, saying that I mentally adjust all error margins by doubling their width. This is an arbitrary rule of thumb but it does suggest the importance that I attach to Nicholas’ main point in this clip.

This article is from the free online

Accounting for Death in War: Separating Fact from Fiction

Created by
FutureLearn - Learning For Life

Our purpose is to transform access to education.

We offer a diverse selection of courses from leading universities and cultural institutions from around the world. These are delivered one step at a time, and are accessible on mobile, tablet and desktop, so you can fit learning around your life.

We believe learning should be an enjoyable, social experience, so our courses offer the opportunity to discuss what you’re learning with others as you go, helping you make fresh discoveries and form new ideas.
You can unlock new opportunities with unlimited access to hundreds of online short courses for a year by subscribing to our Unlimited package. Build your knowledge with top universities and organisations.

Learn more about how FutureLearn is transforming access to education