Analysis of quantitative data

In this step, we will discuss the analysis of quantitative data There’re many different ways to present and analyse data from quantitative studies. As we discussed in week one, quantitative studies collect data that can be measured or counted. They may answer questions such as how often, how many, how much? For example, we may want to know how many people are living with a disability, or how often were different challenges attending school reported by children with disabilities, or averages, such as how much do people with disabilities spend on health care? In addition to these basic measures, there’re many ways that statistical methods can improve how we analyse and interpret quantitative data.

This includes how to decide how certain we are about our results, and more accurately compare results between groups. Let’s first look at how statistics can help us decide how certain we are about our results. As we discussed last week, we don’t usually collect data from everyone in a country or area, but instead, collect data from a sample of people that we think represent the whole population in that area. For example, we did a survey of disability prevalence in the Maldives. We selected over 5,000 people from across the country as our sample. The data from our survey showed 7% of our sample, which we selected to represent the whole country, has a disability.

But how confident are we that our estimate of 7% reflects the true prevalence of disability across the Maldives One way to be more certain about study estimates is by looking at a confidence interval. A confidence interval– usually a 95% confidence interval– gives us the range of values we think could realistically contain the true value of our measure. So in this example, we find a 7% prevalence of disability. Let’s say we have a confidence interval of 68% percent. This means that, while we think the best estimate of prevalence of disability in the Maldives is 7%, it is reasonably possible that the prevalence could be anywhere from 6% to 8%.

There’re more details on how to calculate a confidence interval in the see also section. But in brief, we will generally see smaller but more precise confidence intervals with larger sample sizes. Let’s look at an example to better understand confidence intervals. Two studies measured how many children with disabilities were not attending school. Both studies found that 20% of children were not in school. However, study 1 reported a confidence interval of 18% to 22%, while study 2 reported a confidence interval of 10% to 30%. What do these findings mean? Which estimate do you think is more useful for policy and planning? Study 1 has a much smaller confidence interval than study 2.

Study 1 is saying that although in our sample we found 20% of children were not in school, we think it’s reasonable to believe anywhere from 18% to 22% of children are actually not at school. Study 2 says that they think the true proportion of children not in school could be anywhere from 10% to 30%. This is a much wider range, which can make planning difficult since we are less certain of the actual– Next, let’s look at how statistics can help us compare results between groups.

For example, here we see the proportion of people with and without disabilities living in poverty in different countries If we look at, say, South Africa, we see 78% of people with disabilities are living in poverty, compared to 47% of people without disabilities. This is called data disaggregation. This means that we break down our estimates to see how they vary amongst different groups, such as between people with and without disabilities, or men and women, or children and adults. Sustainable development goals call for data disaggregation by disability and other characteristics so that they can track any inequalities between groups in reaching each of the goals. Statistical analysis can help us make more meaningful comparisons between groups.

For example, statistical analysis can help us determine if the differences that we’re seeing between groups are true differences, or what is called statistically significant differences. If we say the difference between two groups is statistically significant, this means we are reasonably confident that the difference we are seeing is not due to chance. Let’s look at an example. Let’s say we are looking at primary school completion between adults with and without disabilities. We find that 88% of adults without disabilities completed primary school, compared to 80% of adults with disabilities. Is this enough information to convince you that primary school completion is different between people with or without disabilities? Let’s think back to earlier when we talked about confidence intervals.

Remember, we said that we use confidence intervals to show the range of values we think could realistically contain the true value of our measure– so in this case, the true proportion of people with and without disabilities who completed primary school in the study area. Let’s say the confidence interval is 75% to 85% for people with disabilities and 83% to 93% for people without disabilities. We can see that there is an overlap in these intervals. This means that there is a possibility that there is actually no difference in primary school completion between people with and without disabilities. It’s possible that both groups could have primary school completion rates of 83%, 84%, or 85%.

We, therefore, can’t say that the difference we observe between 88% and 80% is statistically significant. You may see an indicator called a p-value, which also indicates statistical significance. It shows how likely the relationship you observe is due to chance. P-values can take values from zero to one. The lower the p-value, the less likely it is that the relationship is due to chance. A p-value that is less than or equal to 0.05, or 5%, is usually used to mean your result is statistically significant. This means that there is a 5% chance or less that your findings are due to chance.

When looking at differences between groups, a p-value of less than or equal to 0.05, also means that your 95% confidence intervals do not overlap. The lower the p-value, the more confident we are that there is– Let’s look at a few examples to test your knowledge of statistical significance. In example 1, 90% of children with disabilities were enrolled in school, compared to 98% of children without disabilities. We find a p-value of 0.48. Is this statistically significant? No, here, the p-value is much greater than 0.05. If we had 95% confidence intervals around these two estimates, there would be a lot of overlap. Let’s go to example 2.

70%, 95% confidence interval, 69% to 71% of adults with disabilities are working, compared to 75%, 95% confidence interval, 74% to 76% of adults without disabilities. Is this statistically significant? Yes, there is no overlap between the confidence intervals. The p-value will be less than 0.05. Finally, in example 3, 10% of people with disabilities were living in poverty, compared to 8% of people without disabilities. P-value is equal to 0.02. Is this statistically significant? Yes, here, the p-value is less than 0.05. You may be asking, why are some results not statistically significant? This might be particularly confusing when you see there’s a big difference in estimates between groups.

One very common reason for not having statistically significant results is the size of your sample. Having a result that isn’t statistically significant doesn’t necessarily mean that there really is no difference between the groups. But it can mean that you just don’t have enough data to detect a difference of a certain size. For example, if we go back to these figures on primary school completion between people with and without disabilities, the confidence interval is plus or minus 5% of the estimate for each group. We can, therefore, only detect differences of more than 10 percentage points. If the difference is 10 percentage points or less, we won’t be able to comment on if it’s the real difference.

It is, therefore, important to think through the types of analysis you want to do at the beginning of planning a research study so you can calculate your sample sizes accordingly. Second, it’s possible that other factors explain the difference we’re seeing. For example, if we look at school completion again, some characteristics such as age and gender might be associated with both school attainment and with having a disability. For example, we know people with disabilities are more likely to be older and female. We also know that women and older adults have lower school attainment than men and younger adults in many settings.

If our sample of people with disabilities has more older adults and more women than the sample of people without disabilities, gender and age might be the real reasons why we’re seeing a difference between groups. We can use statistical approaches such as regression analysis to take into account these other characteristics so that we focus only on the difference that is due to disability. This is called adjusting for confounding, and we’ll talk more about this in a later step. In summary, you should now understand how to present data from quantitative studies, explain the importance of statistical significance and data to disaggregation, and interpret common analyses of quantitative studies.