Skip main navigation

Exploring the use of effect sizes in education research

Hear from Gemma Goldenberg and Dr Sam Sims as they discuss the use of effect sizes in education research
Gemma, what’s your experience of using effect sizes and looking at research? I mean, to be honest, in schools I hadn’t come across effect sizes until fairly recently. It was only when John Hattie’s research became quite well-known to teachers that I think we’ve started to talk about effect sizes more in schools, but it’s, you know, not always a hundred percent clear what a large effect size actually is.
I mean, at university I’m studying psychology and in that context we’re taught that 0.2 is kind of a fairly small effect, 0.5 is a moderate effect and then 0.8 and above for a large effect but whether or not that applies directly to educational research, I’m not sure because it seems like in a lot of educational research you never get an effect as high as kind of 0.8 or above. Yeah, so the the EEF tend to describe effect sizes on pupil attainment in terms of extra months of progress to try and make it relevant to, you know, the education world. So they say that an effect size of 0.14 is equivalent to two months
additional pupil progress: what pupils would have learned in two months anyway. 0.23 is equal to three months of additional progress and 0.31 is equal to 4 months additional progress. I think having those kind of scales and measures are useful when you’re comparing different interventions as well. So you can use the effect sizes to look at a range of different interventions or strategies that you might want to put in place and kind of rank them in order of their effect sizes, although obviously it’s not just the effect size that you need to take into account, it’s also about how easily you’re going to be able to implement that strategy or that intervention and what the costs are going to be, both in terms of money but also teachers’ time and effort.
So sometimes, you know, you might see a study and there’s an intervention with a very high effect size but it would be very costly to put into place or it might be logistically very difficult to put into place and so we might end up going for something that’s got a slightly lower effect size but can be, you know, very readily implemented or rolled out across the whole school. Yeah, for sure. I think it’s important that…
Effect sizes are important but we shouldn’t fixate on them so, you know, the effect size has to be interpreted alongside statistical significance so how confident are we in the findings of this study and also just whether it comes from a good study to begin with. You know, you may find an intervention which has a huge effects claim for it but actually if the research it comes from isn’t well designed then we would want to question how useful that information is.

In this video, we hear from Gemma Goldenberg and Dr Sam Sims as they discuss the use of effect sizes in education research.

Key learning points:

  • Effect sizes have only relatively recently emerged in research engaged with in schools due to work from John Hattie, the Education Endowment Foundation and others
  • The different ways in which significance is measured can make it challenging to engage with and understand effect sizes
  • Implementation and its associated challenges can make it hard for schools to put into place interventions that are seen to have a high effect size
  • The effect size should be considered alongside our other judgements of the reliability, trustworthiness and usefulness of the research study
Once you’re ready, click ‘Mark as complete’ and then click ‘Meta-analysis: a useful approach to synthesis?’ to move on.
This article is from the free online

Education Research That Matters: Ways of Researching

Created by
FutureLearn - Learning For Life

Our purpose is to transform access to education.

We offer a diverse selection of courses from leading universities and cultural institutions from around the world. These are delivered one step at a time, and are accessible on mobile, tablet and desktop, so you can fit learning around your life.

We believe learning should be an enjoyable, social experience, so our courses offer the opportunity to discuss what you’re learning with others as you go, helping you make fresh discoveries and form new ideas.
You can unlock new opportunities with unlimited access to hundreds of online short courses for a year by subscribing to our Unlimited package. Build your knowledge with top universities and organisations.

Learn more about how FutureLearn is transforming access to education