Skip to 0 minutes and 9 seconds Gemma, what’s your experience of using effect sizes and looking at research? I mean, to be honest, in schools I hadn’t come across effect sizes until fairly recently. It was only when John Hattie’s research became quite well-known to teachers that I think we’ve started to talk about effect sizes more in schools, but it’s, you know, not always a hundred percent clear what a large effect size actually is.
Skip to 0 minutes and 32 seconds I mean, at university I’m studying psychology and in that context we’re taught that 0.2 is kind of a fairly small effect, 0.5 is a moderate effect and then 0.8 and above for a large effect but whether or not that applies directly to educational research, I’m not sure because it seems like in a lot of educational research you never get an effect as high as kind of 0.8 or above. Yeah, so the the EEF tend to describe effect sizes on pupil attainment in terms of extra months of progress to try and make it relevant to, you know, the education world. So they say that an effect size of 0.14 is equivalent to two months
Skip to 1 minute and 15 seconds additional pupil progress: what pupils would have learned in two months anyway. 0.23 is equal to three months of additional progress and 0.31 is equal to 4 months additional progress. I think having those kind of scales and measures are useful when you’re comparing different interventions as well. So you can use the effect sizes to look at a range of different interventions or strategies that you might want to put in place and kind of rank them in order of their effect sizes, although obviously it’s not just the effect size that you need to take into account, it’s also about how easily you’re going to be able to implement that strategy or that intervention and what the costs are going to be, both in terms of money but also teachers’ time and effort.
Skip to 2 minutes and 0 seconds So sometimes, you know, you might see a study and there’s an intervention with a very high effect size but it would be very costly to put into place or it might be logistically very difficult to put into place and so we might end up going for something that’s got a slightly lower effect size but can be, you know, very readily implemented or rolled out across the whole school. Yeah, for sure. I think it’s important that…
Skip to 2 minutes and 23 seconds Effect sizes are important but we shouldn’t fixate on them so, you know, the effect size has to be interpreted alongside statistical significance so how confident are we in the findings of this study and also just whether it comes from a good study to begin with. You know, you may find an intervention which has a huge effects claim for it but actually if the research it comes from isn’t well designed then we would want to question how useful that information is.
Exploring the use of effect sizes in education research
In this video, we hear from Gemma Goldenberg and Dr Sam Sims as they discuss the use of effect sizes in education research.
Key learning points:
- Effect sizes have only relatively recently emerged in research engaged with in schools due to work from John Hattie, the Education Endowment Foundation and others
- The different ways in which significance is measured can make it challenging to engage with and understand effect sizes
- Implementation and its associated challenges can make it hard for schools to put into place interventions that are seen to have a high effect size
- The effect size should be considered alongside our other judgements of the reliability, trustworthiness and usefulness of the research study
Once you’re ready, click ‘Mark as complete’ and then click ‘Meta-analysis: a useful approach to synthesis?’ to move on.
© Chartered College of Teaching