What’s the story about completion rates?
FutureLearn CEO, Simon Nelson, tackles the frequently asked question about how we compare to other online learning platforms.
If ever there was a tricky area in the delivery of online learning at scale, it’s the analysis of meaningful data from learning platforms. Despite the rapid growth of MOOC platforms around the world, the fact remains that massive open online learning is still a fairly nascent area, and as such, one thing it lacks is common benchmarks for measuring success.
Selecting the right metrics
A starting point for us at FutureLearn is to focus on the numbers that give us an indication of how we’re performing against our objective to create a product that supports learning through conversation. As a social learning platform, learner interaction is one of our most important metrics, alongside others that we’ve outlined in the past.
In selecting our data pool, we’ve also found it more meaningful to focus on ‘learners’ who showed up to a course, rather than ‘joiners’ who initially expressed an interest in the course when it was advertised. A percentage of these joiners find that for a number of reasons they end up unable to commit the time to begin the course.
Measuring participation
Among the metrics we obsess about are rates of ‘full participation’, which we define as completing the majority of steps in a course, as well as all the assessments. We do this so we can assess the quality of the whole learning experience, and make sure learners are taking away the knowledge that is important to them. For us it’s not enough that they’re simply visiting the course in the final week, or have just done the assessments. At FutureLearn, only fully participating learners qualify for our Statements of Participation.
On average, 22% of our learners are fully participating, which tells us that they’ve been motivated to engage in the full learning experience. That also speaks to the effectiveness of the storytelling techniques that we build into courses to compel learners through to the end. Even if we looked at the number in comparison to everyone who signed up to the course, that number still comes in at 12%.
Comparing to other MOOC providers
Compared to other MOOC providers who’ve shared data, that’s still a good indication that our learners are enjoying their time on FutureLearn. This report from MIT and Harvard points to completion rates of 8% among the equivalent of learners on the courses they analysed, as well as 5% of all joiners of the courses. Another references 5% of joiners officially completing a course. These numbers are representative of what many others have reported as being average completion rates for MOOCs.
Of course the caveat to this discussion is that different providers select metrics that are relevant to them – each product is different, and so it’s impossible to make a direct, like-for-like comparison between them. However, we believe that when benchmarking our rates of full participation against how others measure ‘completion’, we stand out as having highly engaged learners who are enjoying their experience.
Full participation is definitely a number we care about. But is it the only measure we look at? Certainly not. Our aim all along has been to create a product that gives people a high quality learning experience by provoking conversation, telling stories and celebrating progress. And so we measure and celebrate the different ways that learners engage in our courses. Our high levels of social engagement are also something that we’re incredibly proud of. It’s because of these measures that we feel confident about claiming that we produce the best free online courses in the world.
We’ll keep sharing our data on a regular basis so we can continue to make a meaningful contribution to the ongoing development of MOOCs and our general understanding of how people use them.
