Skip main navigation

Comparative judgement

In this written case study, learn how technology can be used to support more efficient assessment using comparative judgement.

This case study appears as part of our report entitled, ‘Testing the Water’. Permission has kindly been granted by LKMCo and Daisy Christodoulou to include the case study as part of this course. You can read the report in full with other case studies here

Improving the reliability of teachers’ judgements – No More Marking and comparative judgement

No More Marking is an organisation developing comparative judgement. Comparative judgement is based on the principle that people are better at making comparisons between pieces of work than at making absolute judgements about quality. No More Marking has developed software that enables teachers to compare two pupils’ work (for example, two essays exploring the question: ‘What were the causes of the Second World War?’). Each teacher simply judges which response he or she thinks best addresses the question. Once enough teachers have judged enough pieces of work, it is possible to see where an individual pupil’s work fits into the broader distribution of achievement, and standardise pupils’ achievements to see how they compare with other pupils, nationally.

The government said it is going to explore the potential for comparative judgement in supporting assessment in primary schools in its assessment consultation response, published in September 2017.

Daisy Christodoulou is No More Marking’s Director of Education, and said she feels the approach has three key benefits:

1. Reliability: Pilots of the comparative judgement approach suggest it is more reliable than methods traditionally used to assess pupils’ written work.

2. Validity: The assessment of written work using marking criteria has historically proven problematic, leading to pupils jumping through hoops (for example, using clunky or nonsensical frontal adverbial phrases). Comparative judgement involves making a holistic evaluation of a piece of work. In Christodoulou’s words, “you’re actually assessing the thing you want to assess.”

3. Efficiency: Comparative judgement enables markers to assess pupils’ written work more quickly. Christodoulou explained that the average time it takes users of comparative judgement to make a judgement is 38 seconds, far quicker than the time taken to grade an essay using marking criteria.

Image taken from the No More Marking website of pupils’ work, as displayed to markers on screen: image of marking

Christodoulou also explained that, while in its relative infancy, comparative judgement could be used for peer assessment by pupils too. The approach enables evaluation of the sorts of features judges identify and reward when assessing work, “so if you set up pupils as judges you can see to what extent they’re agreeing with the teachers.”

While the evidence supporting comparative judgement is encouraging, the approach represents an entirely different way of evaluating pupils’ work: “If you’re used to marking with criteria and having those moderation meetings it’s really different and it can feel very subjective. The weird thing is it’s much more rigorous than traditional marking but it doesn’t feel it.” Other challenges are practical, such as inputting pupils’ work before conducting the assessment.

If you choose to focus on this case study as part of this week’s learning, you can share any initial reflections and questions with the course community in the comments space below.
  • How might any of these approaches be applied in your own context to solve a challenge you’ve identified?
When you are ready, click the ‘Mark as complete’ button below and then select ‘Discuss your learning’ to connect with other course participants.
This article is from the free online

Using Technology in Evidence-Based Teaching and Learning

Created by
FutureLearn - Learning For Life

Our purpose is to transform access to education.

We offer a diverse selection of courses from leading universities and cultural institutions from around the world. These are delivered one step at a time, and are accessible on mobile, tablet and desktop, so you can fit learning around your life.

We believe learning should be an enjoyable, social experience, so our courses offer the opportunity to discuss what you’re learning with others as you go, helping you make fresh discoveries and form new ideas.
You can unlock new opportunities with unlimited access to hundreds of online short courses for a year by subscribing to our Unlimited package. Build your knowledge with top universities and organisations.

Learn more about how FutureLearn is transforming access to education