Skip to 0 minutes and 1 secondMATT CORNOCK: Hello, and welcome to our question and answer session for Planning for Learning. We're joined today by expert educators Dillon William and Chris Harrison. I'm Matt Cornock, the Online Learning CBT Coordinator at the National STEM Learning Centre. And I'd just like to say thank you very much for all your contributions and your participation in the online course this time round. Our questions have been posted on the Q&A step from our learners, and we've collated those and sent them on to Chris and Dillon. And now we'll hear their responses. So the first question comes from Maria, and it's on the topic of planned as opposed to spontaneous responsiveness.
Skip to 0 minutes and 39 secondsHer question is I think planning learning would kill all flexibility and responsiveness within the lesson. Of course, she should be ready and can predict or make assumptions based on learner's performance. However, the spontaneous elements will appear in a lesson without planning them. She'd like to ask if you agree and how to incorporate them into planning if it is possible. So we'll go to Chris, first of all, to answer Maria's question.
Skip to 1 minute and 2 secondsCHRIS HARRISON: OK. Thank you, Maria, for a really interesting question and for raising this point. One of the things that we found, having worked with teachers now for many years in assessment for learning is that the important part of the actual process is the responsiveness, the formative action. It's the way that teachers come to notice things in the classroom and then to take that action through that noticing to try and move learners forward. And it's clear that you actually are doing this, that that spontaneous in the moment thing is what we're looking for.
Skip to 1 minute and 36 secondsI think what you can do in terms of planning is to actually go back through and think through in a lesson that's, say, gone well, where you know you've responded well to students, to just go back through and think about what those interactions actually took place and to then think, how are they actually being framed? One of the things in a paper I wrote recently with Bronwen Cowie from New Zealand and Joe Willis from Australia was about bringing together ideas from teacher professional noticing and assessment for learning. and what we found when we actually looked at this is there was a lot of similarities in the two sets of literature going back to Mason's work of 2002.
Skip to 2 minutes and 23 secondsAnd certainly sort of you may be connecting with your curriculum through those interactions to that responsiveness. You may be connecting with the student's own ideas and community so bringing points of knowledge of the students to the fore. Or you may be connecting with the collaborative way that they're working with others and making sense of what others are saying in the classroom. All three of those frames are ones that you can actually focus on at different times. And that will actually help not just you in terms of you finding out what's inside students heads but also the students and helping them build and construct their ideas through the interactions that happened in the classroom.
Skip to 3 minutes and 5 secondsSo you know, I would just say go for it. Just maybe be reflective on what you're doing, what's in those particular in the moment spontaneous parts of AFL, and then to afterwards think about, so, how have I been responding, what have I been responding to, and is that the best thing to do in similar situations just so that you can refine it as you're going on.
Skip to 3 minutes and 28 secondsMATT CORNOCK: Thanks. Chris. Dylan, anything you'd like to add onto that?
Skip to 3 minutes and 31 secondsDYLAN WILIAM: I think the research on expertise shows that when experts seem to be acting very fluently, what they're often doing is remembering similar situations before and piecing together scripts that they've picked up in their previous practise and assembly them in new ways. So you know, a jazz trumpeter isn't play things completely originally. It's maybe making things that combining things in unusual ways. And of course, the more of these scripts that a teacher has, the more likely they are to behave in a way that actually gets the learning moving forward. So I think that the idea how you set pieces started with hinge questions, for example helps a teacher get a kind of repertoire of these moves.
Skip to 4 minutes and 15 secondsAnd ultimately, we are trying to move towards a practise where every single moment in a classroom is one that's potentially a branching point for a set of actions. And there's no doubt that in our work on formative assessment, this idea of a moment of contingency, a moment where you need to make a decision about what could happen next, is a very powerful focus for professional development. And I think the other thing that we've seen is that this is built on the work of Jim Hiebert at the University of Delaware.
Skip to 4 minutes and 49 secondsExpertise in teaching seems to involve being able to be really perceptive about whether something that a student says is something that can be used to advance the whole class's learning or is just a kind of blind alley that the teacher should really ignore. And we see novice teachers either dealing with a student's question and going down the rabbit hole or completely ignoring it. And I think the important point in terms of expertise and teaching is how does this take the learning of the whole class forward. If it does, we can use this, and we can actually extemporise an approach to taking the discussion forward. If it doesn't help the whole class's learning, then we just park it.
Skip to 5 minutes and 34 secondsAnd I'll say to the student, you know, that's a very interesting question, but that's not relevant to what we're talking about here. And I'll talk to you about it afterwards. So I think it's becoming much more clear about the purpose of the lesson, the goal that you have in mind, and then whether the contributions of the students are helpful in that respect. And I think that as you get more experience, you have more of these scripts to play with. And therefore, you're more likely to make decisions that reflect the learning needs of the students. And it's a continuous development process, you know? The more you do this, the more likely you are to make smart choices as a teacher.
Skip to 6 minutes and 12 secondsBut you never stop learning.
Skip to 6 minutes and 15 secondsMATT CORNOCK: Thank you very much, both of you, for that. That idea of being spontaneous, drawing upon your repertoire, and building up that repertoire, but planning those moments of decision. Very much picking up on your points you made there, Dylan, about looking about what the intentions, the objectives, of the lesson are. Some of the course discussions have focused on what do we actually mean by planning for using success criteria, whether that's students being involved in the process of their own learning, generating the success criteria with the teacher, and how it differs from giving students an idea of what a good one looks like, what a good outcome from the lesson looks like.
Skip to 6 minutes and 55 secondsHow do we align those two ideas, and how should teachers balance what to expect with generating success criteria together? Who'd like to go first on that one?
Skip to 7 minutes and 6 secondsDYLAN WILIAM: I'll start. I've changed my mind quite a lot about this. I've now become very clear that teaching should be an intentional activity. Teachers should have a clear goal in mind when they walk into a classroom about the kinds of things that students will develop in their thinking as a result of the time together in the classroom. So I think the teacher needs to have some clear learning intentions in mind. I'm now much more relaxed about whether it's a good idea to share those with students at the outset. Sometimes it is, sometimes it isn't. I am very concerned, however, about the idea of negotiating the learning intentions or the success criteria with students. The teacher is the subject experts.
Skip to 7 minutes and 42 secondsThe students are not. And therefore, this process that is sometimes called co-construction, where the teacher discusses what does a good one look like well, let's look at some examples of this painting, this lab report, and discuss what's good about the good ones is fine as a pedagogical device to bring the students in. But the negotiations have to be rigged. The point is that the discussion has to end up where the teacher is happy with because the teacher is accountable to the discipline of science or mathematics that they represent when they're in the classroom. So for me, the learning intentions are about the long term learning.
Skip to 8 minutes and 17 secondsAnd because learning is a change in long term memory, we can't judge the learning in the lesson. So what we need are ways of judging has this activity been completed successfully enough to be likely that the intended learning takes place over the next two weeks or whatever. And that's why I think success criteria are useful as descriptions of the performance in the lesson that the teacher will accept. So learning is a change in long term memory. The learning intentions are your desires for learning. The success criteria are the ‘I’ll be happy if’. And so there are at least four ways in which success criteria can be used.
Skip to 8 minutes and 56 secondsOne is I'll be happy if they get two thirds of these, the names of the bones of the human body. I'll be happy if they get the major limb bones, the rib cage, the clavicle, and scapula. So the idea is sometimes, success criteria can say how far through the task the students need to get. The second is a scaffolding. Here's the steps you need to go through. The danger with scaffolding, of course, is that students just follow the instructions and aren't actually able, then, to do it without the scaffolding. The third is a description of what a good one looks like rubrics or success criteria that describe successful performance.
Skip to 9 minutes and 35 secondsAnd again, I think we rush to those too quickly, and we'd be better off talking about examples so that then if you do extract some principles from those examples, they're grounded in actual artefacts of quality rather than being vague descriptions that don't mean anything to the students. And the fourth reason, I think, that we might want to use success criteria is as practise in applying something to a different context. So the work of Graham Nutall showed that when students apply something in just one context, that learning is not usually available in other contexts.
Skip to 10 minutes and 6 secondsBut if you get a student systematically to apply the same idea in three or four different contexts, then that learning seems to be relatively detached from the context of the learning and is available in other settings. So I think those are at least four reasons for having success criteria. But the important point, is success criteria are descriptions of performance because we can't judge learning in real time.
Skip to 10 minutes and 30 secondsMATT CORNOCK: Thank you, Dylan. Chris, something to add onto that?
Skip to 10 minutes and 34 secondsCHRIS HARRISON: Yeah, well, I mean, I’m very similar thinking to Dylan in that success criteria can be very useful. Learning intentions can be very useful but more for teacher thinking than for really student thinking. And that's because in assessment for learning, you're trying to close the gap. You're trying to move them forward. And the student can't yet visualise what that point B they're moving towards looks like. They only got some idea about it. And whatever you show them is not going to really get that thought in their heads until they actually are almost there or when they're there.
Skip to 11 minutes and 9 secondsSo I think in terms of teachers actually planning out what the learning is going to be about and where they're going to take students, Yeah, learning intentions are important. So I'd be much happier with teacher spending time on that than worrying too much about sharing everything with their students because what tends to happen sometimes is that as a lesson takes place, the teacher might find that what they intended students to be able to do isn't going to work in that lesson. Either there's something that they need to do, some skill they need to develop because students can actually move forward, or maybe the students have already got that and need to take a bigger jump forward.
Skip to 11 minutes and 48 secondsAnd so that flexibility for the teacher, if learning intentions are something that they've actually made a big point about with their students, no longer are useful for taking things forward. So I think that they're OK in terms of actually understanding what does a good performance look like at various times so to help students get more into and understanding of assessments and what you're heading for. But again, like Dylan, I worry about rubrics, that they give sometimes quite a superficial account of what the actually performance is about. And it's hard to write the rubrics, and we're not particularly good in the UK especially. It's not an approach that we've taken.
Skip to 12 minutes and 31 secondsIt's much more an American approach to looking at assessment than we've taken here in the UK. But I think the other thing to bear in mind when we're doing this is that as you move along and start to find out more about your students, you may want to reposition those learning criteria for yourself and then for your students as a teacher. And so that flexibility is much easier if you've not made too much about it. Our problem is that many of the people who took assessment for learning forward in the early days after we'd done the initial research made a lot of using learning intentions success criteria, particularly in primary elementary schools, is very much a practise that's there.
Skip to 13 minutes and 21 secondsSo getting those teachers to maybe loosen up on that I think could be difficult because they actually are quite wedded to that particular approach. And maybe what we need to do is to say to teachers what you should be doing now in terms of your assessment for learning practise is look at where it's at, where it was maybe a year ago or the beginning of the year, and what can I actually do to take it forward because you will change as your learners get better learners through this approach. So you need to change the way that you do your assessment for learning practises.
Skip to 13 minutes and 54 secondsMATT CORNOCK: Thank you very much, both of you. So just to clarify in my head, so in terms of generating success criteria with the teacher, the success criteria can be co-generated, but the learning intentions must not be co-generated.
Skip to 14 minutes and 8 secondsCHRIS HARRISON: I think it depends.
Skip to 14 minutes and 9 secondsMATT CORNOCK: It depends.
Skip to 14 minutes and 11 secondsCHRIS HARRISON: What they're up to.
Skip to 14 minutes and 12 secondsDYLAN WILIAM: I think the really important point is that success criteria are ways of judging whether an activity has been completed successfully. Sometimes, it's helpful to share that with kids. Sometimes, it's not. But the important point is you can only judge the success of a learning activity at some point in the future when you see that long term learning has taken place. And as I said, I'm much less convinced than I was, partly because I've seen what people have done with this idea, of starting every single lesson with the success criterion written by the teach on the board copied by the students into that books.
Skip to 14 minutes and 49 secondsI think it I think probably does help the students to know when they've satisfactorily completed an activity. But I think that that should not be taken as evidence of learning. It is just evidence of a successful completion of a learning task.
Skip to 15 minutes and 5 secondsCHRIS HARRISON: There is one more thing I want to add which has just sort of come into my head, and that's a practise that I've seen used a lot in schools where they try and get differentiated success criteria. So they have all must be able to do this. Most should be able to to do this, and some will be able to do this particular thing. For me, and certainly I know when Brian Cartwright was actually offsetting OFSTED Inspector for Science, he also said this. It was that, you know, really, we should be trying to get all children as far as they can actually get with these ideas. You shouldn't be limiting them by saying, you know, all can do this.
Skip to 15 minutes and 46 secondsSome should do that. Most should do that, some should do the other. You should all be working to what all can actually do, and it's finding ways of actually moving them forward. So sometimes, it's much of a straitjacket to what's actually going on because you actually accept quite low level performance because you want all kids to be able to do that when in actual fact, it's probably something that they're way past in terms of their learning. It's not at that edge of learning where we need to take them to move them forward.
Skip to 16 minutes and 17 secondsDYLAN WILIAM: I share that concern, but I don't think it's differentiated success criteria that are the problem. If you want all kids to do something, then all should be the minimum standard.
Skip to 16 minutes and 27 secondsCHRIS HARRISON: Yeah.
Skip to 16 minutes and 28 secondsDYLAN WILIAM: I think that what Brian Cartwright says is fine in principle. But the fact is, some children learn faster than others, and some children will take themselves further than others. And I think we have to think seriously about how to stretch them. So I think I'm OK with differentiated success criteria provided the teacher is very aware of the possibilities that students might be just phoning stuff in. And somebody posted a tweet on Twitter some months ago where a child said to his friend, I'm feeling a bit tired today. I think I'll just be ‘most’ because the teacher had given them some, most, all success criteria.
Skip to 17 minutes and 5 secondsAnd the child was deliberately avoiding the challenge by just doing the task in a lazy way. And so I think we have to be aware of that tendency of students sometimes to game the system. But I think it's naive to expect that all students will have the same level of achievement. And if we're not expecting more from the high achieving students, then I worry about that a bit. It's a difficult balancing act. I'm much more concerned about the bottom end of the achievement range than the top end. So I'm much more concerned about making sure that all students reach a certain standard.
Skip to 17 minutes and 34 secondsBut I'm also sensitive to the idea that we should be trying to stretch our highest achieving students at the same time. And I think differentiated success criteria used sensibly are a way of doing that.
Skip to 17 minutes and 45 secondsCHRIS HARRISON: Well I think teaches use it in a limiting way because I think what we do
Skip to 17 minutes and 49 secondsDYLAN WILIAM: Fine. Well that's bad.
Skip to 17 minutes and 51 secondsCHRIS HARRISON: [INAUDIBLE] yeah, so the same sentiments as you, Dylan, but it's actually the way I see teachers using them doesn't actually achieve that. Because they're so focused on the criteria, they aren't flexible in what they're doing with the learnings that's going forward.
Skip to 18 minutes and 5 secondsMATT CORNOCK: Thank you very much, Chris, Dillon, for your response to that. Like I said, it's a theme that came out of the discussions from both weeks one and two. And I think that will be really valuable to help the learners on that. We'll move on now to the next set of questions, which is all about questions about evidence and using that evidence. And I think Patrick is asking which was about the focus of achievement and scores and the problem of unwedding students from their schools, particularly thinking about the context of the student but also the teacher and school systems as well and how much scores and achievement is focused on.
Skip to 18 minutes and 43 secondsWhat's the latest consensus in terms of peer reviewed publications on this and, of course, your own thoughts?
Skip to 18 minutes and 48 secondsDYLAN WILIAM: I think the important point is to go back to the review by Kluger and DeNisi in 1996, who said, let's stop worrying about what the effect size of feedback is and instead start worrying about what it does to students. So I say not only to teachers if your feedback is getting you more of what you want form your students, it's good feedback. And if it's getting you less of what you want from your students, it's bad feedback, and that's it. There's nothing more to say about feedback. And so this idea that grades are bad and that comments are good, well, some comments aren't very helpful.
Skip to 19 minutes and 17 secondsAnd if you've got a student who needs a B at A level to get into the University of their choice and telling them, at the rate you're going, you're going to get a D might be very helpful feedback. So I think the important point is we have to accept seriously the research of people like Ruth Butler who found that when you give kids both comments and grades, the children look at the grades first, and then they look at their neighbour's grade. So there's obviously a problem there. But I'm open to the possibility that in certain contexts, grades can be motivating. So I think we ought to be much more flexible.
Skip to 19 minutes and 50 secondsThere are lots of people telling teachers that feedback should be descriptive rather than evaluative, should be immediate rather than delayed, should be specific rather than generated. And none of those is supported by the research evidence. Feedback can be delayed and be much more effective than the immediate feedback because it acts as a kind of delayed restudy. Evaluative feedback can be helpful if you have a certain goal in mind. If feedback is too specific to the task at hand, it's never going to be useful to the student in doing a different task. So there has to be a degree of genericness built in.
Skip to 20 minutes and 20 secondsAnd so I come down to this idea: the thing that matters with feedback is what the students do with it. And Kluger and DeNisi lay out a very helpful framework. You know, they can change the behaviour. They can change the goal. They can abandon the goal, or they can reject the feedback. And you know, some of those are good sometimes. Some of those are bad sometimes. And so the important point is just look at what your students are doing. If it's getting you more of what you want, then carry on doing it. If it's getting you less of what you want, stop doing it.
Skip to 20 minutes and 48 secondsMATT CORNOCK: Chris?
Skip to 20 minutes and 48 secondsCHRIS HARRISON: Yeah, I mean, I'd like Kluger and DeNisi review as well and how they actually approach it. I think one of the problems is that people are often interpreting the research studies as you do one thing or the other. What it's saying is that there are times when you maybe should be focusing on more descriptive feedback because that's actually probably going to be taking more notice of. And it's detailed enough and it's actually appropriate, then it'll maybe help the student in their learning. There's times when you need to focus on grades in schools. And at different times in a child's learning journey, you'll look at one or the other.
Skip to 21 minutes and 31 secondsSo possibly the actual proportion of one compared to the other within, say, the 16-year-olds as they're approaching GCSE or 18-year-olds as they're approaching their A levels in England would be different to what you'd expect from an 11-year-old. And teachers are the best to know how it's actually going to function. I mean, it's the same with things being actually written or oral. People argue about that all the time. And certainly there's not been a lot of studies done looking at the detail of this since way back in the 1990s.
Skip to 22 minutes and 10 secondsSo you know, you could do some more research, but I think at the moment, what we're down to is the teacher has to be in control and decide what's best for them. What they need to do is to focus on looking at is this having the effect that I'm actually hoping for. Am I happy giving more feedback on the way that they're doing their working out in mathematics, is that actually helping them to see where they're making their mistakes, and are they actually therefore doing maths more competently and with more confidence? So yeah, I think it's very much down to the teacher to decide on that.
Skip to 22 minutes and 46 secondsDYLAN WILIAM: I think it's also important to realise that how students react to feedback depends to a very large extent on the relationship between the student and the teacher. First, the teacher needs to know the students, know when to push, and know when to back off. And secondly, the student needs to trust the teacher. They need to trust that the teacher has their best interests at heart and knows what they're talking about. That's why the work of David Yeager is so interesting. He's shown that when students understand why teachers are giving feedback, students are far more likely to use it to improve their work.
Skip to 23 minutes and 16 secondsSo in the research now, people talk about recipience processes in other words, getting students ready to receive feedback. And I think that if a teacher wants to develop that practise of feedback, I think the most productive use of that time will be working with students so that the students understand why they're being given feedback and how to take it on board. We worry about should be feedback be spoken or written. It's irrelevant. The important point is, does the relationship exist that allows a student to actually use the feedback in a productive way?
Skip to 23 minutes and 44 secondsCHRIS HARRISON: And it's the amount as well. Sometimes students get too much feedback
Skip to 23 minutes and 48 secondsDYLAN WILIAM: Yeah, too much, yeah.
Skip to 23 minutes and 49 secondsCHRIS HARRISON: They don't know which to actually follow. What shall I do? I've got all of these things. So they just go, oh, I'll just try harder. I'll revise more general things that are not going to help them that much really.
Skip to 24 minutes and 3 secondsMATT CORNOCK: So the score is not the end of all learning if you just get a score, but it's part of the whole feedback mechanism, and it's all dependent upon context and to make students make the best use of that feedback, there has to be a relationship there. There has to be that formative classroom environment that they can actually have that discussion with the teacher about why they're giving feedback, yeah?
Skip to 24 minutes and 22 secondsCHRIS HARRISON: And it lets students know it's all about improvement. A lot of the feedback is given in a very convergent way. It's almost like just correcting it but done in a different way. teacher might as well be correcting the work by the comments that teachers give. Or sometimes, if they're using techniques like green pen where they're getting students to respond to their feedback, again, if the feedback they've got is just about correcting it and the student puts in the right answer, that's not really feedback. It's correcting.
Skip to 24 minutes and 53 secondsDYLAN WILIAM: Well, I disagree, actually. I mean, it could be. What I'm saying is it might be it might not be helpful, but it might be. Important point is the context is the thing that's really important. And we have to be very careful about giving these kinds of blanket rules because, you know, I've seen lots of teachers mark in red because red stands out. And because those students understand why the teacher is giving you feedback, they like feedback in red because it's easy to find. And that's where I think the work of Carol Dweck is so important. It's been criticised recently for its failure to replicate.
Skip to 25 minutes and 22 secondsBut I don't have any doubt that if a child has a growth mindset, they welcome feedback because feedback is a chance to actually direct your improvement. And if you have a fixed mindset, then feedback is often unwelcome because it shows you that you're not as smart as you thought you were. So I think that for me, Carol Dweck's work on mindset is an important part of these recipience processes. If you can get students to understand that you can get better at this and my feedback can help you get better at this faster than you'd get better at it if you left to your own devices, then students are more likely to use that feedback productively.
Skip to 25 minutes and 57 secondsMATT CORNOCK: There's a really nice flow from that question, then, and your response there to what John's asking about what the evidence is saying about the impacts on correcting misconceptions when you present students with mistakes that do not have accompanying correct options. We have a couple of examples in the course such as a concept map that contains errors and a concept cartoon where the incorrect answers can be compared with the correct one. So question, to Chris, to start off with, then, do students learn better when the correct answer is offered or when they have to construct it for themselves?
Skip to 26 minutes and 33 secondsCHRIS HARRISON: The answer is I don't know because I've not tried one way or the other. But I do know that what I see as a productive way of using concept cartoons is to actually do it in a way where you're asking students to say what do they think. So you present them with this scenario, and it's what do they think. They're not the ones where my guess is, the ones are where there is a correct answer and incorrect answers, then the students start just searching for that correct answer. So they don't really think it through and work out where it is.
Skip to 27 minutes and 6 secondsAnd in the classroom, you know, it's rare, unless you have very closed questions or a very closed approach to interruptions, to get at all questions right or all questions wrong. It's somewhere in between. And just going back to this thing about noticing this contingency is what can be used from the interactions that ensue from using the concept of cartoon or from using the concept map that allows the teacher to then key into where they might take learning forward. So it's looking for that potential within that interaction the teacher should be doing. So I would tend to go for just putting up things that students generally say about that topic.
Skip to 27 minutes and 50 secondsAnd certainly one of the best lessons I ever saw was where a teacher just wrote down everything that the student said about a particular scenario that they were looking at. And they then talked through and gradually in their groups either rephrased what some students had said or took some of the ideas out because they said that these were not used in taking it forward and gradually had a more collaborative approach to sort of deciding what's there. You can't correct misconceptions. You can tell them what most people think, but they may well go back to that tacit knowledge they're holding about electricity in a circuit or whatever it is that they are struggling with.
Skip to 28 minutes and 33 secondsAnd I think one of the things that's sometimes problematic is sometimes it's just that it's not a misconception but an emerging conception that the students have got. They've not yet built up something. And the interaction can actually move things forward much better than just sort of telling them what the scientific view of this is. And then they've got to build that big gap between their emerging conception and what, say, scientists or mathematicians think about what the actual conceptual understanding is. So the interactions are what's important. So whatever will cause those interactions, I think that's what teachers should go for.
Skip to 29 minutes and 10 secondsDYLAN WILIAM: I think the important point for me is that, as George Box said many years ago, all models are wrong, but some are useful. And most of what we're teaching kids in science in particular are models. So we have models that are powerful in some level and therefore useful. But we can make them more sophisticated and more complex. The important point is, a more powerful, more accurate model may not be better because it may be so complex that it's less useful. So there's always a trade-off in our models. I think the important point is, what's the thing at hand right now?
Skip to 29 minutes and 43 secondsAnd I think for me, one set of things that are really useful is to have partially correct as well as incorrect answers. So Mark Wilson uses a very nice example. A ball is resting on a table, and students are asked, why does the ball not move. And one of the options is gravity is holding it onto the table. And that's actually correct. Gravity is pulling the ball down, but the table is in the way. That's correct too.
Skip to 30 minutes and 5 secondsOne of the options is the table pushes up with the same force that gravity pulls the ball down, which is the one the science teacher is looking for because they're trying to test whether the student has actually mastered this idea of no movement signals either a lack of forces or forces in equilibrium. So I think that those kinds of things can be very rich. And I like the idea of multiple correct answers, maybe some at more sophisticated levels than others, as a way of starting a discussion. But I think that in general, our models get richer and more powerful, but the old models never go away.
Skip to 30 minutes and 38 secondsSo as Daniel Kahneman has shown in his book, Thinking Fast and Slow, we all have these intuitive reactions, and then our more thoughtful selves kick in. So when I touch a cold door handle on a cold day, my immediate reaction is, the cold is coming into my hand from the door handle. But as a physicist, I've learned to think about this as heat travelling from my hand to the metal of the door handle. And because metal conducts better than wood, that's why it feels colder. But the important point is that instinctive reaction, as Chris said, really probably never goes away.
Skip to 31 minutes and 15 secondsThere have been some studies of university physicists who show that they often, in their first reaction, think the same way that a 10-year-old would. But then their physics training kicks in, and they think about it in a more sophisticated way. So I think realising that these things are natural, as Chris said, the emerging conceptions is a powerful model for thinking about, how do we move this idea forward? Why is this model more powerful than the model that you like to use intuitively? And in a way, science is about learning these models that enable us to think more powerfully about scientific phenomena. And they're often very unnatural.
Skip to 31 minutes and 53 secondsSo kids aren't going to come up with them on their own, and they need to be taught these things. And for me, it always comes down to a matter of opportunity cost. Kids will probably remember things for longer if they discover them for themselves. But if it takes six weeks to discover it for themselves and I can teach it to them in 10 minutes, then teaching it is a good idea because, and I come down to this issue of equity, some students find learning more difficult than others. And they take longer.
Skip to 32 minutes and 20 secondsSo every time we waste time on kids discovering things for themselves when equally good learning could be produced by the teacher explaining it well, then we actually make the achievement gap wider. And that's what worries me is that if the learning is more sophisticated when they discover it for themselves, then there's a case to be made for that, and we have to look at the trade offs. But if there's no more sophisticated learning, it's not good for the slowest learners to be discovering things from ourselves if they could reach the same understanding with a teacher explaining it to them.
Skip to 32 minutes and 49 secondsMATT CORNOCK: Thank you, Dylan, for that. That's quite a complex idea. I mean. You're making a judgement call there, aren't you, on, forgive me for paraphrasing this which bits are more easily understood, they're the ones that you can teach quickly. And those that require a bit more depth will take longer. That sounds obvious now, doesn't it? So maybe that's the point.
Skip to 33 minutes and 10 secondsDYLAN WILIAM: Well, I'm not sure that's true. I mean, I'm not it may be that these can be explained that even the deepest ideas can be explained more quickly by a really good and gifted teacher. Look at the work of Richard Feynman, for example. I'm just saying that every teacher needs to be aware of the trade offs. And if kids discover it for themselves, it's going to take longer. The question is, is that extra time worth it in terms of the depth of knowledge that they will actually acquire? If it is greater depth of knowledge, it's worth considering. If it's not, then maybe you should just teach it formally.
Skip to 33 minutes and 43 secondsMATT CORNOCK: Thank you very much. Moving on to our next question, John has a second question here. Looking at a spiral curriculum, which was favoured model, whereas now there seems to be more emphasis on mastery, there's often a bit of a context in the UK where mastery is becoming quite mastery model is becoming quite a key concept. But is there actually any evidence that prefers mastery over spiral curricula?
Skip to 34 minutes and 10 secondsDYLAN WILIAM: First, they're not alternatives. And whoever thinks that spiral and mastery are kind of different options is getting things mixed up. So the first thing is the spiral curriculum was proposed by Bruner. And the thing that he imagined was that students would return to ideas at a higher level. And his criteria for including things in the curriculum was very interesting. He said that if it is useful for a child to know this as an adult and not learning it now will actually affect how well they know it as an adult, then you should teach it.
Skip to 34 minutes and 47 secondsBut if you can leave it for a few years without affecting the depth of knowledge that a student would acquire as an adult, then he says it's cluttering up the curriculum. So he actually had a very, very clear view of what a spiral curriculum meant. And it meant not doing things now just in case it might be useful. He was doing things now only if not doing it now will harm that child's future development. So the spirals that Bruner imagined, we're quite long. So you might not come back to something for two or three months. And in a way, we've always done that in science because we revisit the same concepts at different levels of sophistication.
Skip to 35 minutes and 24 secondsThere's not much evidence either way on this. What is becoming clear from the research evidence is the idea of interleaving the idea of distributed practise and doing things in short bursts and coming back to it two or three weeks later. That does seem to be a very powerful tool for organising a curriculum in ways that result in more learning.
Skip to 35 minutes and 45 secondsSo a recent public paper by Doug Aurora and his colleagues showed that when teachers did things in blocks of up to 10 lessons on the same topic, the same teachers tried a different approach where they had one lesson on one topic and then tomorrow a different lesson on a different topic and didn't come back to that first topic for 10 days. Those students who did the interleaving made significantly more progress and scored much better on a delayed test some weeks later. So I think interleaving and breaking things up is one of the strongest research results we have on memory. And yet, everybody hates it because the students hate it because they keep on chopping and changing.
Skip to 36 minutes and 25 secondsAnd they said, well, I used to be good at this, and now I've forgotten it. But it's the fact that you've forgotten it that makes the next piece of study more effective. The other thing is we have mastery learning as a new hot topic in English education. And the problem is that nobody's doing mastery learning. They say they're doing master learning, but they're not doing mastery learning at all. Mastery learning is just basically formative assessment. It was Benjamin Bloom who pointed out 50 years ago now that traditionally, teachers taught stuff. Some kids got it, some kids didn't. The teacher moved on.
Skip to 36 minutes and 57 secondsBecause the idea was that you didn't get it the first time, it probably wasn't your thing, and so it didn't matter. What Bloom suggested was we need to find out what kids are learning, and if they're not learning, we do something about it. That's the foundational idea of formative assessment. Let's find out what kids are learning and make adjustments. The point is that nobody said how slowly should you go and what proportion of the kids should you take with you. And so mastery learning is just formative assessment, but it doesn't tell you what to do, and do you need to wait for all the class to get it, half the class to get it, all but one kid?
Skip to 37 minutes and 31 secondsAll those things are not being addressed. And the problem is that the differences in the rate of learning of students is so great that nobody is really doing proper mastery learning in terms of everybody getting it because you'd still be on chapter one at the end of the school year. John Carroll in the 1960s estimated that the fastest learner in a class of 30 will be learning five times faster than the slowest learner in a class of 30. And so the question is, what do we do with that range? And it's not at all obvious to me. But I think the crucial thing for me is how Bloom summed up his work.
Skip to 38 minutes and 5 secondsHe was pointing out that if you're getting a bell curve of results, you're just basically treating each kid the same. And he said we can regard our educational efforts as a failure to the extent that our results approximate a bell curve. If you're treating all kids the same, because kids have different propensities for learning, you'll get the bell curve. And I think the idea behind mastery learning as defined by Bloom was we have to destroy the bell curve. We actually have to make sure that all students are getting a chance to make the same level of educational achievement, not the same level of progress.
Skip to 38 minutes and 39 secondsAnd so the idea of setting a floor and saying, we're not moving on until everybody gets this, I think is a very powerful idea. It's just that nobody's doing it because it's too unacceptable because you're not then pushing the highest achieving students fast enough through the curriculum. So I think that it's a very old idea. It's a fundamental foundation of formative assessment. But I don't see anybody doing that in the UK.
Skip to 39 minutes and 3 secondsMATT CORNOCK: Thank you, Dylan. Chris, would you like to follow up on that?
Skip to 39 minutes and 7 secondsCHRIS HARRISON: Yeah. I was going to talk about Benjamin Bloom as well, But I think Dylan's done a very good job of that there. So I think just going back to our original KMOFAP project, one of the ideas that they picked up was a sort of mastery learning approach. And that was because at the time, they were unsure about what to do about end of topic tests, particularly the science people, not so much the Maths and English. And so quite a lot of them did move the end of topic tests earlier in the actual sequence. So there might be a six week block, and they'd move it maybe to the end of week four.
Skip to 39 minutes and 43 secondsAnd then depending on how the students did in that test, they then would either do some remediation work with the whole class or some remediation work with some groups. And then other groups could do actually add on extra work. So they were taking a mastery learning approach. And for some of them, they actually in the end decided that the best thing to do was to just select those questions that really were showing could the kids get it or not get it rather than give them a whole hour test.
Skip to 40 minutes and 14 secondsSo they reduced the amount of summative assessment that they were doing and just used those questions that were actually useful in helping decide whether students needed more support with that particular topic, parts of that topic, or whether it was that they could actually move forward. So they already they weren't quite into questions at that time. It was early on in the work of assessment for learning. But they were already making pedagogic decisions about which assessment questions were useful in helping you decide whether the students had understood or not and then taking formative action.
Skip to 40 minutes and 47 secondsDYLAN WILIAM: There's a very nice example of that in Philadelphia. So each year, they took the material they wanted to cover in the year and divided into six six week blocks. And then in each six week block, they decided what is the essential content and what is the desirable content. And at least 20% had to be identified as desirable, not essential because otherwise, teaches would just say that everything's essential.
Skip to 41 minutes and 10 secondsSo if the idea was over the first five weeks, the teacher covered the essential content, and at the end of week five, the kids do a test, and that is the score they get on that module, the teacher then looks at the scores and the contents of the students' responses to the test and decides what do I do in week six. And if the students do do well, the teacher will do the desirable content. But if they do badly, the teacher will go back and revise the essential content. So the nice thing about that test at the end of week five is it functions both summatively and formatively.
Skip to 41 minutes and 45 secondsIt actually gives the students a grade which is reported to parents for how well they covered the stuff that they learned in that first five weeks. But it also then gives the teacher insights about whether the students can afford to move on or they need to go back and get some of the more essential content more solidly understood. That was a very nice example of operationalising that across a whole education system.
Skip to 42 minutes and 6 secondsCHRIS HARRISON: Has that not been written up, Dylan? I don't know the Philadelphia stuff.
Skip to 42 minutes and 10 secondsDYLAN WILIAM: Yes, there's a paper by Leslie Ola. Her name is Ola, which we can actually add to the notes for the Q&A.
Skip to 42 minutes and 20 secondsMATT CORNOCK: Thank you both. Yes, I am making notes on all references that are coming up. So I will add those links onto the comments at the bottom of the Q&A video. Thank you for your responses there. We move on to a new topic now, questions related to pupils learning. And the first question comes from Amanda. Chris, first of all, please, is it good practise to train students to self-assess assessments they do in lessons, or is it more effective to have the tests marked by the teacher and then the student corrects them? Are we disadvantaging students in any way? It also gives me an opportunity to plan for differentiated lessons. What about lower ability students?
Skip to 42 minutes and 58 secondsThere's a lot of questions there on basically how do we involve students in the assessment process in terms of self-assessments and teacher-led marking.
Skip to 43 minutes and 9 secondsCHRIS HARRISON: Well, certainly what we've learned is that we do need to involve students much more in the assessment process than what was happening 20 years ago when Paul and Dylan did the initial review on formative assessment. And I think it just depends, really, on where in the learning cycle this particular assessment comes as to whether it's more useful to be teacher marked or whether it's more useful to be done as a self-assessment. You need to have a mix of both within teaching and learning. I certainly think that marking with students is better.
Skip to 43 minutes and 49 secondsSo even if the teacher is going to mark it, actually, if the teacher marked it within the lesson with students, just choosing maybe one or two questions from what they've actually done where the teacher can actually give feedback on maybe for some of the more difficult questions could be quite useful. And what I've seen work really well within this is a class of 14-year-olds was where the teacher used to give a test. And then the students would self-assess on the first couple of questions because they were relatively easy ones.
Skip to 44 minutes and 23 secondsAnd then the teacher would set them some sort of task to do and would go around and work with particular students that she'd actually was aware would probably have difficulty with this particular topic in the lessons that she'd done previously with them. And she would mark and go through that with them and then use that to have a whole class summary. So sometimes, a mixture of the two, even within the class, can work productively. But students do need to learn to self-assess to some extent. So it's something that you gradually need to build over time.
Skip to 44 minutes and 54 secondsBut they'll only start to do that when the feedback they're getting either from the teacher or from the students is actually useful for them because they start to realise that feedback then can actually help and that when they start then thinking about how they don't compare to others in more of a peer assessment approach to it, it actually then takes their self-assessment skills forward so they become more self-regulated in their learning. It is hard to do that and takes quite a while to develop that understanding.
Skip to 45 minutes and 25 secondsDYLAN WILIAM: I think for me, the question I'm using increasingly in my own practise, whenever anybody asks me how did I do, I just say, well, how well did you think you did? And I'm not doing that to be coy. I'm doing it because if they can give me a good answer to that question, my work here is done. They're able to criticise their own work, and they are, in the psychological jargon, becoming a self-regulating learner. So I think for me, the purpose of self-assessment is not to save the teacher marking time. It is to help the students develop their own critical self appraisal skills.
Skip to 45 minutes and 56 secondsThat said, I'm now also convinced that students have very little insight into their own learning, and they don't know that they know something. I don't think any humans do. until you try to retrieve things from memory, you don't know whether you know something or not. And learning, this process whereby our experiences get translated into long term memories, is very mysterious. I've lost track of the number of times I've planted something in my garden and I haven't bothered to label it because I was absolutely convinced I'd remember what I planted there. Now the fact that I forgot it isn't interesting. What is interesting is how certain I was that I would remember it, and I didn't.
Skip to 46 minutes and 33 secondsSo it seems to be that as humans, we don't have very good access to the processes that lead to experiences becoming long term memories. And I think we have to be very sceptical about students saying I get this because we need evidence. That's why I keep on coming back to this term formative assessment. It's about evidence. And when a student says I get it, what's your evidence for that? And so I think, to use that memorable phrase from Brezhnev Mikhail Gorbachev and Reagan, it's your ’Trust but verify’, you know? So the idea is we should ask students to assess themselves because developing that capability is important.
Skip to 47 minutes and 7 secondsBut the fact that a student says they get it does not mean we should actually trust that judgement.
Skip to 47 minutes and 14 secondsMATT CORNOCK: Thank you both for your response there. That's a really useful way to think about the role of self-assessment and student self-assessing. The next question comes from Michael, and it's on I think we've touched upon this already, in fact. Assessment for learning is great, Michael says, but as a model, how does it lead to embedding knowledge? Dylan.
Skip to 47 minutes and 39 secondsDYLAN WILIAM: I think the first thing to recognise is that formative assessment, as Paul and I and Chris I think prefer to call it, is independent of what you want students to learn. So I don't care whether you think learning is about memorising facts or constructing models or developing schemas you can use in different situations or information processing. Whatever you think learning is, formative assessment is just a recognition that students do not learn what we teach and that good teaching starts where the students are. We’d better find that out and act accordingly. So for me, a formative assessment is just a recognition that we need to find out what students learned.
Skip to 48 minutes and 19 secondsThe embedding of knowledge, if that's what you want, then you do need to test the knowledge that's been embedded. And that's why I think that although it's in America, for example, people talk about formative assessment as being something that happens every 6 to 10 weeks. And it's very easy to be sniffy about those kinds of things. I think we need both medium cycle and short cycle and long cycle formative assessment. So I think we need to be testing or checking students' learning in the lesson.
Skip to 48 minutes and 44 secondsBut I think we should also be checking that at the end of the week and maybe a month later because the idea of constantly getting students to retrieve their knowledge is what increases really robust, long term memories. So for me, there's no conflict here.
Skip to 49 minutes and 1 secondThey're basically just to be absolutely clear, there's what you want students to learn, OK? So you might be Archimedes' principle. Then there's epistemology. What does it mean to know Archimedes' principle? You could state it, but that's not the same as being able to work out what happens when a man in a boat or in a swimming pool throws a 5 kilogramme weight into the water. What happens to the level of the water? It's the same principle, but that's what demanding a much greater level of thought about how to apply that in principle and practise. Then there's what happens when learning takes place.
Skip to 49 minutes and 33 secondsScience educators in the 1960s and '70s pushed very hard the idea that this receptive model of making links between stimulus and response might work for some things, but it didn't explain how students formed misconceptions about what causes the wind, for example. And so people started saying that learning was a constructive process. And it partly is. But it's also partly a receptive process and it's also partly [INAUDIBLE] to societal situations. It's all of those things. So you can have different views about what happens when learning takes place. And the important point even beyond that is none of that tells you how to teach.
Skip to 50 minutes and 9 secondsSo how you get students to know stuff is completely independent of what you want students to learn, what it means to learn, what it means to know. And so for me, formative assessment is just a recognition. You need to find out what your kids learn before you do anything else. That's it.
Skip to 50 minutes and 28 secondsMATT CORNOCK: Thank you, Dylan. Chris, anything you'd like to add onto that?
Skip to 50 minutes and 32 secondsCHRIS HARRISON: No, I think Dylan's that's fine. I mean, I actually because I believe learning in a constructive way is the way that teachers should envisage it in order to get the teaching to work because I work a lot with pre-service teachers, for example. Then I can actually visualise formative assessment working better with more constructivist view of learning than others. But I agree with Dylan in terms of the principle of formative assessment. It is about collective evidence of where they are at in their learning, how that learning's got to, and then making those decisions about where to take students next, to take their learning forward. It's that action that's important. And sometimes the action is not there.
Skip to 51 minutes and 14 secondsPeople just too readily just notice where kids have got to. They don't actually relate it to what's coming next. And that's the most important thing they need to do.
Skip to 51 minutes and 24 secondsDYLAN WILIAM: So really, the most important thing is recognising that the model of learning needs to depend on what is to be learned. So you know, I don't want kids constructing their model of the definition of a Pascal. You know, basically, you just need to learn this. And you need to be able to state it. And if you don't know it, you don't know it, and I need to reinforce it. So I think the important point is you need to be really clear about what kinds of knowing this is and then have appropriate models of what it means to know and then have appropriate models of how to get kids to know.
Skip to 51 minutes and 54 secondsAnd so the idea that there's one model is, I think, self-defeating. The important point is to choose a model of knowing that best suits the task at hand. And size is a really good example of something that has all these things at different levels.
Skip to 52 minutes and 10 secondsMATT CORNOCK: Thank you both. It's a really interesting discussion so far, and it's so interesting to see very similar questions in some respects, but the discussion goes on a completely different angle inn every Q&A we do. And I think this idea of talking about what learning is and how we understand and how formative assessment fits into that and how teaching fits into that, this is really interesting. Thank you. We're moving on to a new topic, which is on questions related to changing practises. So this is about how teachers and their colleagues can take the ideas from the course and the other work that you've done into their own practise. And Irene starts with a question.
Skip to 52 minutes and 47 secondsWhat should she do to adapt new system to change completely? What should I do to base the teaching assessment? So she'd like to break the old ideas down and try and get people into this new teaching method. I say new teaching method. It's quite well-established now, but you get my point.
Skip to 53 minutes and 7 secondsCHRIS HARRISON: Getting teachers to change is not easy. Lots of people have written about that, and it takes time. And lots of people have written about that as well. However, most teachers want to do the best for their students. So if you can either get teachers within the school who are already actually portraying some of that practise and then for others to be able to look into that and see how they're doing it or maybe to bring in a vignette of some sort of practise that's going on so that teachers actually understand what the practise looks like, then what teachers are able to do is to enable, first of all, to understand somewhat what is happening in terms of how is this particular way of working helping children learn.
Skip to 53 minutes and 56 secondsSecondly, what they can do is to start thinking about their own practise and how might they make space within their own practise to bring these ideas in. In other words, how can they change, what they're already doing in a way that means that this new innovation can be brought in? It's not a matter of taking one type of practise out and replacing it with another. It's a matter of melding in those key strategies that are needed to make, in this case, formative assessment work. And there might be some things that mitigate against this that's actually there within the practise that maybe have to go.
Skip to 54 minutes and 30 secondsOr there might be things that go simply be cut from the practise simply because they're not needed in this particular time of change. But maybe they might come back later, or they might disappear altogether. So it's no wonder that it's difficult for teachers to change because they do things in a certain way for quite a long time. And it mostly seems to sort of work for them. And then getting them to sort of say, so is it worth putting in these changes because it's going to be a change for me, and that means I've got to put more time in. And that can be more stressful than what I was doing previously.
Skip to 55 minutes and 9 secondsIt also means changes for the students , and students also don't like to change. So there might be a reaction from the students at first, that They don't actually settle down well and start behaving in the way that the teacher expected when this new innovation came in. So there's lots of challenges in terms of making change. Certainly from the reading I've done in this area on professional learning but also from the direct work that I've done with many groups of teachers now, it's very clear to me that this is a very difficult thing to do as lone teacher. It's something that actually works much better if you're working with like-minded colleagues.
Skip to 55 minutes and 49 secondsI know Dylan will probably talk about it in a minute. He talks a lot about professional learning communities, teacher learning communities. And I've seen them work very well in schools. I think the important thing for me that needs to be maybe looked at longer maybe Dylan might comment on this from the work that he's done. But it's how you actually sustain those working relationships as things move forward. Because initially, you usually get support and you usually get some enthusiasm from teachers who may get an innovation that they believe in. But keeping that up long enough for it to actually change habits and change the ways that classrooms work can sometimes be difficult.
Skip to 56 minutes and 28 secondsAnd so it's that sustainability but also the evolution of that new practise because things are not going to stay the same. You make changes. But then as you make changes and the learners change and the learners get better through the particular innovation, then you need to change again. So it's a forever moving thing for teachers. And some teachers just sort of say, I've got enough on. I don't want to be forever sort of making changes. I want to settle with something that I'm actually confident with now. So it's a lot to work on in terms of bringing innovations into practise.
Skip to 57 minutes and 4 secondsBut the best way to do it, I'm sure, is to do it with colleagues who want similar thanks to you.
Skip to 57 minutes and 12 secondsDYLAN WILIAM: I think the thing that we've learned I mean, this is a paper that Claire Lee and I wrote after the KMOFAP project was we saw five things as being particularly important to the success of efforts to improve formative assessment. The first was choice, giving teachers the choice about what to work on rather than a school-wide directive that you will all work on questioning. The second was flexibility, encouraging teachers to adapt these techniques to make them work in their own classrooms. The third that Chris has already mentioned allowing teachers to take small steps. You're talking about changing habits. You know, if you're a 20 year veteran teacher, you probably asked half a million questions in your classroom.
Skip to 57 minutes and 49 secondsAsking your questions in a different way is going to be really difficult. So the important point is giving people time. The last two things are two sides of the same coin accountability and support. So if I was a head teacher, the first thing I would say is, if you're a teacher at this school, you'll be getting better at something. That's non-negotiable not because you're not good enough but because you can be even better. And every teacher in this school is improving. The second thing is you're going to be getting better at something that helps our students. So that's basically formative assessment. The evidence is stronger for this than it is for anything else.
Skip to 58 minutes and 23 secondsSo if you as a teacher are working on anything else other than formative assessment, the chances are, it's going to have a smaller impact on students. Therefore, you can do that on your own time. But your contract time should be focused on formative assessment. And the leaders have to get behind this by, first of all, creating this atmosphere where everybody is getting better. I think it should be the most natural thing in the world for a head teacher to say to a teacher, what are you working on right now? What are you working on getting better at? In the corridors, ask them. And then the teacher would have a clear answer about that.
Skip to 58 minutes and 56 secondsThe second job of the leader is to keep the focus on formative assessment because it's very easy to get deflected from that. The third and I think the most difficult thing we found is giving teachers time. Whenever we talk about formative assessment, people say to me, we haven't got time to do this. And my answer is, yes, you do. But you're currently spending it on something less valuable. And this is the real challenge for leaders. The essence of effective leadership in schools is stopping people doing good things to give them time to do even better things. And Chris has talked about, for example, homework. You know, teachers are slaves to their homework. Doing their marking, to the marking, you know?
Skip to 59 minutes and 34 secondsThey basically mark stuff all the time. And I say to teachers, I think you're spending too much time marking. And they say, are you saying what I'm doing is no good? And I'm saying, no, marking is good. But taking a lead from Chris's book, maybe what you could do is look through all the books and see if there's any points that you can actually take up with the whole class or look at the books of the kids who are most likely to struggle with this and see how they're doing. So the important point is, there are no more hours in the day.
Skip to 60 minutes and 2 secondsBut the question is for every hour you spend on something to do with your teaching, could that hour be spent in a different way that would result in even more student learning? I think that's the important point. There are no more hours in the day. Let's be sure we're using the hours we've got in the best way possible. And then that's the time element. And the fourth one, and I think the hardest one for the leaders, really, is creating a risk taking culture. Every head I've ever met says they believe in risk taking, but they don't, really. They only believe in risk taking if the risk pays off.
Skip to 60 minutes and 34 secondsSo what I say to leaders is if you want to encourage risk taking culture, here's what you do. You praise people at the time the risk is taken when nobody knows how it's going to play out. If you praise people for when risks pay off, you're just saying to people, play safe and be lucky. That's not a recipe for creating a risk taking culture. And with those four elements of the leadership creating a learning culture, keeping the focus, giving teachers time, and creating a risk taking culture we've seen schools able to sustain this for many years. There's a junior school in Sheffield that is on their 10th year of working on these teacher learning communities.
Skip to 61 minutes and 12 secondsAnd they're constantly refreshing it because now, the teachers have taken it up for themselves. it is their agenda. It is them choosing what to work on together what aspects of formative assessment do we want to get better at individually?
Skip to 61 minutes and 24 secondsMATT CORNOCK: Thank you very much. And Dylan, I just pick up on that I'll just skip over a couple questions and just pick up on the idea of the learning communities because Patrick's asked about the professional learning communities and basically how to set them up within his school in a face to face or online or whether you bring in outside organisations as well. So many organisations focus on UK-based teachers. I don't know if Patrick's international or UK, but there seems to be different ways you can set up these professional learning communities. Obviously, at the STEM Learning Centre, we have our online courses and our online community groups, our science learning partnerships.
Skip to 62 minutes and 1 secondBut I think within schools and within multi-Academy trusts as well, that would be an interesting question. I'll just pick up on that. We'll revisit some of the points as well from the early responses to that question.
Skip to 62 minutes and 15 secondsDYLAN WILIAM: If there's 10 of you, buy the Embedding Formative Assessment Professional Development Pack from the Specialist Schools and Academies Trust. A recent randomised controlled trial found it increased the rate of student learning in key stage four by 25%. I mean, just basically, you don't need to reinvent the wheel. It's been done before you. Get those materials. If there's fewer of you, buy our book Embedding Formative Assessment. It's designed as a workbook for the teachers to work through these ideas. I used to be nervous about pushing my own work. But now we have so much evidence that it actually works and there's so little evidence that anything else works as well.
Skip to 62 minutes and 50 secondsThat I think we should stop having teachers reinvent the wheel and just get started.
Skip to 62 minutes and 55 secondsMATT CORNOCK: They are, Patrick, a small investment but worthwhile nonetheless. Thank you very much, Dylan. And so I'm just going to pop back to the questions about changing practises. Claire asks what do you see as the biggest barriers to embedding formative assessment in all classrooms today. I think you touched on a lot of those already the idea of leadership needing to establish a risk taking culture, the allowing of time, the sustaining relationships with your colleagues, and the issues of doing this as a lone teacher where you should be working with colleagues and also trying to build the classroom culture with your students as well.
Skip to 63 minutes and 30 secondsSo is there anything else we look at as the barriers to embedding formative assessment and hopefully some of the solutions to those barriers as well? Chris, Dylan?
Skip to 63 minutes and 39 secondsCHRIS HARRISON: I think the barriers that are there really are outside of this. It's in the UK and particularly in the cities, we're getting such a turnover of teachers that it's sometimes not got the stability of staff to set up learning communities that do sustain and do move forward. We certainly need to do something in London about recruitment and retention of teachers. So that would mitigate against it to some extent. It's a difficult time, I think in schools, in some schools, at the moment. However, I think the most important thing to do for teachers is to spend some time just talking about what they do within their lessons and how they move these ideas forward, the sharing of ideas.
Skip to 64 minutes and 34 secondsYou know, I still go back to when we did that with the [INAUDIBLE] teachers and just saw how productive that was. I mean, I'm still using some of the questions that I picked up from a couple of the science teachers in my own science teaching because, you know, most teachers have really good questions for some topics and not for others. And the sharing of that is really important. So I think even at that level of doing it informally, we need to encourage that within sort of science meetings or mass meetings within schools.
Skip to 65 minutes and 8 secondsAnd we should be trying to move a lot of the bureaucratic stuff out of those meetings so that most times when teachers meet, even if they've not set themselves up yet as a professional learning community, what they're discussing and what they're swapping ideas on is about children's learning and the best way to teach that. And clearly, formative assessment has a part to play in that.
Skip to 65 minutes and 29 secondsSo it's focusing on the teaching and learning and, you know, that's essentially what the job's about and, when they can, you know, doing things like joining MOOCs like this or maybe finding other ways of actually trying to get into looking at some of the literature and studies that have been done that have shown how this has worked for different groups so they can start to understand just what that practise might look like if they actually want to move forward.
Skip to 66 minutes and 1 secondDYLAN WILIAM: I think that for me, the starting point is the recognition that the knowledge that expert teachers have is more like the knowledge of riding a bicycle than it is the knowledge of how to balance chemical equations. I can explain to somebody how to balance a chemical equation. I cannot explain to somebody how to ride a bicycle. And I think where we often go wrong with professional development is to think about it as a process of knowledge transfer, that we actually tell teachers stuff. And until teachers figure out a way of applying this in their daily practise, it's always going to be inert knowledge.
Skip to 66 minutes and 34 secondsSo I think that the after-school meetings of these teacher learning communities, they're the active ingredients in that they actually spur people to action. But this is successful to the extent that teachers try these ideas out in their classrooms and persist. And ultimately, it's about just improving that practise. And I've learned in 30 years of professional development it is much easier to change what teachers do when students are not present than it is to change what teachers do when students are present. And too many of our models have teachers meeting after school and having nice chats and going back to their classrooms and teaching exactly the same way as they did before.
Skip to 67 minutes and 11 secondsAnd I think we have to keep that focus on changing what happens in classrooms, making teachers more reflective, and that iteration between reflecting in and reflecting on practise. These are old ideas. But the important point is the focus of these ideas being applied to formative assessment seems to be more powerful than applying it anywhere else. And as Chris said, you know, you never get on top of this. I'm still learning how to do this better after 25 years of thinking about this stuff. And so we constantly reflect on our practise.
Skip to 67 minutes and 42 secondsBut as long as every teacher is reflecting on the relationship between what did I do as a teacher and what did my students learn, as long as teachers are reflecting on that, then they'll be able to advance their practise in meaningful and significant ways.
Skip to 67 minutes and 56 secondsMATT CORNOCK: Thank you very much for those words of wisdom there. There's a final point I'd like to get your thoughts on, please, which is particularly focused at new teachers. As we're coming to the end of the academic year, many teachers, new teachers, will be finishing their teacher training and be starting their career proper in schools in September. So we've covered a lot of content within this course but also within introduction to assessment for learning and then the follow on course, which is the differentiation for learning. What advice would you give to an NQT, a Newly Qualified Teacher, when they've just got hold of all this information perhaps slightly overwhelmed by it, shall we say?
Skip to 68 minutes and 38 secondsHow would you support them over their first part of their career?
Skip to 68 minutes and 44 secondsDYLAN WILIAM: Well, I think the answer is in what we've already said. It's small steps. So basically, just choose one or two ideas and work on those until they're second nature, until they're habit. And then, only then, consider something else. The other thing to remember is that I'm becoming increasingly unhappy with this idea of novices and more experienced teachers. We now know that an outstanding novice teacher is better on their first day than the average teacher will be after 10 years in the classroom. So there are some naturals. There are some people who find it more difficult to learn. The important thing is, it doesn't matter how good you are.
Skip to 69 minutes and 18 secondsWhat matters is you're getting better and getting the support of your colleagues in that progress journey is the only thing that matters to me. So I've learned lots of things from novice teachers. I've probably learned more thing from novice teachers who challenged my preconceptions when I was a head of department than I did from more experienced teachers. So the idea of every teacher can learn from another teacher and every teacher has something to offer, I think that's the important idea and getting away from this idea of novices. Give yourselves a chance to improve slowly and gradually and accept that you're going to do it for 40 years.
Skip to 69 minutes and 55 secondsCHRIS HARRISON: Now I agree wholeheartedly with that I, mean I am a teacher educator and current cohort to pre-service teachers who are finishing their course this Friday. I took them to Kew Gardens yesterday, and we looked at an inquiry approach to adaptation in the Alpine House in the Princess of Wales Conservatory. So really exciting day that started up very much with them, you know, really unsure about this new environment, teaching, and whatever. And by the end of the day, they were presenting to one another how they might set up activities within the Princess of Wales Conservatory in order to have different age groups of kids learning about adaptation. And it was really great just that move forward that was there.
Skip to 70 minutes and 37 secondsAnd I think what's really important for these new teachers is to maintain the links with their colleagues that they're going through with, that people who are at a similar stage to you and sharing that practise and talk about that will actually help sustain you and take you through. But I think particularly in terms of what you do next year, you know, you are still very much at the early stages of your professional career. There's still a lot to learn. The curve is still going to be quite steep that you're going to be on. However, as Dylan says, take those steps one at a time and gradually build on the things that work. Think about what you're doing.
Skip to 71 minutes and 22 secondsTry and keep on top of things so that you actually are always as well-prepared as you can be when you go into your lessons because you'll certainly be teaching at least double what you're teaching at the moment as a free service in the UK. But I think the most important thing is to just keep keep looking at your learners and just see what they're getting from the enthusiastic that you work. Most early career teachers are still terribly enthusiastic as are many experienced ones as well. But I think the important thing for people going into the profession is always remembering why you've gone into it. My own daughter is in her second year.
Skip to 72 minutes and 5 secondsShe was an NQT last year, and she's in her second year. And she absolutely adores working in classrooms with kids. In fact, you know, she comes home occasionally and bores me with the tales of what her students have got up to. This has become so important in terms of your viva is that your students are actually doing well. So keep focusing on those gains that you're making with these students but also occasionally look at your practise and think, OK, I've got this bit sorted. That's the next bit I need to work on. And take it one step at a time and you'll do great.
Skip to 72 minutes and 40 secondsI hope they're going to have a very good new year when they start in September in the UK in their classrooms.
Skip to 72 minutes and 48 secondsMATT CORNOCK: Thank you very much both. I think that's it's a lovely message to end on for all teachers whether you're newly qualified or have 20 years under your belt. I hope that will be very useful to you. Thank you again, Dylan and Chris, for your time today, for answering our learners questions on the planning for learning course. The course will again run twice next academic year. We have the assessment for learning, introduction to assessment for learning course, which runs once a term, and the differentiation for learning course, which follows on from this one. It's the next step for you, which we running again in January. Dr ANDREA MAPPLEBECK; Thank you. So this is unusual.
Skip to 73 minutes and 27 secondsIn the past, whenever we've had the Q&A with Dylan and Chris, I have sat live and been able to listen to them talk and then come up with a summary at the end. However, this time, unfortunately, I was away teaching elsewhere in the country. So Matt has asked me if I will come back and do a summary having recorded the Q&A. It was really interesting. First of all, it was really interesting to listen to Dylan and Chris. I love listening to Dylan and Chris and I love the fact that we have these opportunities on our online courses. And so for me, I went through and I listened.
Skip to 73 minutes and 57 secondsAnd I've kind of pulled out four key things, I think, for me that really influenced my thinking from what they were saying in response to the questions that were raised. So my first category is what teaching actually is. The second thing that came out for me is what learning actually is and then how formative assessment supports both of these teaching and learning and then finally how we can change our practise. And for me, it was interesting because all of the answers interlink. So everything that Dylan and Chris were talking about, even though I've pulled out discrete categories, actually, they all interlink.
Skip to 74 minutes and 28 secondsSo from my notes, which, as you can see, are scribbled all over, what came out for me about what teaching actually is is that it's an intentional activity. And that links very much to what we've done in this course, which is about planning so teaching can be planned for. This intentional activity of planning is actually linked to what we want the students to learn. So it's intentional, and it's for us as well as for the pupils. So we need to think about how are we supporting the pupils in their learning by what I'm planning to do in terms of teaching.
Skip to 74 minutes and 59 secondsAnd some of the things I felt that Chris and Dylan pulled out in terms of what they were discussing that linked to this were how we must be careful that we don't limit pupils. I think that was something I got from their answers-- that my intentionality of planning is not a limiting but actually a floor rather than a ceiling. So how do I scaffold that in terms of my planning? And so how do I raise expectations so all can get there?
Skip to 75 minutes and 23 secondsOne of the key things I think that they talked about in terms of my intentionality of teaching then was how I can help students understand what my intentions are for the learning and also my intentions of the quality of their performance in terms of success criteria and how it's not about them generating the success criteria. They're my success criteria, but how do I help scaffold, set expectations, give exemplifications, and practise in different contexts so that I know that they've learned that? So that's what I felt about teaching. And then I thought an interesting concept that Dylan and Chris talked about was this idea of models and considering what models are going to help.
Skip to 76 minutes and 4 secondsHow do I plan to use those models? How do I plan to use different answers within models to help increase the sophistication of the pupils' thinking? So it's that intentionality of my teaching to raise expectations for all and increase the level of their thinking because of what I'm doing. And then for me, then, distilling what Chris and Dylan said in terms of what learning is, it came out very clearly that I think they mentioned it several times that learning is about this change in long term memory, that learning is not quick. It doesn't necessarily happen in a lesson. It might not happen across an activity.
Skip to 76 minutes and 39 secondsBut actually, the intentions of those activities to build that learning over time is going to help our pupils make progress. And so one of the things that we can help is help children to understand how they retrieve from their long term memory. And that's that practise again in different contexts. And also, that can be achieved so that their memory develops over time and I think a couple of ideas that really resonated with me were these ideas of interleaving, using different ideas over time, and that distributed practise so that it's looking at concepts developing across time. And I thought that was really interesting, then.
Skip to 77 minutes and 15 secondsSo the teaching, the intentionality of it, and then the learning how am I planning for that across time? So for me, the third thing that came out for me was Chris and Dylan's discussion about what formative assessment is and how that actually supports this teaching and learning. So looking at my notes, a key thing that came out from them is that formative assessment is about formative action. It's about doing something. I think that came out really loudly. It's not about just planning. It's about what am I going to do. And one of the key things they both mentioned several times was this idea about noticing.
Skip to 77 minutes and 48 secondsSo when I'm in the classroom during that time of teaching and learning, what am I picking up? What am I developing? What am I noticing and reflecting on those. And I think Chris said it at one point. The key is that I take action. And the actions are taken to develop the pupils' sophistication in their thinking because they have to do that learning. So I felt that they also discussed the importance of feedback in that. So me planning for that, me noticing what's going on, but that feedback is about improvement. It's not about doing what I'm doing, but it's about what do I need to do to develop.
Skip to 78 minutes and 24 secondsAnd I think there was lots of ideas taught about feedback, but the one that resonated with me was the importance of the teacher pupil relationship that I know what it means for my children. I know what's going to be the right kind of feedback at the right time and so that that feedback is efficient for that particular learner. And then finally, for me, as I said, the key thing, I think, that came out about formative action was that we reflect on it and that we keep refining our practise in light of what we're finding out and that the key thing that helps us do that is evidence.
Skip to 78 minutes and 54 secondsSo we need to decide on when we collect evidence, how we're going to do that. And we can plan for that even though we're doing this responsiveness in the moment. And then finally, the last little thing I had which Chris and Dylan, I thought, talked about extensively and it was interesting. The questions were asked at the end of the Q&A about how we help support other change practise. But actually, Chris and Dylan talked about it right at the start of the Q&A that the important thing for them is that as teachers, we never stop learning, that we are constantly thinking it's not that I'm not good enough.
Skip to 79 minutes and 25 secondsIt's not that what I'm doing is not right, but actually, what can I work on to develop. And I thought it was really interesting. The whole Q&A was setting up from the beginning to the end that actually, it's about this fact that I'm going to change my practise in small steps over time whether that's 40 years, which I thought was interesting, Dylan saying 40 years at the end. But it's about the little things that I do that I reflect on and see how they embed.
Skip to 79 minutes and 46 secondsAnd they talked about this is going to be so much more powerful if we engage in this discussion with other teachers, which is the beauty of our online courses but also encouraging you to have an impact back in school. And I thought it was really, really powerful what they said at the end that all teachers, no matter where you are in your career, can offer ideas, and all teachers can learn. And I think that's the beauty of it. For us as teachers, learning never stops. And it's professional dialogue with each other that helps us improve. So again, a really valuable and interesting Q&A from Dylan and Chris where ideas were discussed and debated, and they disagreed.
Skip to 80 minutes and 22 secondsI thought that was nice. We had different perspectives brought. And as always, I've written down a plethora of research articles that they've mentioned which I know that Matt talked about in the Q&A he's going to reference. So for you to keep that dialogue going small steps, changes, reflections, and working with each other. So good luck, and it has been a pleasure a pleasure watching it and a pleasure interacting with you on the course. So thank you.
Q&A with course educators
The question and answer (Q&A) sessions on all online CPD courses from the National STEM Learning Centre are your opportunities to post detailed questions to the course educators about your teaching practice, your context and planning for learning.
Thank you to learners who posted questions. We recorded responses from Chris Harrison and Dylan Wiliam, and Andrea Mapplebeck provided a summary. The video is now online above (and on the STEM Learning YouTube channel) and a transcript will be available shortly. Topics discussed and references are listed below.
- 0:34 - Planned responsiveness vs spontaneous responsiveness
- Coweie, B., Harrison, C. and Willis, J (2018) Supporting teacher responsiveness in assessment for learning through disciplined noticing, The Curriculum Journal, 29(4), 464-478.
- Hiebert, J. Publications
- 6:35 - Success criteria and learning intentions
- Graham Nuthall
- 18:17 - Types of feedback and marks
- Kluger, A. and DeNisi, A. (1996) The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory, Psychological Bulletin, 119(2), 254-284.
- Ruth Butler
- David Yeager
- Carol Dweck (re. growth mindsets)
- 25:55 - Correcting misconceptions and constructing answers
- Kahneman, D. (2011) Thinking fast and slow. Farrar, Straus and Giroux.
- 33:45 - Spiral curricula and mastery
- Bruner, J.S. (re. spiral curriculum)
- Carroll, J.B. (re. fast/slow learners)
- Bloom, B. (re. bell curve results)
- KMOFAP research project
- Ola, L. (re. Philadelphia project)
- 42:30 - Involving students in marking and self assessment
- 47:25 - Assessment for learning and embedding learning
- 52:40 - Changing practice and supporting colleagues
- Lee, C. and Wiliam, D. (re. changing habits)
- 1:01:30 - Professional learning communities
- Embedding formative assessment professional development pack from Specialist Schools and Academies Trust
- Wiliam, D. (2011) Embedded formative assessment. Solution Tree Press.
- Science Learning Partnerships
- ENTHUSE Partnerships
- 1:08:00 - Advice for newly qualified teachers (NQTs) and others just starting out in teaching
This step is an open step, so you can bookmark it to your favourites to return to it at any time.
Please note that questions posted to this step will be attributed by your first name and copied into the video recording that will be uploaded to this step and also to the STEM Learning YouTube channel or other STEM Learning social media.
© National STEM Learning Centre