Skip to 0 minutes and 4 seconds So, in previous weeks, you’ve looked at gathering and analysing your data in order to draw conclusions about the effectiveness of your program. This week, you’ll be thinking about action – what steps are you going to take to maintain a cycle of continuous improvement? In effect, this week is about closing the loop of our improvement–findings–action cycle. Basing decisions about your actions on the analysis of your data from the previous cycle is commonly expected in many organisations. These decisions are implicitly well-respected and emphasise a rational, objective and empirical approach to addressing your findings. You’ll find it easier to persuade your team to support the actions being taken if they carry the connotation of being logical and obvious conclusions from the data.
Skip to 0 minutes and 59 seconds You should note that I’ve been referring to multiple ‘decisions’ about data, rather than a single decision. Models of decision-making don’t always encourage experimentation to identify the best options. Instead of deciding upon a single course of action, you should try to operate in a ‘culture of experimentation’, where you look at ‘what is possible’. Let’s look at a finding from a set of data and examine the differences in a single decision approach, compared to an experimental approach. Imagine the finding that one-third of portfolio submissions from students are being rejected and sent back for revision because they were missing elements.
Skip to 1 minute and 41 seconds In a single decision approach, your faculty might decide to provide more oversight of portfolio work, in order to reduce the number of returns for revision. From this decision, you might end up instituting three checkpoints throughout the final year of coursework, where faculty members will review any portfolios under construction and warn the student about any missing elements. Is this necessarily the right approach? Is it the best use of resources? It might be, but let’s look at how we might take an experimental approach to this finding. Instead of deciding to oversee portfolio work throughout the year, the experimental approach would be to try several different methods to reduce the number of portfolios being returned.
Skip to 2 minutes and 34 seconds This decision, when turned into action, might result in designing a few short experiments to identify the third of learners who need prompting, rather than planning a response that involves the whole faculty and every student.
Skip to 2 minutes and 51 seconds So, your short experiments could ask: Can we launch a self-reporting online checklist to make learners aware of missing elements? Can we create an auto-scoring quiz to provide feedback and prompt learner action? The learners’ responses could be used to inform them their portfolio is only two-thirds complete, then ask them if they have a plan to complete the remaining third before the deadline. Another option could be to ask the learners themselves what level of monitoring or support they need, whether that’s a monthly meeting with an educator, text message reminders of each upcoming submission, or no assistance whatsoever. This experimental approach will give you more data to work with than a ‘one-size-fits-all’ single decision approach.
Skip to 3 minutes and 47 seconds If you analyse the data from the three experiments, you will quickly identify which approach provided the most value to students, which ones you can adapt, and which you can get rid of altogether. As you work through this week of the course, remember – if you’re not taking action based on logical conclusions, then you aren’t closing the loop. Enjoy Week 3.
Acting on findings
This video introduces us to this week’s area of focus.
As the term ‘action step’ indicates, there is a turning point in the assessment cycle that marks new activity. The simplest representation is the start of a new cycle in which the action generates new data, presumably demonstrating a program improvement.
When deciding on what actions to take based on your data, you don’t need to decide on a single action - you could take a more experimental approach. Consider this scenario:
The finding: One-third of students’ portfolio submissions must be returned for revision due to missing elements.
|Single decision approach||Experimentation approach|
|Decision||Faculty should provide oversight of portfolio work to reduce the number of returns for revision.||Try several methods to reduce the number of returns for revision.|
|Action||Institute 3 check-points in the final year of coursework at which faculty members will review portfolios under construction and flag missing elements for attention by the student.||Design several short experiments to identify the 1/3 of the students who need prompting, rather than plan a response that involves all faculty and impacts all students.|
Watch the video to hear some examples of three potential ‘experimental’ actions for this finding, which should provide more data to work with than a ‘one-size-fits-all’ single decision approach.
Give some more thought to the scenario and finding above - what other experimental actions could you take to reduce the number of incomplete portfolio submissions?