The ‘why’ and the ‘what’ of learning analytics

Welcome to this lecture on learning analytics. My name is Hilde Segers. I’m a Data Scientist and Learning Analytics Expert at Qomple. This is actually the first part of two series of lectures on learning analytics. In this first part, I will discuss the ‘why’ and the ‘what’. Let me first start with the definition. Learning analytics is the measurement, collection, analysis, and reporting of data about learners and their contexts for purposes of understanding and optimising learning and the environments in which it occurs. It might sound strange to start with what seems to be quite a dry definition, but I think it’s very important with this definition from base.

It was thought of in 2011 for the first Learning Analytics conference, but I will argue, in the second lecture, why it’s actually even more important today. Let’s focus on the ‘why’. This is actually integral part of the definition. It defines the purpose, hence to understand and optimise learning and the environments in which it occurs. When we look at data or data use cases in the context of education, there’s actually more different use cases that we can come up with that would not be learning analytics. For example, we could also use data for marketing purposes or for up-selling courses, which I would argue is not a great thing to do.

You could also use data as a teacher to punish students, for example, if they’re not watching all your videos until the end, by giving them a lower grade, but this is not learning analytics. Learning analytics really is only for the projects where you’re trying to understand and optimise learning and the environments in which it occurs.

Understanding and optimising learning and the environments in which it occurs, it’s still rather vague. So if we want to make it more explicit, I drew up three examples here that could be learning analytics projects. For example, we could want to improve the course quality, improve the student’s success rates, or come towards personalised learning paths, which means that if you have an online learning platform that not every student follows exactly the same path through the platform, that it might depend, for example, on their performance of quizzes on which the learning elements that are shown. Then when we’re looking at different types of analytics that exist, there’s actually different layers of analytics that exists.

The first one is descriptive analytics, which is, if you’re driving a car and you’re looking through the rear view mirror, that’s an analogy for descriptive analytics. At the end of a course, for example, you’re going to see what has happened in the course. Diagnostic analytics is analytics where you’re really going to ask a question that you want to have answered by using the data. Then when you move towards more complex types of analytics, predictive analytics, you’re really going to predict some things, and prescriptive analytics, you’re even going to automatically act upon predictions. To make it more tangible, let’s look at a different example.

For example, if you want to improve course quality, descriptive analytics could be that as a teacher, you’re going to look at the multiple-choice quizzes that you had and look at what the success rates were at the first attempt. If you’re really doing diagnostic analytics, you could go look into those questions where the success rate of the first responses was, for example, lower or higher than you expected, but as you move towards deeper layers of analytics, for example predictive analytics, this could be that you have a system that automatically predicts which questions are actually need your attention or off where something strange happens with the patterns of learning data of students, and then prescriptive analytics, you would even have automated suggestions on how to improve your questions.

Similarly, to improve the student success rate, you could also come up with different use cases from the different layers of analytics. When we look at personalised learning paths, this is actually already an example of a prescriptive analytics. As we move through the different layers of analytics, the most basic one is descriptive analytics, but don’t misunderstand, it’s very powerful in itself. But basic in the sense that there are basic statistical techniques such as significant analysis that can be used in this layer, while if you’re moving towards prescriptive analytics, then your technical complexity increases and you have more advanced machine learning algorithms needed to implement these consents.

Now that we have defined the ‘why of learning analytics’, the purpose of what we’re doing with our projects, let’s look at the ‘what’. The ‘what’ also makes up the interior part of the definition, which is the measurement, collection, analysis, and reporting of data about learners and their contexts. In this part, I want to zoom in on the data and what type of data that you can expect coming out of online learning systems. Typically, in an online learning system, you have a course. In this course, you have learners, and learners interact with learning elements. For example, learner A watches video, learner B answers questions from quiz 1, etc.

It’s very, very basic principle, but if you gather all the interactions that all the learners have with all the learning elements in a course or even across different courses, this becomes very rich and very interesting data on which many of the complex ideas that were said before can be built. If you look at the interaction, for example, in this use case, learner D answers x to quiz 2 at timestamp t. This is actually quite powerful also because of the timestamp, because the timestamp not only implies the time and the day, for example, the day of the week when exactly students log onto an online platform to their learning, but it also implies the duration if you have two consecutive timestamps.

Also, when you different timestamps for all the interactions in the platform, you actually know the order in which they went through all the elements. This can be different for different students and you might be able to relate this, for example, to performance or something like that. Very interesting data and patterns arise from this seemingly very simple concept of a learner that interacts with a learning element.

If we can look at the first example that we use for the ‘why’, it was to improve course quality, then we are looking really at the learning elements, and then under specific learning elements, for example quiz 1, we’re going to look what were all the interactions that all the students in this course had with this particular learning element. What can we draw from this? For example, we could see how many students pass on the first attempt.

But on the other hand, when we go to second example that we had, which was to improve student success, we are not going to aggregate the data on the learning elements side but we’re going to look at really the learner side, and we’re going to look at a specific student, and then see what interactions that they had with all the different learning elements. Perhaps, we could do this for all the students and then compare them and then also take into account the results on quizzes. We might be able to draw some interesting insights there.

Now, I have discussed why we’re doing learning analytics projects and what data comes out of learning analytics projects, and this is the end of this lecture. In the next lecture on learning analytics, we’ll go further with the ‘for whom’, the ‘how’, and discuss the constraints.