Skip main navigation

£199.99 £139.99 for one year of Unlimited learning. Offer ends on 28 February 2023 at 23:59 (UTC). T&Cs apply

Find out more

Predicting human behaviour with big data

Find out how large data sources can help us make better predictions of future behaviour, by allowing computers to spot patterns in large datasets.
So Tobias has talked about how we can use these new data sets to get a better idea of what’s going on in the world right now. But making good decisions is not just about knowing what’s going on right now. It’s about taking the best guess we can of what we think might happen in the future. As humans, we have to do this all the time. We have to try and predict what we think other people are going to do. We’re like walking prediction machines in a way. And we make these predictions on the basis of human behaviour patterns we’ve seen repeat in the past.
So for example, if I asked you to tell me when you think traffic is going to get busy near you today, you’d probably be able to take quite an accurate guess because you’ve seen that pattern repeat many times in the past. Even the fact that humans have personalities is, in a way, a reflection of the fact that human behaviour repeats itself in some ways. So for example, you might have a friend who you know is always late to meet you. Or alternatively, you might have a business partner who you know can be relied on to deliver on time. The new data sets we’re collecting through our interactions with technology document human behaviour on a much larger scale.
This means that we have the opportunity for computers to try and find patterns of repeating human behaviour, which are perhaps more nuanced than a human brain can see on its own. Or perhaps these patterns are only evident if you look at behaviour on a very large scale - a national or an international scale. More than you and I come into contact with on a day to day basis.
Recent work has started to look at whether what we do online might also be a good indicator of what we then go on to do in the real world afterwards. If you consider how you and I use the internet, we often go online to look for information to help us make decisions about things you want to do in the future. So for example, we might look for information about a restaurant we’re considering going to. We might try and find out which film we should go and see. Sharad Goel and colleagues looked at data from Yahoo on what films, songs, and games people had been looking for online.
And they showed that if you look at this data, you can actually anticipate the rankings and revenues of films, songs, and games in the next week. Now you might realise if you know the ranking of a song in the chart in the previous week, that’s going to give you a pretty good idea of what the ranking of that song is going to be in the chart this week. And Goel et al. showed that indeed this is the case. You can make good predictions on this basis. But if you add this search data into that predictive model, then your predictions improve. You get better at guessing where those films and songs and games are going to be in the next week.
However, with all of this, we have to bear in mind that trying to predict human behaviour is difficult, not least for one special reason. It’s rather different to forecasting what natural systems might do, for example, like the rain. If I go outside and I say, I think it’s going to rain later today, then the rain isn’t going to change its behaviour as a result. But if I say I think I know what a large group of people are going to do, they might possibly decide it’s not in their interest for them to do that anymore.

Making good decisions requires us not only to make our best possible guess about what is going on in the world right now, but also to make our best estimate of what we think might happen in the future.

As humans, we try to predict what other people are going to do all the time. We often do this by identifying repeating patterns in behaviour – for example, someone’s tendency to be late, or times at which traffic is particularly bad.

Watch this video to find out how large data sources can help us make better predictions of future behaviour, by allowing computers to spot patterns in large datasets that humans alone might not have identified.

This article is from the free online

Big Data: Measuring And Predicting Human Behaviour

Created by
FutureLearn - Learning For Life

Our purpose is to transform access to education.

We offer a diverse selection of courses from leading universities and cultural institutions from around the world. These are delivered one step at a time, and are accessible on mobile, tablet and desktop, so you can fit learning around your life.

We believe learning should be an enjoyable, social experience, so our courses offer the opportunity to discuss what you’re learning with others as you go, helping you make fresh discoveries and form new ideas.
You can unlock new opportunities with unlimited access to hundreds of online short courses for a year by subscribing to our Unlimited package. Build your knowledge with top universities and organisations.

Learn more about how FutureLearn is transforming access to education