Want to keep learning?

This content is taken from the The University of Warwick's online course, Big Data: Measuring And Predicting Human Behaviour. Join the course to learn more.

Skip to 0 minutes and 4 seconds CHANUKI SERESINHE: Hi, Suzy. Hi, Tobias. How did you enjoy this week’s discussion around big data and stock market?

Skip to 0 minutes and 9 seconds TOBIAS PREIS: Hi, Chanuki, it has been fantastic. I mean, thanks to all the Learners, all their fantastic answers and comments that they have come up with.

Skip to 0 minutes and 18 seconds CHANUKI SERESINHE: OK, so like last week, I’ve collected a few questions which the Learners were bringing up. Shall we get started?

Skip to 0 minutes and 24 seconds SUZY MOAT: Yep, let’s go.

Skip to 0 minutes and 26 seconds CHANUKI SERESINHE: So the first question is, what is the point of building models to understand behaviour in financial markets when it is clear that these models are simplified models of reality.

Skip to 0 minutes and 38 seconds TOBIAS PREIS: That’s a good question, yeah. This has to be seen in context. Obviously, models have a certain degree of complexity, depending on how many different variables you’re building into it. and so on and so forth how much data you’re plugging in and so on, so forth. So from that point of view, a very simple model is obviously extremely simple to calculate and very simple to build. But as people have picked up, obviously, they are not as realistic as you might wish for. But on the other end of the spectrum, if you think of the most realistic model obviously this becomes incredibly complex.

Skip to 1 minute and 19 seconds Because if you think of the economy or the financial markets, then you would need to actually build in every individual actor which is taking part in economic processes, like companies, like all of us individuals in the country. And even each individual then, obviously, is a very complex human being, and this might complicate matters further. So from that point of view, you have to find a very good place in the middle, and usually you want to build models which are as easy as possible, as simple as possible, because this allows you to learn something about underlying process.

Skip to 1 minute and 56 seconds And it needs to be as simple as it is still possible to explain certain feature, certain phenomena which you can measure in a complex system like the economy. And so to get this right, it’s a very tricky thing. Obviously, there is an entire field of, for example, building agent-based models, where it’s more or less the idea of having individual agents acting with each other, and they can become extremely complex if you want to make them more and more realistic. But then the question emerges, how are you initialising these individual variables, these individual agents, and your parameter space just becomes extremely huge.

Skip to 2 minutes and 36 seconds And then the question is– it becomes very transparent– the question is, what can you then actually learn about underlying system. Because with all these different parameters, you’re changing one and what does it actually mean in the parameter space of thousands or tens of thousands of other parameters and other certain, things you might want to modify in terms of rules, how people are interacting and so on and so forth. So to build a simple model which is still capable of measuring and explaining what we see and experience in the real world has a lot of value, because this actually allows us to make progress in our understanding of human beings.

Skip to 3 minutes and 14 seconds CHANUKI SERESINHE: Well, that makes sense. So it would seem reasonable to suppose that people might primarily use sources other than Wikipedia for financial information. So why would you then expect Wikipedia to be a good source for understanding stock markets?

Skip to 3 minutes and 27 seconds SUZY MOAT: That’s an excellent observation, as they often are from the Learners. And indeed we weren’t trying to suggest that Wikipedia or even Google would be the primary source of information for the majority of people making decisions in the stock market. But we did get this opportunity that it is Google and Wikipedia are all places where people search for information. And crucially we have data on what people are looking for at this huge scale. And so what we wondered was whether there would be at least some sort of hints of a connection, so would some people who are making decisions use these sources some of the time.

Skip to 4 minutes and 12 seconds Would it be enough to get a little bit of insight into what might happen next. And the analysis we carry out does suggest that there is a relationship between what people are looking for on Google and Wikipedia and what then happens in the stock markets, at least historically, as we’ve emphasised. And so I think you can take that result as a starting point, and perhaps use it to motivate further analyses of the relationship between stock market moods and what people are looking for in other sources of information. But of course, the crucial issue is that to carry our any of these analyses you do need the relevant data.

Skip to 4 minutes and 54 seconds So if we wanted to consider the relationship between another source of information and subsequent stock market moves, you would need to know, you would need the data on how people were using that information source.

Skip to 5 minutes and 5 seconds CHANUKI SERESINHE: So finally, we’ve all seen that the Learners are making excellent progress with R, and I know you’re as excited about this as I am. Now, how should the Learners deal with errors, which R might give them however?

Skip to 5 minutes and 16 seconds SUZY MOAT: That is a good question. You know that one of the most frustrating things that people who are learning to code can come against is error messages, being told, all right, you’ve done something wrong. Now, I think the first thing is, as we’ve mentioned before, is to realise you’re unlikely to be the first person who’s got that message. Indeed, getting an error, it’s not just because you’re learning, it’s simply because you’re programming. And so it’s quite likely that many other people have had that error. So you can try copy pasting it into Google and seeing if somebody else came up with a solution to this particular problem that you’re having. Also, remember there are differences between errors and warnings.

Skip to 5 minutes and 58 seconds So just because R gives you some feedback, it doesn’t necessarily mean you’ve done something wrong. It might just be trying to alert you to something, but not saying you’ve done something wrong. So if it’s clear that something’s labelled as a warning for example. And finally, if nothing else works out, something of course you can consider in the context of this MOOC is carefully explaining what you’ve done and pasting it into the comments on the course, and then, you can see if either perhaps the course community can help. Maybe somebody else doing exercises has had the same problem. Hopefully, this is a good environment to get over these first obstacles and make progress.

Skip to 6 minutes and 40 seconds And again, we see many times, with many people, that it’s really worth it if you can just grit your teeth through those first few error messages.

Skip to 6 minutes and 49 seconds CHANUKI SERESINHE: Yes, definitely keep going. So yeah, great, so thank you for your answers. What are we going to be looking at next week?

Skip to 6 minutes and 56 seconds TOBIAS PREIS: Hm, there’s an exciting week which is actually ahead of us. So we will talk about crime, and as part of it, as we’re talking about big data, we’re obviously looking into the question can we actually forecast, is there any possibility to forecast, where crime might occur. And obviously, we will also touch a little bit on the topic of terror terrorism, a very sad topic, obviously, given recent events around the globe. We would hope that the Learners will enjoy what we have put together.

Skip to 7 minutes and 32 seconds CHANUKI SERESINHE: Excellent, looking forward to it. So speak more next week.

Skip to 7 minutes and 37 seconds TOBIAS PREIS: Thank you, speak more next week.

Skip to 7 minutes and 38 seconds SUZY MOAT: Thanks a lot, Chanuki, see you then.

Week 3 round-up

In Week 3, we began to explore how big data might help us understand and even predict behaviour in the stock markets. Here’s a brief summary to help you prepare for Week 4.

Gene Stanley talked to us about how he and his colleagues use big data in combination with approaches from physics to help us understand infrequent but catastrophic stock market crises. You also learnt how data on information flow via the Financial Times, Wikipedia and Google can be linked to trading patterns in financial markets. Finally, we described our own findings which suggest that changes in searches for financial and political information on Google and Wikipedia may have contained early warning signals of stock market moves. Please do heed our warning to be very careful if you’re considering trading yourselves however!

You’ve come up with some excellent suggestions of other data sources that might offer insights into stock market movements – well done. We’ve also seen some great discussions of what might make certain kinds of collective behaviour easier to predict than others. We’re really delighted to see you all getting such a great grasp on this material.

Keep up the fantastic progress with your own analyses of Wikipedia data in R too! We know how useful these skills are in a wide range of areas, and so it’s particularly exciting for us to see you all picking this up.

You’ve been doing a brilliant job of helping each other with any error messages you’ve received while working with R. To make it easy for others to help you find the problem in your code, it’s always a good idea to detail all of the commands you typed in for this exercise before the error occurred, as well as the exact error message R has given you. You might also be surprised at how good Google is at decoding error messages in R and offering useful advice, if you try just copying the error message and pasting it in as a search query – it’s quite possible that many others have seen your error message before.

We very much hope you’ll find that you can build on the practical skills you’ve learned here following the course. Enjoy this week!

Share this video:

This video is from the free online course:

Big Data: Measuring And Predicting Human Behaviour

The University of Warwick

Get a taste of this course

Find out what this course is like by previewing some of the course steps before you join: