Null hypothesis testing is just a simple question we’re going to ask of our data: are the trends that we see in the data real or just random noise?

We always have some sort of trend that we’re looking at when we do a null hypothesis test.

In the above image, we have the performance of two teams: Team A and Team B, and we see very clearly that Team B looks like it’s performing better. The question is, is Team B actually better, or is this just random noise?

If we calculate the mean for Team A we get a score of 8,11, and the mean for Team B is 8,73. This gives us a difference of 0,62 between the two teams. Again, the question is: is this a real difference or is it just due to random noise?

According to null hypothesis testing, we need to calculate the trend, and there’s two reasons for this:

1. It could be that there’s a real difference. We often call this a signal in the data. ie, it could be that there is a real difference in the ability levels and our data just happens to have picked up on that.
2. It’s possible that it’s just an idiosyncrasy of this measurement time point. It’s possible that just at this point, Team B did better, but Team A might do better next time. It’s just a quirk of the data, there’s not a real difference there.

To figure out which one of these reasons is likely to be true, we can apply the null hypothesis test. The null hypothesis says that any trends in our data are just due to noise, there’s no real difference. However, if we can get evidence against the null hypothesis we can infer there is a real difference and that the trend is likely to continue. And that is the basics of null hypothesis testing.

Join the discussion

Why do you think it’s important to be able to run null-hypothesis testing on a business scenario?

Use the discussion section below and let us know your thoughts. Try to respond to at least one other post and once you’re happy with your contribution, click the Mark as Complete button to check the Step off, then you can move to the next step.