Skip main navigation

Designing Basic Experiments

There are four rules to establish causality in your marketing experiments. Watch Raj Venkatesan explain more.
So what is the causality thing? How do we know marketing affects sales? Well it turns out that there are four rules that helps us establish causality. Let’s look at them now.
The first rule says, that change in marketing mix produces change in sales. So for example, if you increase advertising it leads to increased sales. The second rules says, that no sales increase is observed and there is no change in marketing mix. So if you do not increase advertising, you do not see a change in sales. The third rule is called time sequence. It basically says that increasing advertising today, leads to an increase in sales tomorrow. So advertising has to first increase, and then sales increases. The fourth rule is called no external factor. What it basically says, is that when advertising was increased, nothing else happened in the market.
One of the competitors did not leave the market, or some of the competitors did not reduce the price. Nothing else happened in the market, right? So that’s the no external factor rule. So there are four rules of causality. I tried to compare causality rules to what I call the faucet rule of causality. So, Here you go. Here is a faucet for you guys. Imagine this is a faucet. And the first rules says, that you have to turn the faucet for water to come out. The second rules says, if you do not turn the faucet, water does not come out. The time sequence says, that you first turn the faucet and then water comes out.
And the last rule says, that there is no leak in the plumbing anywhere. So water comes from the faucet and not somewhere else. So these four rules allow us to say where marketing actually causes a lift in sales. Now causality is like a unicorn. It’s not easily established. It’s not easily seen. But these four rules allow us to see whether we get to causality. And this is where experiments comes in, because that is the closest that gets us to establishing causality. So experiments, the holy grail, what does it do? In experiments, you’re taking one or more of independent variables. And they’re manipulated to observe changes in the dependent variable.
Now independent variables and dependent variables, we’re hearing it for the first time. What it really means is, an independent variable like advertising is a lever the management uses and changes. And the dependent variable is something that is dependent on the independent variable. So sales is dependent on the lever that the manager pulls to affects sales, so that’s independent and dependent variable. And experiments allow you to see whether the independent variable actually causes a response in the outcome variable, or the dependent variable, the sales. Now how does an experiment, how is it constructed? So let’s look at a basic experiment here and see how the construction of that happens. So experiments has a history in life sciences, right?
It has a history from pharmaceutical industry and from medicine where they test it if a medicine works. So you would have heard of lab rats. So what is a lab rat? So basically you have rats that are grown in labs. And then you’ll give, split the lab rats into two groups. Test and control. The control group rat gets a sugar pill, and the test group rat gets the medicine. And then you observe whether the test group rats are actually cured of their disease versus the control group rat who just had a sugar pill. Now that’s the origin of experiments and that’s where we take the inspiration for the conduct marketing experiments.
Now I’m not really comparing lab rats to consumers, but that’s where the inspiration comes from. And the medicine is the advertisement that we give to the consumers, and then we see the response. So if you have the basic fundamental idea. Now let’s see how an experiment is done in the advertising sense. So let’s take the example of Cheerios. Now they want to test two ad campaigns. One could be about feeling good, feeling healthy. And a new campaign that they want to test which is about tastes good. So how do they know whether the new campaign and message tastes, good actually does better or worse than the old campaign, which was about feeling good and feeling healthy.
So let’s look at that now. So basically, what they would do is choose say, thousand customers, any thousand customers that are typical of Cheerios customers and assign them to control and test group. So the control group observes the old advertisement which was about feeling healthy. Now the test group looks at the advertisement, which is a new campaign, which is about tastes good.
Now we observe the total sales from the test group was 1,200 units and the total sales from the control group was 1,000 units. And the difference in sales between test and control was there for 200 unit. So you see that the test group customers who saw the new campaign and were identical to the control group customers who saw the old campaign. And you see a difference in sales between them that you can then attribute to the test group consumers seeing the new campaign. Now one of the main questions and experiments is, how to assign customers to experiment and control group.
Now I have been using a term as we talked through this experiment on causality that is key to understanding how to assign customers to test and control groups. Thank about it. I’ll be back and we’ll talk about how to assign customers to test and control groups.

Learn the four rules to ensure that the results of your marketing experiments are actually caused by the experiment rather than some other outside factor.

This article is from the free online

Marketing Analytics

Created by
FutureLearn - Learning For Life

Our purpose is to transform access to education.

We offer a diverse selection of courses from leading universities and cultural institutions from around the world. These are delivered one step at a time, and are accessible on mobile, tablet and desktop, so you can fit learning around your life.

We believe learning should be an enjoyable, social experience, so our courses offer the opportunity to discuss what you’re learning with others as you go, helping you make fresh discoveries and form new ideas.
You can unlock new opportunities with unlimited access to hundreds of online short courses for a year by subscribing to our Unlimited package. Build your knowledge with top universities and organisations.

Learn more about how FutureLearn is transforming access to education