Analyzing an Experiment: Etch A Sketch

So let’s look at how Ohio Art did their TV advertising experiments. So, what they did was, they took Cincinnati as the test city. Then, they had a bunch of control cities. They were, Charleston- South Carolina, Cleveland- Ohio, Indianapolis- Indiana and Pittsburg in Pennsylvania. So, what they did was, the ads ran in the test city and they did not run in the control cities. We have provided links on YouTube for you to watch the Etch A Sketch TV ads. Go take a look at them and when you’re done, come back and we can see what happened. So, did you watch the videos? They’re nice, aren’t they? I thought they were good. I like them. Let’s see what happened.

Did they move people to buy Etch A Sketches? So, like the before after design, Etch A Sketch had access to some data about 12 weeks of data, to know pre-existing differences in sales of Etch A Sketch between Cincinnati and the control units. In Cincinnati, they sold about, in these 12 weeks, about 162 units. In all the control cities, they sold about 1526 units. Cincinnati share of Etch A Sketch sales among all these cities was 9.6. The way I get that is by taking 162 over 162+1526. This is the pre-existing condition here. Now what happened was, they played these ads you watched on YouTube for three weeks.

During these three weeks, sales of Etch A Sketch was about 240 in Cincinnati and in the control unit it was 1598. The test was run from 27 November to 16 December. This is very close to their peak season, so we have to control for that. Let’s see how we do that. But lo and behold, the share of Etch A Sketch units in Cincinnati jumped from 9.6-13.1%. That seems like a decent lift here. Now what we need to see is whether it was big enough, how to make sense of this lift, should we make a decision on launching a campaign based on this lift? What do we have to do next? The lift was about 136%, that’s 13.1 over 9.6.

Now what you can do is look at another product called Doodle Doug. This is also Ohio Art company product, but the beauty of having a control product is that there were no TV ads placed for Doodle Doug during this test period. There were no TV ads around this time, there were ads over here. The lift for Doodle Doug in their share was 96% so sales actually declined in Cincinnati during the test time period. If we take the ratio, then we get the difference between the lift in the test product Etch A Sketch and the lift in the control product Doodle Doug, that is 136.1 minus 96.7%, you have a net lift of 39.4%.

This TV ad gave Etch A Sketch in the test city, a net lift of 39.4%. Now the question is, okay, there is some lift. Is this lift making economic sense or not? How do we find that? Let’s see. The net lift that we saw was, 39.4%. Now the first step after that is to know how much Ohio Art makes from the sale of a single Etch A Sketch. To do that we first look at the retail price, that’s about $10. Now the retailer keeps about 36% of this $10 for themselves. The manufacturer is selling, manufacturer in this case Ohio Art, is selling Etch A Sketch to the retailer for $6.40. How did we get that?

We take, $10 times one minus the retail margin, 36%. Now Ohio Art, after accounting for variable costs of production, can keep 58% of $6.4 as contribution margin, and that is 3.71. So that’s 6.4*58%, that gives you 3.71. $3.71 is the amount Ohio Art makes in contribution margin from the sale of a single Etch A Sketch. The next thing we need to do is look at the national budget. The national budget that Ohio Art had planned for Etch A Sketch was five million. That’s over here. Now how many Etch A Sketches do they need to sell to recover this 5 million? You can get that by taking 5 million and dividing that by 3.71, which is the contribution margin right here.

And that gives you about 1.3 million units. In the whole year, before the ad experiments, Ohio Arts sold about 3.1 million units in the whole year. During the test period, Ohio Art sold about 1.08 million units, so that’s the base off of which we need to calculate the lift. The break even lift, the amount of lift necessary to make sense of the national investment in TV advertisement is obtained by looking at 1.3 million over 1.085 million. You’re looking at the break even units, which is this, over the base units, which is this one right here, the 1.08 million. 1.3 million over 1.08 million gives you a break even lift of the base of 124%.

The net lift from TV ads was how much? That was 39.4%. So clearly, it does not make any economic significance to invest in a national TV advertising campaign for Etch A Sketch because the net lift is much lesser than the break even lift possible, a break even lift that is necessary to make this any economic sense.