Designing Full Factorial Web Experiments

So, we’re going to look at Web experiments now. Web experiments have taken off like crazy and the reason for that, I think, are two fold. One, Web experiments are cheap, they don’t cost much. And they are fast to execute and change. And the third reason is also that you can change a lot of variables at the same time. In the basic experiment design that we just saw, we manipulated only one variable, the ad copy. But in web experiments because they are cheap and fast, you can manipulate a lot of variables at the same time. Let’s consider a simple example of many plating prize and ad copy.

And this type of multiple variables being manipulated at the same time is called the Full Factorial Design. Okay, so let’s consider a box of cereal, let’s call it Cheerios.

Now Cheerios is priced, at the moment, at $1.89. And they want to see how sales changes if the price either decreases to $1.59 or increases to $2.15. At the same time they also want to test the ad copy. Current ad copy is good for you. Now they have two other ad copies that they want to test. The effects of Cheerios last longer, you have more energy for a longer time. And it is better, so these are the two ad copies they’re trying to test. So they’re change in price and they’re changing advertisement copy at the same time. So let’s first consider each of these one at a time.

Here we see the result that if they just keep the price the same and change the ad copy, the best ad copy seems to be at the price point $1.89. This one right here, lasts longer because the sales are highest for this ad copy for the price of $1.89. Now what if they changed the price but kept the ad copy the same. That will be this, so if they changed the price, but kept the ad copy the same as good for you, the decision would be to reduce the price from $1.89 to $1.59 because that’s when sales are the highest.

So you see there are different decisions they could be taking if they just changed ad copy or just changed price. What if they change all of these things at the same time? They tried all nine combinations and saw what came about.

That’s over here. So what have we got here? Ah-ha, so you have something over here which is 1,500. That’s the highest sales. Now what combination is that? It says is tastes better and has a higher price. Wow, by actually changing the ad copy and the price at the same time, you are able to find the sweet spot,

Where you can actually charge a higher price because people are willing to pay a higher price for something that tastes better. So you can see the power of these full factorial designs where you’re changing different components of your experiment at the same time. And the web experiments because they are cheap and fast allow you to do this very quickly and effectively.