Want to keep learning?

This content is taken from the University of Leeds & Institute of Coding's online course, Evidence and Data Collection for Problem Solving. Join the course to learn more.

How to Plan Business Experiments

Managing problems in businesses involves identifying problems, finding metrics to measure the effects of the problem, then experimenting with changes to try and improve those metrics over time.

images from the course

In technology or digital businesses, this feedback loop is sometimes referred to as Build, Measure, Learn, and is a key principle in the Lean Startup approach developed by Eric Ries. If you have something you need to change in your business, you probably have an idea for how to do it, but you need to take a scientific approach to gathering evidence to see if your idea is successful. If you change something, do your metrics go up or down? Do you make more money, or less? If it doesn’t work, you go round the cycle again, learning from what happened in your last attempt.

Proving that ideas work with evidence means you aren’t relying on people’s opinions or guesses. Experiments can help solve disagreements about the best way to approach a problem, too. We have provided an example experiment in the See also section below.

Business experiments can be very quick and simple, but they all start with a question that is linked to a business need, strategy or KPI. The process of running an experiment is the same as the basic process of data analysis you saw earlier in Step 1.9:

  • Formulate your question
  • Collect your data
  • Prepare and clean your data
  • Analyse your data
  • Communicate your insights

Formulating the question is the most important part of running an experiment to fix a problem. The questions we answer with data are called hypotheses: educated guesses about why something happens, which can be tested and measured, and proved true or false based on those measurements.

A good hypothesis needs to be specific, so that you can see a clear cause and effect from the actions you take. It should describe the problem, the change you’re going to try, the effect that change will have, and an estimate of how your metrics will change if your change is successful. You can use the format:

We think that… (Your problem statement, or guess)
because… (Why your problem is occurring)
So if we try… (Your experiment or test: what simple thing are you going to try changing?)
and measure… (Your metric: what pieces of data should you measure or collect to prove your hypothesis)
then we should see… (The positive result: changes you will expect in your metric, KPI or data, with an estimate of the size of the change.)

The final line gives you a way of deciding if your experiment is successful: if you see the change that you’ve specified, your hypothesis is true. If not, it’s false and you need to try something else.

If Aisha were to run an experiment to check if her idea of promoting a Tuesday special offer was working, her hypothesis might be:

We think that… our delivery orders are down on Tuesdays compared to the rest of the week
because… people do not have a habit of getting takeaway on Tuesday nights.
So if we try… sending a Tuesday-only special offer email to 100 customers on Tuesday morning,
and measure… the number of orders on Tuesday, and the number placed with a discount code,
then we should see… our Tuesday orders in that week go up by 10% over our normal average of 52, and 5 orders are placed with the Tuesday discount code.

Have your say:

  • Why do you think running business experiments is a good approach to solving problems?

Share your thoughts in the Comments.

Share this article:

This article is from the free online course:

Evidence and Data Collection for Problem Solving

University of Leeds