Culture of Experimentation

Let’s talk a little bit more about what this means, this culture of experimentation. So if we apply the scientific method to our work, that’s a lot of what is the foundation of this culture of experimentation. So in that take on things, and I’ve adapted this from the lean starter greases book, which did an excellent job of describing how to use the scientific method and lean methods for innovation. We start with an idea and we’re looking to figure out do we know who our buyer is, do we have a vivid view of who they are? That helps us groom and texture ideas and figure out which ones are most likely to be valuable. Then we create a hypothesis.

It’s a sort of a fancy word but it just means that some testable formulation of if we do this for the user then we think they’ll like it and respond in a certain way. That’s how we instrument observation into every single thing that we do, we have hypothesis and assumptions. And then we think about how we’re going to test that and we look at efficient focal item appropriate ways of doing that. We’ll work a lot on that over the rest of this course. And then we’re experimenting, so rather than worry about should it be red, should it be blue?

We say, look let’s just try blue or Not blue as the case may be and here’s how we’ll know whether that was the right decision or not. And we’ll make a decision, we’ll do something, and we’ll instrument observation into the end rather than arguing about should it be one way, should it be another. So we’re driving this moment on every single thing where we say we had an explicit idea about what was going to be good and we should persevere and scale this up and keep going. Or as they say in lean startup, we should pivot and try something else. And this works for big strategic, should we build this product, should we not build this product thing.

But it also works at a really tactical level, with individual features and even tweaks to individual features. One everyday thing that’s important to do to cultivate this culture of experimentation is the ability to distinguish between facts, assumptions, and opinions. All of which are okay, but it’s important to just be clear about which is which, and to identify them and help your team identify them. Here’s an example from the California drought. This first item you see here, what do you think that is? I would say this is a fact. And we’ll assume, for the moment, that it’s true. And then, the second thing, what do you think that is?

Residents of Orange County aren’t going to reduce their water usage this summer. I would say that’s an assumption, because it’s something that’s testable, and you can start into this summer and sort of see if it’s happening or not. So I would say that’s an assumption. We don’t really know if that’s going to be true, but we could test it and observe it. And then this last item here, this thing about the moral imperative, I mean that’s clearly an opinion, it’s not testable, it has normative items in it should, whatever. So, that’s the difference between these three things.

And so just think about in your next conversation with somebody about something touchy or important or whatever, when are we talking about, when are you talking about, when are they talking about facts, versus assumptions, versus opinions.

As we close, I’ll leave you with this other idea. Building a really great product. So making sure that you’re not building a product who has 80% of it’s features going unused and irrelevant to the user is about being specifically right or maybe wrong a lot of the time instead of just being generally not wrong. So instead of building something that’s not terrible, we’re trying to build something that’s very specifically right or that we can observe that it’s not right and change it. So let me give you an example of how this might work. Let’s say we have a screen for a user, okay? And it could have any number of ten elements on it. Okay?

Now in a non-agile environment, we’d just say, well, I don’t know there might be these ten things. And the developers are working on it, so let’s just put all those ten things in front of a user, and we’ll release that. Well, then guess what happens. The users are confused. Different people use different fields in a different way. And your getting something that, maybe since you put everything in there, nobody is going to say you’re missing anything, but they’re also not going to like it very much because you weren’t focused and good design comes from focus, and focus comes from understanding the user. In an agile environment you would ideally have done the work to understand what’s important to the user.

So is it these two fields, is it five, what is it? And you should at least be able to hazard a guess. And what you would do on an agile team, is work in small iterations with observation, and let’s say you’re pretty confident that there’s at least too many things that absolutely must be in there. We’ll put in those two, release it, observe users, talk to them about what they’re doing, and then figure out if something is missing, and then in subsequent small batches add it back in.

So that’s just an example of what this means, but this is something that I think as you go through and think about how to apply agile, you’ll find a lot of examples of decisions that are being made to be not wrong instead of being made to be specifically right.