Let’s talk about different ways to measure activity of our application. And I want to talk about two specifically, RUM, or Real User Monitoring, and synthetic transactions. Those two things are important to an understanding of how we’re going to measure the behaviour of our application, both before and during production. So what is the definition? First, real user monitoring is capturing usage data from real users in production, where synthetic monitoring is where we emulate some user interaction with our application. It’s a predictable way to find whether we have some changing metrics based on different environments, different scalability requirements, different things, as well as to identify potential issues.

So between the two of those, real user metrics tracks, as you might expect, real user sessions. It monitors what’s in production. And it’s not nearly as predictable as we might want it to be. Now, if you have traffic that measures in the millions of users per day, yes, it’s going to be relatively predictable because that’s not going to change that much. But very often, real user metrics go through waves that are a little bit unpredictable.

You might get a lot of usage at 9:00 to 10:00 AM in the morning, and then a big drop-off, but that might change after holidays or at different behaviours. So sometimes, real user metrics aren’t as predictable, but they are so good at determining and teasing out how users are interacting with your system. You want to take that knowledge back, understand it, parse it, and use it to drive what you’re going to build next. Now, on the flip side, synthetics are very useful, too. Synthetics track emulated users, fake users. We’re going to create a bunch of fake users and point them against our system. This can be done before production, potentially, or during production– fine in both.

But the advantage here is predictability. We know exactly what those users are going to do, because frankly, we’ve scripted those users. They’re fake users. We’ve scripted them. So every time we replay them, we can expect the same or similar behaviour from our system. This is where the advantage comes in. We play synthetic transactions against our system, and then we can start to understand how our system is changing over time. We wanted to move from an A series machine to a D series machine in Azure. Well, how did that change the performance? Well, we can find out by running those synthetic transactions against and looking at the differences. Very easy to pull those apart.

So two important things– real user monitoring and synthetic transactions with synthetic users. Both useful in finding out what’s going on in production.