Home / IT & Computer Science / Business Technology / Microsoft Future Ready: Continuous Integration Implementation / Availability and Performance Testing
Availability and Performance Testing
.
[SOFT MUSIC]
Hello. And welcome to this video on availability and performance testing. So far, we’ve seen a lot of stuff. But what we’ve seen is deployment, smoke testing. Smoke testing tells you a bit. Smoke testing tells you everything’s OK. But it’s testing for performance that’s key to rolling out new features in your application. What you don’t want to do is write a feature that introduces a dependency on a third-party API. You decide to automate everything, press the button, have it rolled out, and next thing, your application is falling apart because you have thread locking issues, you have a dependency on this third-party library, which has gone down. So you have no circuit break pattern within your code to recover from such failures.
And if you’re working on a monolith, it might just end up taking down your entire application. So wouldn’t it be better if there was a way of releasing functionality into production, but then ahead of releasing into production, releasing it into a test environment and mimicking load that is production-like? Now, in the past, mimicking load that’s been production-like has always been a challenge. And it’s been a challenge because infrastructure had been expensive. You go and invest in buying a server that’s production-like only to realise that when you aren’t releasing stuff as often, what do you do with that high-end server which is costing you a lot?
And then chances are if it’s a high-end server that’s physical, then it’s going to start degrading over the number of years. Now you need to find a replacement for it. Well, luckily, with the cloud era, you don’t need to worry about that. And with the era of Visual Studio Team Services, performance testing and availability testing is offered as a service. So you don’t even need to set up your own kit. If you have an application that doesn’t require installing components and can communicate over HTTP or HTTPS, then this is a perfect match, a match made in heaven.
In this video, we’ll look at how we can supplement our build and release pipeline by adding out-of-box tasks to test for performance and availability. I’ll also quickly show you that there are other libraries available with support running open source and third-party performance test, like Apache, JMeter. This video entirely focuses on test automation, but more specifically on performant testing, which I think is one of the fundamental things that you need in any meaningful pipeline. So without further ado, let’s jump right into the demo. So summit.master– that’s the pipeline we’ve been working on.
And you see this pipeline has grown from just semantic versioning to one that’s replacing tokens, creating packages, publishing artefacts, consuming those artefacts to provision infrastructure, get some information– sends back from that infrastructure provision, deploy the web app, deploy the DB, deploy– run the functional tests. Now, what we want to do after this is we actually want to run performance tests. So what I should do is look for a task that does that. I could navigate into the Test section. And you would see that there are a bunch of options available. There is cloud-based load test.
Now, if you are coming from an existing investment that you’ve made within performance testing using Visual Studio, then you would have a lot of Visual Studio-specific load tests. This task allows you to use those tests and run them in your existing pipeline. If you’re starting afresh and you don’t have any tests available, you could actually launch Fiddler, start using your application, extract the Fiddler trace, and publish that as a test. There’s a converter that allows you to convert your Fiddler traces into Visual Studio performance tests. You could also just submit the same Fiddler traces as a perf test within Azure. So if you haven’t started, getting started is fairly easy.
But in this case, what we want to do is we want to add the Cloud-based Web Performance Test task. And if we click here, it asks for a number of things. The first thing it needs is a connection back to Visual Studio Team Services. It’s kind of funny you would say, hey, I’m running this task from an existing Visual Studio Team Services. Why do I need to tell it which Visual Studio Team Services instance to use? Now, certain organisations might have multiple VSTS accounts. And they may be stacking up a lot of credit for Visual Studio load test virtual minutes in one of the accounts. This gives you a way of consuming virtual load test minutes from any account.
So you don’t have to specify the same account that you’re running from. You can optionally use any account. And that’s the key purpose of having it here. Now, I’ve set up an endpoint here. You could do that by going over in the settings and creating an endpoint and putting your personal access token there. Next thing– you need to specify the web URL of the server that you need to test, like in this case, we have this specific page. But what if we are adding a new page called Contact and we simply wanted to check as part of the deployment whether the Contact page was deployed and it can bear the new load that we anticipate?
So we could fully qualify this, take this URL, put it here. And then it says, give it a name. So let’s call it Perf– let’s call it Availability Test for Contact. Now, it’s giving you an option of how much user load do you want to mimic. You can go from 25 to 250. But there’s nothing stopping you from putting more load on this. So we’ll start with the basic 25. And then it gives you an option of how long do you want to run that load up for. In this case, we’re running for 60.
But if you were using your own custom tests rather than this out-of-box task, you could set up a step load to say, I want the load to go up from 25 users to 250 users at a step of every 10 seconds. And that’s more realistic because if your marketing team is going to go and launch a campaign, you’re not going to get all the 100,000 users you think you’re going to get in the first second. It’s probably going to be a few people checking their emails and then browsing to your website. So you should really be testing your application for step load rather than just one static load.
And if you want to do threshold testing, then you should keep pushing that step load up till your application starts to exhibit areas of concern. And that will help you understand what is the maximum threshold capacity your application can survive. But the core purpose of having this lightweight task, which is out of the box, is so that you can sense-check for availability. But for advanced scenarios, you can create your task using Visual Studio. So in this case, we have an option to use automatically provisioned agent. And that’s the one we want to use. But if we had some very specific scenario where the agent needed some bespoke software, then we could self-provision our infrastructure.
This option is also useful if you are hosting your own infrastructure because of investments you have made in data centres or physical kit. Apart from that, you don’t really need to change anything. You do now have conditional tasks within VSTS. So if, for example, this task fails, you could specify a custom condition to say if that task failed or if this task failed, then do that. So it helps you build conditions. We don’t need to set anything specific here. We’ll just skip over this and save this.
Now let’s trigger the execution of this pipeline. It’s going to take a bit of time because we’re running a bunch of tasks. But let’s come back to the results when they’re ready.
The build’s gone green here, as you can see. Let’s scroll up to see what the end result was– complete. High five. It’s successfully executed the tests. You can see that it used this agent, consumed the 250 virtual user minutes to run the load up using the specifications that we called out. So let’s see the results. Let’s go back into the build summary. And let’s go into the Test section.
And let’s go into the Summary section. We can see load test results. Now, ready to be amazed– you will find the test results really, really useful. Click on the Test Run.
Look at this. You’re not only triggering the test from the cloud test service, but you’re getting very, very fine-grained results about what the average response time was, what the user load was, what the request per second was. And you have the ability to drill in and figure out your response time, percentage failure, what type of HTTP responses did you have, what kind of thresholds were hit, what errors happened. All of that available right from within the web is just amazing. I come from the background where you had to run your test for load from Visual Studio. And so you had to had the high-end version of Visual Studio, which was called Visual Studio Ultimate back in the day.
And then the only way to see the test results was after the test execution completed. With the cloud version, you can actually see the test as the execution is in flight, which means if the tests are going off track and you’re running for thresh and you start to see a unique pattern, which could mean the– either the app is not deployed correctly or there are other issues, you have a chance of stopping the test. But in the early days, you could only see the results at the end of the execution, by when it was pretty late to change any of or adapt your test. So bringing it into the web is meaningful.
And surfacing that data in a way, in charts, where you can see the performance, the throughput, the errors, the tests– being able to see the data points, being able to go into diagnostics to see the specifics of IP, protocol, and I think being able to drill in further into this with the logs, I think, is amazing. And this is the– this is what you say, like infrastructure is a commodity now. And I would say virtual load testing is a commodity. And if you aren’t using it in your build process or in your release pipelines, then now’s the time to get started.
So with that, we’ve seen how we can structure all the elements from a deployment point of view, infrastructure provisioning point of view, testing point of view within the build process. Join me in the next video, where we’ll take all of these elements and put them in a release pipeline. And that would be the true test because ideally, all of these activities should be part of your release pipeline. And so we’ll see how this fits in into the structure of phases and how Release Management will behave when we run the individual phases on different agents. Join me in the next video. [SOFT MUSIC]
So far in this activity, we have learned a lot about testing and the importance of testing. We discovered test automation with Selenium, using coded UI and using Test Manager and we experimented with adding and running tests and explored the outcomes of those tests and we were able to see that everything is kind of ok with our build.
But, the testing we performed did not test performance. And testing performance is key to successfully roll out new features.
Would it not be great if there was a way of releasing functionality into production, but then, ahead of releasing it into production, releasing it into a test environment and mimicking load that is production-like?
In this step, we will look at how you can supplement your build and release pipeline by adding out of the box tasks to test for performance and availability.
Run a Performance Test
By now, you should have experimented with semantic versioning, creating packages, publishing artefacts, consuming artefacts to provision infrastructure, deploying a web app and database and running functional tests.
The next step is to run a performance test.
To do this, you can navigate to the test section and find a cloud-based load test or cloud-based Web Performance Test Task. Follow these steps to complete the performance testing:
-
First, you need to establish a connection back to Visual Studio Team Services.
-
Next, you need to specify the web URL of the server that you will be testing. You will have the option of how much user load you want to mimic and the duration that you want the load to run for. Here you can also combine step load and threshold testing.
-
From there, you can opt to automatically provision your agent or you could self provision your infrastructure. You can also specify custom conditions within the task.
Triggering the execution of the task may take some time, but you will gain valuable insights from the results once thee execution is complete.
Join the discussion
What are benefits and challenges of releasing functionality into a test environment?Use the discussion section below and let us know your thoughts. Try to respond to at least one other post and once you’re happy with your contribution, click the Mark as complete button to check the step off, then you can move to the next step, Automated Provision and Deprovision Environments.
This article is from the online course:
Microsoft Future Ready: Continuous Integration Implementation
Created by
This article is from the free online
Microsoft Future Ready: Continuous Integration Implementation
Reach your personal and professional goals
Unlock access to hundreds of expert online courses and degrees from top universities and educators to gain accredited qualifications and professional CV-building certificates.
Join over 18 million learners to launch, switch or build upon your career, all at your own pace, across a wide range of topic areas.
