Home / IT & Computer Science / Business Technology / Microsoft Future Ready: Continuous Integration Implementation / How to create an Automated Release Pipeline
How to create an Automated Release Pipeline
By automating your release pipeline, DevOps enables you to release your software faster. Let's explore how to do this.
Hello and welcome to this video that you have been waiting for. It’s called creating an automated release pipeline. Over the last few videos, we’ve been doing everything we needed to to create the building blocks to lead up to this point. We looked at creating the artefact from our build, but then we decided to stay in the build and provision the infrastructure. Stay in the build and automate the deployment. Stay in the built, execute the Selenium test. I mean, it’s good to have all of this conflict transform and everything in the build, but, really, the right place for this is within the release definition.
By no means is this stopping you from branching off and creating your own build deployment process so that you can validate a walk-through of the pipeline before actually pivoting on the release pipeline used by the bigger organisation to get the release out, but why not have an additional environment within the existing release pipeline for your own experimentation? And it doesn’t have to be just yours. It could be like a team environment where you can go off and test your release, smoke test it, validate the scenarios are working before then clicking the button to have it rolled out to the other test environments.
So in this video, what we’re going to do is we’re going to bring everything that we’ve learned over the past few videos and construct a release pipeline that allows us to mimic the same behaviour but then leverage some of the advanced features that are available within release management, such as multipliers so that we can then distribute the load of tests execution. And along the way, we’ll look at how easy it is to take benefit from phases so that when you only want to run selective parts of your release pipeline, or you want to dish that workload in a phase to a separate agent, then how that can all be managed.
So let’s get into the demo and cover everything we’ve discussed so far.
OK. So I’m here within Visual Studio team services. If we go back to the build hub, we’ll see within the build hub we have a bunch of definitions, but the one that we’ve been working on, I’ve kind of cleansed it a little bit and removed everything to do with deployment from it. So this build pipeline just has a versioning. It restores package, compiles it, and runs the unit tests and then creates an artefact. This is the bare minimum standard you would expect from any built pipeline. I’ve removed the conflict transforms as well. So here we are just generating artefact, and then that artefact would be handed over to a release definition.
This goes back to the principle and practise we discussed where. We said we should only create a package once and then deploy that across all environments. So the underlying build definition used to create the package should be generic enough to live up to that principle that we talked about. Now that we have the pipeline in place, let’s go back into the release hub. And within the release hub, the workflow that we have seen in one of the introductory videos is you can simply create a release pipeline from here, or you have an option of going back into the build hub.
And from the build hub, seeing the summary of one of the existing builds that you have run, and use that to instantiate a new release definition. So I could go back here into the build. And from the build summary page, I could pick up one of the builds and say, you know what? I’m kind of ready to create a release process for this. And as I do that, it would take me back into the release workflow where it would plug a few things in for me so that I could then start to create a release definition. So let me cancel that and go back into release. So I’ve created [INAUDIBLE] new release definition here.
And before I demo this, let me just kill one of the previous executions that I triggered off.
OK. So let’s go back into the released definition that I’ve created for us. And as you can see in this release definition, I’ve got one environment set up, and we can actually go into the new– let’s look at the old editing view, and then we’ll go into the new editing experience as well. So within this I’ve divided our set up into multiple phases. The first phase is run on agent, and let’s call this run on agent hosted agent, OK? So I’ve got free hosted minutes, and I don’t want to set up my infrastructure, so I’m just going to leverage the hosted minutes to do the conflict transformation, deploy to Azure, and deploy to Azure db, right?
I mean, it makes no sense for me to set up my own infrastructure to do it. But then once that’s done, I’ve got some specific frameworks that are required to be set up for me in order to run the test that I may have. Now, in this case, I’m still using Selenium tests. I don’t have any specific requirements, but now I’m running these tests, instead of the public agent, I’m running this on my own private agent. So let’s rename this to Selenium agents dev test lab. And then after this point, the application’s deployed. It’s kind of tested. I want someone to manually log in and do some validations.
So I’m going to rename this step as manual checks and sign off, and it’s not unique for organisations to have in their content delivery pipeline a process for someone to manually go in and do some checks. I mean, it’s good to have automation in place. But in certain scenarios, you need someone to go and check whether the deployment is correct, whether the overwrites are done. It could be a temporary thing till you earn your stripes to get rid of this step and have a fully automated process, but the provision is there for you to stop and interrupt the pipeline execution, use that to do some manual checks and come in, execute the pipeline from that point onwards.
Now what’s really cool about this is you can assign a manual intervention to individuals, and they get an alert saying someone’s asked for your approval. And at that point, they can go and read the instructions, sign off. And having an identity specified here or a group specified here means that you have full audit of who approved it, when was the request sent, when it got approved, and what is it that you need them to do. Now once that’s done, we’re going to run the last bit of the pipeline, the last phase to run the load tests on hosted and Selenium agents DevTest Lab.
So what I’m doing here is I’m saying the last phase needs to run a little bit on the hosted agents and a little bit on the Selenium DevTest Lab agents that we’ve set up. Even in addition to this, we have an option of specifying demands and capabilities. We could tag each agent in a pool with a specific capability. So for example, I have one agent which has .NET 2 installed, and I might have an application that’s still relying on .NET 2.
So I can go and tag that agent who have .NET 2, and then I can come in here in this step and say, actually, during this phase, only send the request to an agent which has the skill capability match that I’m demanding. And in this case, that tag would be that the .NET 2 exists. So this is a good way of routing requests for execution back to servers that have that capability. So we’ve sliced this into four phases. We have one environment, and now let’s look at what options we have.
So if we look at the assigned approvers workflow, I have an option at the minute to assign specific users to sign off on the execution of the release on this environment. Now, this is still a test environment. I might not want to do that, but if this was like a prepared environment which had more checks and balances in place, then I could assign this out to a specific individual or a group, and then that group would get alerted any time anyone start a release. And it’s only when they approve it will the release proceed further. If they reject it, the release stops. They also have the provision to approve and specify a time span
to say, approve, but only deploy at 12 midnight, or 2:00 AM, or whatever. So that’s one thing. The other thing is you have an option here to send email alerts when things go wrong. You have an option here to assign someone core ownership of an environment, and this is useful, right? Because some environments have some owners. Some people are sharing environments. So getting all of them on a single release notification chain means that the communication keeps happening. There are some other options, but the one that I want to highlight here is deployment conditions. When you have more than one environment, you can set some policies to say, only deploy to environment B when environment A execution is complete.
Or any time someone creates a release, automatically roll it out on the test environment, but don’t let someone release to the next environment unless release environment one has been completely signed off. So these are some options that are available, but in addition to that, you can set up schedules as well. Some people have nightly processes within their org where the release gets triggered in the night, and then the release just flows through, some smoke tests are run, and when some of the functional testers the users come in back the next day, they have an environment that’s ready for them to start testing.
So now that we’ve covered all of this, let’s start off by instantiating a release here and seeing– actually, before we do that, let me just call out one more thing here. In the Selenium agent DevTest Lab phase where I’m running my Selenium test, I have, on this phase, done a little bit of configuration. I’m saying I want this phase to be a multi-configuration phase, and the multiplier is browser, and I want that workload to be spread across three agents. Now, if we go and look at a variable, in the browser variable I’m saying Firefox and Chrome, comma separated. So it will take this input, split the commas, and say, OK. That’s a keyword.
I’m going to inject into the tasks that are available within this phase, and any task that can take this as an input can perform some action on it. So if we go back into this phase and look at this task, this task of mine is consuming the browser parameter that is the multiplier we have specified at the phase, and it’s passing that into the run settings file. And as you remember in the run settings file, I’m taking the input of the run settings file into the actual code to then say which browser does the test need to be run on.
And considering that the multiplier has Chrome, Firefox, and IE, these three values would get assigned out to the test in parallel, and that will trigger three test executions on three agents. Now, if I had more agents or if I had more browsers, then I could kind of change this. You can have multiple multipliers. Now think about this. If you’re creating a product that needs to run on Windows, Linux, Mac, it needs to run on phones, Android, iOS, and Windows, and BlackBerries, and if any others exist, it needs to run on multiple browsers. Now, all of these are multipliers.
You don’t want to have to go in and trigger builds and releases individually, or test individually, or have separate tasks that do that. Instead, you just one pipeline that’s capable of accepting input and then dishing out the work across multiple agents that are capable of handling it, and that’s effectively what we’re doing here. So with that out of the way, let’s go ahead and trigger a release. And as I trigger a release, it says, I found that you have two artefacts associated with this release.
One is your git repository, and I’m keeping the code for my infrastructure provisioning here, which means, even ahead of actually triggering my release pipeline, I could have a phase which spins up the agents that will run the test or that will actually go and run the load test execution, but I’m not going to do that. So I’m just going to select the last version. And as soon as I create, it’s going to automatically start provisioning on that test environment. So let’s wait. It says, release summit one has been created. Let’s click on this. Let’s go into the log section, and it’s just waiting to have a hosted agent available.
As soon as a hosted agent is available, the release will start triggering.
As you can see, the hosted agent’s available now, and the job for this phase has been assigned out to the hosted agent. And as part of the workflow, it’s downloading the artefacts from the build definition that we’d linked. So build one’s release multiple times is the principle we’re working with here. And the first task it executes is to replace token in the config file, and only once it does that will it hand this off to the web deploy task which is run as part of the deploy Azure App Service, which will then take the transformed set parameters dot XML file and then use that in the provisioning for the web app in Azure.
Once that’s done and it looks like it’s completed, it’s handed that execution over to the database which is now going to start executing the DAPAC file that we have, and that DAPAC file will run against the SQL Azure to ensure that the schema and the post-deployment [INAUDIBLE] script we have gets executed.
As you can see, the first phase of the release process has completed successfully, and the second phase has been instantiated. And you can see that it’s using the multiplier now to pass over three parallel jobs for test execution of Selenium tests, one to Firefox, one to IE, one to Chrome, and all of them are running in their own separate process. Now optionally, if I didn’t want to download the artefacts, I could decide to turn that off. But going back to the theory of constrains, you need a release pipeline first. You then build, you measure, and then you learn.
And then if your tests don’t warrant downloading a git repository or the artefacts there, then simply disable it so that you can save time which are otherwise– you know, just adds to the whole release timeline execution.
All All right. So the test execution has completed across the multipliers, and now it hits the manual checks and sign off phase where the task requires someone to manually go in and approve once they’ve done the checks that are required. So as the task is assigned to me, I come in, and I would have been in email alert as well. I can see the instructions, and I can just put a comment there say, yup. Reviewed.
And I can click resume.
And after I click resume, it’s going to start progressing with the next phase which, in this case, is running the Selenium– sorry. Running the performance test using the hosted cloud based load testing service by triggering that off on the back of the Selenium agents that we have in place on DevTest Labs.
Now as you can see that everything but the last step has passed in this pipeline. So if we see why this is failed, the URI seems to be correct. Let’s see what the error message is.
OK. So it’s failed for a reason. It’s not failed because of some random reason, you know, incorrect set up something. It’s failed, as it says, in your set, you’ve specified that if any of the response times are more than 100 milliseconds, then mark the task as failure, and the task is marked as failure as there were threshold violations for the average response time. Now this is a good check, right? Because if I’m running a perf test, I want it to be actionable. I want it to alert me when things are running slow. So I can go back into the summary view. I can see that it’s failed because the threshold was violated. Now I can go back into the environments.
If I fix something, then good. If not, then I can run the same step. Let’s change the threshold to 250 milliseconds, and then I can go back and disable the rest of the pipelines that we have. So let’s do that by just switching these to disabled.
And let’s save that, and let’s restart the execution which will pick up exactly from the last phase, and that’s the whole point of having phases. So if I go back and say redeploy and test, it says, fine. You’re redeploying a release that you’ve previously redeployed, and it keeps track of all the previous attempts. And as you can see, it’s just going to [INAUDIBLE] through the previous phases because it’s got nothing to do there, as I’ve disabled everything. And, you know, we’ll see as soon as it executes the load test now.
This is time for a virtual high five. We’ve successfully completed the release definition, but what we did here was a tweak. You know, if this was a real life scenario where we were running this pipeline, we found that the perf test failed because the 100 millisecond response time threshold was violated, the 100 millisecond threshold was put there for a reason because we don’t want our web app to be slow, and that’s why we capped it. What I did here was to demo the functionality of phases. I went in and edited that value from 100 milliseconds to 200. And that a lot of people do in their own organisations. Oh, we got to get a release out.
We’ve got to get a release out. Let’s just get the release out. That’s then tweaking the pipeline to make it work for you rather than addressing the underlying issue. What I did here is onboarding technical debt. What we should have done if this was like a real org scenario is gone back, fix the issue, done a commit, that would have rolled a new build, the build would have triggered the release. And because the release itself is so quick, we would have had a working solution in no time, but you get the gist of it, right?
Now if we go back into the release, you can see that the release is all green, and we can track which version of the app is deployed on this environment, as you can see, and that’s the benefit of having the integration. So if we go back here and click on the overview section, then it’s telling me that the summit zero one release is set up on the test environment. And, of course, I can go in and set this up on my dashboard, and let’s just quickly look at the new preview of the editor experience. Now, the editor experience is going through a change.
And hopefully as it pans out over the next few sprints, you’ll start to see more actionable workflows surface within this page where you can see the execution on a diagram rather than in a horizontal, vertical pipeline. OK. So we’ve come quite a distance. We’ve come from continuous integration, which was all in the previous module.
Then, we decided in this module we were going to make the journey from continuous integration to continuous delivery, and we’ve taken the steps to create our own infrastructure by using our ARM template, deploy, run our test, whether it’s functional, whether it’s smoke, whether it’s performance, and then bring that all in nicely into a release pipeline that’s decomposed into separate phases, and that’s actionable and gives us the insights we need. Well done. In the next module, we’re going to look at some exciting stuff, make our journey from continuous delivery to an all automated continuous deployment. Join me back in the next module.
In this article, we are going to construct a release pipeline that leverages the advanced features available within Release Management.
Creating an Automated Release Pipeline
From within the building hub in Visual Studio Team Services, you will see that our build pipeline has a versioning task, it restores packages, runs unit testing and creates artefacts.
With our pipeline in place, we can move back into Workflow in the Release Hub. From here, you can either create a new release pipeline from within the Release hub, or you can initiate a new release definition from the Build Hub. Within the Release Hub, the Release Workflow will plug a few things into the pipeline so that you can start creating your release definition.
Note: You can divide your release definition into phases. Here we have four phases:
- Run-on Agent (Hosted Agent): To do configuration transformation, deploy to Azure and deploy to Azure Databases.
- Test Assemblies (Selenium Agents Dev Test Labs): To set up frameworks that are required to run tests, such as the Selenium test.
- Manual Checks and Sign Off: To manually check that the deployment is correct and to approve the deployment.
- Run Load Test (Hosted and Selenium Agents Dev Test Labs): To only send requests to agents who have the skill capability match that is demanded.
Additional Environment Options
We have established that we have one environment available in our example. Some additional options we have at our disposal include:
- Assign Approvers Workflow: This is useful when you want to assign specific users to sign off on the execution of the release in your environment
- Email Alerts: This option is useful when you want to notify specified users if something goes wrong in your environment.
- Core ownership: This is useful for communication, especially when you have multiple owners of an environment or multiple people sharing an environment.
- Deployment Conditions: When you have more than one environment, you can set policies that only execute to the second environment when execution to the first environment is complete.
- Schedules: This is useful for scheduling processes that are run regularly.
Triggering a Release
When your build is set up, you can trigger a release. You will be notified of any artefacts that are associated with the release. In our example, this is our git repository and our infrastructure provisioning.
Your build will now start provisioning in your environment.
This article is from the online course:
Microsoft Future Ready: Continuous Integration Implementation
Created by
This article is from the free online
Microsoft Future Ready: Continuous Integration Implementation
Our purpose is to transform access to education.
We offer a diverse selection of courses from leading universities and cultural institutions from around the world. These are delivered one step at a time, and are accessible on mobile, tablet and desktop, so you can fit learning around your life.
We believe learning should be an enjoyable, social experience, so our courses offer the opportunity to discuss what you’re learning with others as you go, helping you make fresh discoveries and form new ideas.
You can unlock new opportunities with unlimited access to hundreds of online short courses for a year by subscribing to our Unlimited package. Build your knowledge with top universities and organisations.
Learn more about how FutureLearn is transforming access to education
