Home / IT & Computer Science / AWS / Microsoft Future Ready: DevOps Development, Implementation and Azure Automation / Bring Your Own Template
Bring Your Own Template
.
Hello, and welcome back to this video on DevTest labs again, but this time slightly different topic. Over the last videos– a couple of videos that you have seen– we’ve talked a lot about the automation capabilities within DevTest Lab. The one thing I want to emphasise here is that the automation that you perform within the DevTest Lab is not just locked down to the DevTest Lab. It’d be such a waste of resource if you spent all this time creating these wonderful ARM templates to provision environments for you, but you had no way of extracting them from the DevTest Lab and using them to provision real production kind of set up.
Now the concept of BYOT– bring your own template– just exactly solves that problem. Where you can either bring the generic Azure templates that you have into the DevTest Lab and equally take the DevTest Lab templates and provision them outside of DevTest Labs. Bring your own template was a feature that was introduced, well, not too long ago, but it’s made DevTest Labs a lot more popular with people. So we’ll jump right into a, demo and we’ll look at the capabilities of bring your own template. We’ll pick up one of the templates that I have within my GitHub repository, and then we will look at creating a pipeline with VSTS to provision an environment using it.
And as part of the environmental provisioning process, you can see that you’re just not logged on DevTest Lab. You could use that to provision the template pretty much anywhere within Azure. OK, so I’m going to flip right into the demo. OK, so I’m going to navigate into VSTS. Now if you didn’t know, VSTS is accessible from within Azure portal. So you can look for Team Services or Team Projects, and as long as your Team Services account is associated to the same identity that you’re accessing VSTS with, you’d see it showing up here.
So in this case, I’m just going to select the Team Services account, and I should hopefully see the list of all the Team Services accounts that are associated to this identity. Here we go. The AVA devops TAR is the one that I’m going to use for this demo. I’ll click this and then click on the link here, and then it takes me right into the instance. But from the Azure portal, the commercials are enabled. So if you wanted to purchase more build minutes or more user licences, then that can also be done right from within Azure portal. So I’m within the VSTS instance– Team Services as it’s called these days. I can click to create a new project.
Let’s call this ISC Training Demo 02. Select a repository. Select a process to create, and in a couple of minutes’ time, the project will be created and ready to use. All right, so the team project has been created. The first thing I’m going to do is I’m going to initialize the Git repository and select Ignore File for this repository. Initializing my Git repository here, and we’re good. Now I’m going to go into the build section, and this is what I will use for leveraging continuous integration. I have the option of creating a new definition here, so let’s select this option.
You get this really nice form where you could select Create an Empty Build Definition, or choose from templates that are already available for you. So if you wanted to deploy, for example, a cloud service, then a ready-made template is available for you to just pick up and apply. What we’ll do is we’ll start off with an empty process so we can cover off some of the other details. And when I select that, the first option I get is to select the source. Now this is quite amazing here– this bit, the Git sources.
Not only can I pull sources from within VSTS, but I can actually leverage the CI/CD system that’s available within Team Services, with GitHub, with a private Git Repo that I may be hosting myself, or with a sub-version system that I may be hosting myself. Now in this case here, I’m going to select GitHub. And of course I just need to authenticate myself, so let’s create a GitHub connection. I’m going to just authorise behind the scenes. It’s going to do the heavy lifting of generating the odd token, making sure that a service endpoint gets selected. OK, needs to authenticate, but it needs more details to authenticate. Right. So I’m going to have to give it a token.
Let’s try if this token works.
OK, so let’s see if we can– there we go. This is the repository. And again, it asks me which branch I want to associate. It’s just double-checking that this is the right repository. Yep, that’s the right repository. That’s the right branch. So we’re pretty much connected with GitHub, which means not only can I trigger the build and pull the sources from GitHub, but I can actually set this up as a continuous integration build. Now what that means is if I go into Triggers and set this as CI, every time I do a commit in GitHub on this branch in this repository, VSTS will trigger a CI process. Isn’t that spectacular?
It’s event driven, so tools that are outside the scope of VSTS– the open-source tools outside the scope of VSTS– can also leverage the wonderful CI/CD system that’s available within VSTS. So going back into the Task section, the next thing I’m going to do is I’m going to add a task for Resource Group Deployment. Click Add. And within this Resource Group Deployment, I’m going to select the Azure subscription that I want to deploy the resource to. So I’m simply going to select the IDC PDC subscription that I have. Let’s call it Azure IAC Subscription. OK, it’s just behind the scenes doing the communication integration, generating the token, authenticating, authorising– all of that is abstracted away.
You don’t have to worry about it. Once that’s done, you click Refresh and select the Azure IAC subscription. It says what action do you want to perform here? Do you want to create a resource group or do you want to update a resource group? But you’ve got a lot more options than that. You could, in theory, run the same process to start VMs within that resource group, switch off VMs, delete virtual machines, and it’s quite useful. Because you could have separate build definitions that can run different times of the day. For example, in the morning to provision stuff, in the afternoon to just validate that their state is the same as when it was originally provisioned.
And then at the end of the day, run a scheduled job from here, which simply deletes the VMs that are not being used. So in this case, we’ll stick with Create or Update Resource Group. I’m going to have to give it a resource group name, so let’s just call it– just follow the same convention– IAC Training VSTS Demo. And select a location. So let’s select West Europe.
Template location, well, the template is coming from the Git repository. See, I can even browse and see my Git repository here. I can see the folders and where the files are right from within VSTS. So I’m going to select the template, which is the IS 2VMs equal 1VM template. And then I’m going to just select the Azure deployed or JSON file. And then I’m going to also have to specify parameter file. Although it’s optional, I could pass the parameter file in here, because it’s got values I need injecting as part of the deployment process, but you don’t really have to specify it if you don’t need to.
So let’s select the parameter file here, and then if there are certain parameters that you wanted to override, then you could do that by the space that’s provided here. And the deployment mode is incremental, so it’s just going to run and make sure that if there are any dev tests from what the template dictates, then it bridges the gap. You know, the rest of the stuff I’m going to leave as is. Save this build definition and cue a build, and let’s just call this build definition something before we cue it. Let’s call it– let’s just quickly rename it.
Click on rename and call it Demo.Deployment.
OK, now I’m going to cue the definition here. It’s going to run using the VSTS hosted agent. See, the other benefit of doing this is I didn’t have to set up a build system myself. I didn’t have to instal all the versions of PowerShell and ARM and Azure SDK. So I’ll just offer it as a service. So your time to market significantly gets reduced because of that. So it’s just queuing a build.
In essence, what it’s doing behind the scenes is making a connection to the hosted agent, taking the extract of the build and all the settings, passing that over, and then the build agent’s just simply going to run through the steps and orchestrate them in the order that they’ve been specified. So first off, it’s going to Git clone. It’s taking down the repository from GitHub. It’s just going through the Azure Resource Group deployment process now. So if I go into the Azure portal, and within the portal if I click on Resource Groups and search for whatever I had called that, which was IAC Training VSTS, there you go.
Well, it’s just starting to provision the resource group, and if we look at it, it’s a complex template that’s not only provisioning VMs, but going the full mile with NSG rules, availability sets, public IP addresses, and it’s just provisioning it now. So if we go into the deployment section here we can see that the deployment activity is happening. So not only can you see the updates, but it’s incremental updates, so the messaging is quite good from that point of view. So if things fail, you have a clear order of how you got there– what the inputs and outputs were. So this will run for about 5, 7 minutes, because it’s got a lot of resources to deploy.
And we’ll continue– we’ll leave it running. While this is running, there’s another thing that I wanted to show you. In the last video, I talked about creating a pipeline that allows you to spin up environments and run some tests on those environments, maybe run a DSC script or two to check the configuration of that environment. And on the back of that, run some tests. And then when you are satisfied with the test, simply snapshot the environment and then delete the environment. So I’m going to show you a pipeline that I’ve already constructed which kind of does that. It’s really just to open you up to the ideas of what you can achieve with DevTest Labs.
In this release definition here with VSTS, what I’ve done is I’ve created one environment called staging. And I’ve used some of the out of box tasks that are provided by DevTest Labs. Now if you don’t see these tasks within your VSTS subscription, nothing to panic. Simply browse to the VSTS marketplace and look for Azure DevTest Lab.
And there you go it’s. Got over 1,000 downloads– quite well rated. And if you click Instal here, it gives you an option to Instal this free task to your VSTS subscription. Just select your Azure subscription. Because it’s already installed on mine, there’s no point in me selecting to do that again, but then when it kind of does the installation behind the scenes, it then makes this task available within your Task Library. So when you now go into the Task Library and within the Task Library you go into the Deploy section, you’ll be able to see the Azure DevTest Lab tasks. And these tasks are quite well-purposed here. For example, Create VM would provision the VM for you.
Delete VM would delete the VM, and the Create Custom– really, I mean, these are the three operations you would probably want to perform from VSTS. So very, very relevant. And that’s exactly what I’m doing here. I have taken the on definition. I’ve put it in my Git Repo here. So I’m just specifying the endpoint of that. Let me gain some real estate back so I can show you the actual values here. So let’s go back into the code section, and let’s select– so this is the actual template. And I’ve just simply specified the endpoint of that template here. I’m specifying a bunch of parameters. Again VSTS gives you a way of storing these parameters as variables.
You have the option of encrypting them, so if you’re putting in passwords there, they’re not showing up as plain text. Going back to Environments, I’m just specifying the lab that needs this needs to be deployed to. After that, in my pipeline I’m just running a DSC script, which does certain health checks on the server that’s provisioned. It copies my software, i.e. My package onto the server, and then it simply sets up IIS bindings, deploys IIS, configures the binding, the IP address, the AppPool, deploys the actual website, and then it starts the execution of tests. Now at this point my environment’s provisioned. My application software is deployed. My automated tests are running.
And then once it finishes the test execution, you can introduce an optional step, which is a manual intervention. So if you needed to have someone manually log onto the server to do some more smoke testing or any kind of health checks, then this is a way to execute a certain pipeline, wait for a little bit, and then move to the rest of the pipeline execution. So once these health checks are done, the next thing it does is it simply creates an image, puts it in the DevTest Lab Custom Image Library, and then deletes the VM.
So over a process of 30, 40 minutes, you’ve tested the automation, you have run your health checks, you’ve run the automated tests, and now you have a custom image that you can go back to, should there be a problem that you needed to investigate. So I just wanted to give you an idea of how you can leverage DevTest Labs for various scenarios for purposes of testing. So going back into the actual resource deployment, the resource deployment is still happening. If we flip into the Azure portal, we can see it’s probably got a lot more resources in here now. Let’s click on Overview.
The list is growing. It’s probably deploying the SQL pass and the SQL IIS instance now. So we’ll just give it a few more minutes to complete deployment.
All right, so the build is completed. As we can see here that the Azure Resource Group deployment successfully completed– it’s green. So if I flip back to the Azure portal and refresh it, I can see my SQL server IS instance deployments completed, my virtual networks provision, NSG rules provision, public IP storage– all of that in under eight minutes using an ARM template that is provisioning that I created as part of setting up my DevTest Lab, and then I’m using that template to provision stuff outside the DevTest lab as well.
So if I go back in here now, in this environment I’ve got an end point, which is my way into interacting with the web application that’s hosted on the IIS instances. I think it’s this one, or the Load Balancer– Web Server Public IP Address. So if I take this DNS now, I could actually come back into my pipeline here and further build on it, where after deploying the environment I can add another task in there to maybe do some load testing or some ping testing.
Availability Testing or– reload. Yeah, I think that’s the one– Cloud Load Test. So I can just simply specify the Load Test that I want to run, and then from that point onward specify a few more settings. And then after the environments provision, it will use the end point and then run some tests on it. So this is the thing that we’ve been talking about over the last few videos– is you’re starting to bring your infrastructure, operations team into the same tools and to the same processes as that used for development.
And when they converge into single pipelines, and they’re working in collaboration, it’s very, very easy to identify configuration drift and bad things that happen usually later on in the application lifecycle. You’re just pushing that left now. And when you can identify stuff quicker, you can fix it quicker. And that’s the whole goal. So just to recap, Azure DevTest Labs allows you the acceleration to set up environments for development, give your CIOs and your security folks peace of mind, so they don’t have to worry about runaway costs. They don’t have to worry about things like exposing networks, because they can set up the governance layer for you.
For your developers, it gives them a way of getting environments very quickly, but automating from day one. And then for your operations team, they can take that automation and use it for their benefit, using the same tools to deploy to non-dev test environments as well, all the way to production.
The last couple of videos demonstrated the automation capabilities within DevTest Labs. We looked at deploying a VM that automatically generates an ARM template that you can reuse in your deployment pipeline.
The automation that you perform within your DevTest lab is not just locked down to the DevTest lab. In this step, we’ll explore the Bring Your Own Template (BYOT) feature within DevTest Labs and the capabilities it brings.
DevTest Labs in the CI/CD Toolchain
If you are using VSTS, there is an Azure DevTest Labs Tasks extension that allows you to automate your release pipeline in Azure DevTest Labs. Some of the uses of this extension include:
- Creating and deploying a VM automatically and configuring it with the latest build by using Azure File Copy or PowerShell VSTS tasks.
- Automatically capturing the state of a VM after testing to reproduce a bug on the same VM for further investigation.
- Deleting the VM at the end of the release pipeline when it’s no longer needed.
For other CI/CD toolchains, all the previously mentioned scenarios that can be achieved through the extension of the VSTS tasks can be similarly achieved through deploying Azure Resource Manager templates using Azure PowerShell cmdlets and .NET SDKs. You can also use REST APIs for DevTest Labs to integrate with your toolchain.
Complex Environments
If an environment contains multiple VMs that need to be created in a single step (for example, a multitier Web app or a SharePoint farm), you can use Azure Resource Manager templates to define the infrastructure and configuration of your Azure solution. The templates also allow you to repeatedly deploy multiple VMs in a consistent state.
Benefits of this feature include the ability to load templates from your source code repository, the ability for users to create an environment by choosing an Azure Resource Manager template from the portal after configured, and the ability to provision Azure PaaS resources from a template, as well as IaaS VMs.
Other benefits include cost tracking of environments and individual VMs, and VM policy control across environments the same way that you have for single-lab VMs. For more information on how to configure Azure Resource Manager template repositories and create environments from a template, see Create multi-VM environments with Azure Resource Manager templates.
For best practices and tips and tricks for getting the most out of DevTest Labs, see the Azure DevTest Labs Team Blog.
Join the discussion
Explore the Azure DevTest Labs Team Blog. Share a key learning with the group and explain how you’d apply this learning in your DevTest Lab.Use the Discussion section below and let us know your thoughts. Try to respond to at least one other post and once you’re happy with your contribution, click the Mark as complete button to move on to the next step.
This article is from the online course:
Microsoft Future Ready: DevOps Development, Implementation and Azure Automation
Created by
This article is from the free online
Microsoft Future Ready: DevOps Development, Implementation and Azure Automation
Our purpose is to transform access to education.
We offer a diverse selection of courses from leading universities and cultural institutions from around the world. These are delivered one step at a time, and are accessible on mobile, tablet and desktop, so you can fit learning around your life.
We believe learning should be an enjoyable, social experience, so our courses offer the opportunity to discuss what you’re learning with others as you go, helping you make fresh discoveries and form new ideas.
You can unlock new opportunities with unlimited access to hundreds of online short courses for a year by subscribing to our Unlimited package. Build your knowledge with top universities and organisations.
Learn more about how FutureLearn is transforming access to education
