Home / IT & Computer Science / Business Technology / Microsoft Future Ready: Continuous Integration Implementation / Infrastructure as Code
Infrastructure as Code
.
Welcome back. In this video, we’re going to talk about infrastructure as code using ARM templates. Infrastructure as code is important. You have to treat your infrastructure as code if you’re going to stand any chance of scaling your infrastructure on demand. Because scaling infrastructure manually is hard. And you don’t want to be that engineer that
gets called at 2:00 AM when there is a server breakdown in production, not knowing how you are going to set up the infrastructure from scratch. Because, hey, it’s not documented. And it was done a few years back. And no one really has the knowledge of how to redo it. Simply being able to provision infrastructure from code means, if you run into those scenarios, then you have a full backup resilience in place to spin up infrastructure from scratch. Now, there’s this really cool thing called Chaos Monkey that is quite popular in the open source, where high-performing IT organisations will consciously kill their server instances in production to see their ability to recover from failure.
And that’s a good way of testing whether your infrastructure provisioning scripts are robust enough to allow you to recover from such failures. I think this was a concept that was started by Netflix, where it would randomly kill a few servers from the collection, a pool of servers that it has that run its services and applications, and see whether the application can, one, survive such failure, and two, whether they can quickly re-instantiate new infrastructure to bring back the application to its full capacity to handle the tolerance of the traffic. In this specific video, we’re going to cover infrastructure as code.
And what I’m going to do in the next few videos is, rather than just take you directly into Release Management I’m going to set up the infrastructure as code in build pipeline so I can show you the tasks and explain things to you. And then, towards the end of the module, we’ll pull all these individual tasks that we create for infrastructure as code, infrastructure code provisioning, functional tests, and performance test, into a single release pipeline. And we’ll nicely break it down into phases as well, and show you the capabilities of phases. So in terms of infrastructure as code, you know, it goes without saying that you should run your infrastructure as code.
Because the benefits are you have the ability, then, to version control your infrastructure, leverage the same capabilities of Git Hooks, pull reviews, pull request-based reviews. And then aligning it along with the application– so if the application is going to have a new feature for which you are designing new infrastructure, it just naturally makes sense to bring both of them together in a single repository so that if you roll forward or roll back, you’re working on the same version of the code as well as the infrastructure code. Then the other benefits are the integration. It kind of breaks the silo. Why should infrastructure teams and development teams be in isolation? It’s all about the convergence.
It’s all about the same people, process, and tools to deliver continuous value. It does increase the predictability. So when there are failures, it improves your ability to recover from it. But most of all, for me, it saves you from the snowflake server catastrophe, where servers are just so limited that you kind of have to feed them separately and nurture them separately. Because it’s so hard to provision servers that you want to preserve the ones that you have. And some may have a different version of SQL installed. Some may have a different version of the OS installed. Some may have different versions of service packs installed.
And so they become so isolated in nature that they’re kind of a snowflake in their own right. And one server doesn’t match the configuration of the other server, which means, when you’re taking a release package from one environment to the other and things fail, you don’t know whether the failures are because your application has a bug or because the environment has a bug because it’s so misaligned from the other environment that you tested on. Again, going full infrastructure as code saves you from this, where you could spin up infrastructure in a consistent way across all environments, which saves you from having to diagnose issues that could be so infrastructure related and so less application specific.
All right, without further ado, we’ll look into ARM templates and see how we can, in the existing summit solution we have, create a new infrastructure project, and then see if we can add the capabilities to deploy a website and a SQL server database using ARM templates. OK, so I’m going to slip right into Visual Studio here. And as you can see, I have the Summit Solution open. I’ve got the web project, the test project, the DB project. And I’ve added an infrastructure project. Let me show you what I’ve done to do that. So I right-click the solution and decided to add a new project. And then, from within this section, you would see a subcategory called Cloud.
Now, if you don’t see the Cloud category here or you don’t see these templates show up, that’s because you don’t have the Azure SDK installed. You can download the Azure SDK from Azure.com. And once you instal it, you will start to see these templates. Now, these templates are really useful because they give you a readymade format that you can use to quickly create environment provisioning definitions. So say, for example, if I select the Azure Resource Group template– let’s call it Sample– click OK. Then it presents me this dialogue saying, OK, you decided to create a new resource group within Azure. What is it that you want to set up within that resource group?
And you have a few options here, like do you want to set up a web solution, or do you want to set up a web and SQL solution? Do you have a web app that has a content delivery network behind it? And this is just the Microsoft-provided Visual Studio templates. If you flip across to the GitHub-repository-based templates, there are thousands of templates that have been provided by the open-source repository on GitHub. It’s called the GitHub Azure Quickstart Templates. And any new feature or any existing feature within Azure, whether it’s search, blockchain, pretty much all of these have a pre-baked ARM template available.
So if you’re just wanting to import those or take inspiration from them as you design your own template, then it really makes the job easy. So if I go back and select the Visual Studio template, and I search for “web,” then, in the project that I’ve created here, I’ve simply select “Web app + SQL” combination and clicked OK. And when I do that, it generates a whole lot of code for me. And let me bring this up for you here. It generates this, saying– let me bring up the JSON outliner. So it divides the template into three key sections.
One is parameters, where I can specify the values that I want to be injected as part of the environment provisioning process. And some of the things there could be– I’m creating a website. Sure I want to pass the name of the website, right? That must be a parameter. What kind of a SKU do I want to set up the website on? Do I want to have multiple servers, or just one shared tenant? That’s a thing that I can pass from here. What is the admin logon and the password? What should be the database name? What collation should the database have? What should be the predefined set of the database? Should it be 10 gig, 5 gig, something more?
These are options that you can pass as parameters. While it gives you a full list of parameters, you have the choice of going in and adding more parameters, customising the existing ones. In this case, I didn’t have to customise anything. The ones that are provided are good enough to set up a simple setup of a web and a DB on the cloud. In addition to that, you have the variables. And the variables allow you to take the parameters and customise them further. So say, for example, I might want to create a unique resource group name.
So there is a function that’s provided that I can use in my Variables section to say, function unique name, and then give it a parentheses and the name of the variable. And it will use that to generate a unique name. You can refer to the videos that we already have available for infrastructure as code, which go into a lot of detail on how to construct ARM templates, what are the individual building blocks, and talks about how you can write templates from scratch as well as use the existing ones out there. The key section here, for me, is the Resources section, where it’s given me the resources that I need.
In addition to creating a SQL server and a website for which it automatically adds the code, it gives you some predefined rules as well. So like in this case, you can see it says, server error alert rules. So any time there’s a server error, and App Insights alert gets generated. So not only can you define the infrastructure here, not only can you define the networking, but you can actually define the alerting rules as well. And I think, if you’re going full-on cloud, then this is something that you should collectively look at and agree, within the team, to say, what are the basic set of rules that we need in place, and then keep evolving that.
And if you keep this version in a source-controlled repository, as you’re adding new features, you can review the rules that make sense, and keep evolving them and removing the ones that don’t make sense over time. So with that, I’ve got this template in place. In addition to this, I’ve got this other file that gets generated. It’s called the parameter file. And the parameter file allows me to inject a few values into the variables and the parameters that we saw here. This is a pre-baked PowerShell script that you get. And if you directly invoke the script, it will run the execution using these templates and provision the infrastructure for you.
But you could right-click the project, and say deploy, and choose a new deployment. And then, within the new deployment, it takes your Azure subscription. You can select if you have multiple subscriptions. You can specify here what resource group you want to deploy this template to, which JSON file you want to use, and what is the JSON file with the parameters. And you have the option of overriding the parameters by editing it here. But as you edit it here, it very nicely saves the value for you in the Parameters file as well.
So rather than deploying infrastructure directly from Visual Studio on your machine, what you would do is you would create the project, put in the parameters, commit it to source control, and then have a build pipeline or a release phase that uses this template to provision the infrastructure for you. So now that we have this here let me go back into Visual Studio Team Services. And I’ve got the summit.master build definition. As you would see, it’s the same definition that I had before. But I’ve added three steps to it. The first step that I’ve added– and let me just comment this bit out here, because I don’t want to talk about this in the current video.
But what I’ve done here is I’ve added this task. It’s called the Azure Deploy Create task. And you can search for it by clicking Add task and search for Azure. And what this allows you to do, Azure Cloud Service Deployment, Azure Resource Group Deployment. So the resource group deployment is the one that you need. Because you’re saying you want to deploy a new resource. And the template you have has the definition of resources that need to go within this resource group that gets created. So if we look at the one that I’ve imported, it simply allows you to select your Azure subscription. It nicely asks you whether you want to have a Create operation or an Update operation.
There are other options as well. For example, you could tear down the resources that are there. You could switch off the machines, turn on the machines, restart them, delete them. All of that is available from within the task. It gives you an option to specify what is the resource group name. Now, in my case, I’m calling it Summit-sea-rg. It’s a nice naming convention, clean. So I would recommend having the -rg at the end. It makes it easier, then, to find resource group within Azure portal. Select a location. Specify the path to the template. Now the template, we have committed that in the source control. So we can simply specify where the specific template is.
And then we can specify the path to the parameter file. And that’s it. That’s it. You don’t need to do anything else here. You’re just saying incremental. So it’s idempotent. If you run this script multiple times, it will just do a sanity check to see the current state of the infrastructure in that resource group. And if it finds that there are any other resources in that resource group that are not defined in the resource template, then it will remove them. Alternatively, if you have new resources that have been added in the template that are not there, it will create them. But if nothing changes, it will just do a quick sanity check and move on. And that’s useful, right?
That means you can run this build pipeline again and again without fearing this conflict of, oh, I might end up creating more servers if I run this multiple times. The next step I have here is basically just the Azure PowerShell script. So again, if you search for this in the library, you will see you’ve got a few PowerShell options, like Azure PowerShell and normal PowerShell. The Azure PowerShell has all the Azure APIs and modules available to it, which means you can run as Azure-specific commands using this PowerShell script. And that’s what I’m doing here. I’ve written a little bit of code that I’ll explain to you what it does. I’m saying, get the Azure Resource Manager resource.
Look for this resource name that I’m using to create my resource group. And then, from within this, select all the resources that exist within this resource group. And then, for each resource, tell me the name and the type. And if the type is website, then let’s store the URL of that website as a variable. And if the resource type is a database, then store that database name as a variable.
And the reason I’m doing that is because, in the next video, I’m going to extend this pipeline then to take this dynamically generated website, take this dynamically generated database, and then pass that over to a deployment task which can then deploy my package on top of this as well, which means we can have a pipeline that can not only provision infrastructure, but remember what the infrastructure is in a variable and then pass that information to the next task, which can actually go full mile forward to deploy the package that we have created.
And then this next task I have here is simply calling out the web server name and the database name that it figures out as part of the script. So let’s run this up. It will take about a minute or two to execute. But I’ll walk you through the output when it’s ready.
OK, so the build has completed execution. Let’s go back and see what happened in the specific steps. So if I click on the Azure Deploy Create or Update resource group, I can see that it’s saying the resource group already existed. So it did not have to make any changes to it. And if I click on the option Azure PowerShell Script InlineScript then it’s saying that it was able to run my script successfully it finished the execution. So if I go into the next step, you can see that it managed to pull out the name of the web server and managed to pull out the name of the SQL server. And it stored that as a variable.
Just to show you that the resource group template that I’ve created is self-sufficient, if I go back into the Azure portal and I search for “summit”– there we are, [INAUDIBLE] resource group. And if I, within this, go and delete specific components– so let me just delete the database, and then let me delete the website as well.
And while we’re at it, why not delete the App Insights event as well.
OK, so as you can see, I mean, I’ve deleted a bit of the resource group, which means, while I have a resource group and a plan, I don’t have a website or a database to reside on my SQL server instance now. Because it’s idempotent, if I go back and queue a new build and leave it running, when the execution completes, we should have those new resources restored. And we’ll come back and look at it when it’s ready.
All right, the build is completed. So if we look at the task now, which was create or update, successfully deployed. If we go back and refresh this here, refresh this here, you can see that it’s restored the website and it’s restored the database. So if I click on the website now, and I click on the website, I can navigate into the website. And you can see that it’s provisioned an empty website, which is just the default content that you would have in an Azure web app. And then if we go back to the resource group and we click on the database, the database is also empty at this point. It doesn’t have any content in it.
It’s just an empty canvas. So as you can see here, if we look at the output from the PowerShell script, it’s correctly detected the name and it’s correctly detected the name here. So all in all, in this pipeline, what we’ve seen is how easy it is to create an infrastructure project within Visual Studio, adding it to an existing solution you have so that your code for provisioning the infrastructure sits alongside your code to deploy the application. Using the existing templates, the ones provided within Visual Studio and the ones that are available to the open source, you can create infrastructure using pre-canned templates in a matter of minutes.
And then how easy it is to create parameters and bring those across into a build definition and runnable definition, which uses the Azure resource group deployment task to provision infrastructure in your Azure subscription. And in addition to that, then creating some ad hoc PowerShell scripts to query those resource groups, take the parameter values back, and store them as variables in your build definition. Pretty exciting. Because in the next video, we’ll see how we can layer our application on top of this infrastructure to deploy the components of SummitWeb and SummitDB into this newly-provisioned infrastructure.
In the previous steps, we gained a high-level understanding of provisioning and deprovisioning resources, environments and templated virtual machines. Here we are going to discuss Infrastructure as Code using Azure Resource Manager (ARM) templates.
If you are considering scaling your infrastructure and scaling your infrastructure on demand, Infrastructure as Code is important. Scaling infrastructure manually is complex and difficult to maintain.
Being able to provision your infrastructure from code means that if you run into server issues or other problems, you have a full back up available that allows you to provision your infrastructure from scratch.
Create a New Infrastructure Project with Azure Resource Manager Templates
To create a new infrastructure project with Azure Resource Manager Templates, follow these steps:
-
From your build (in this video we refer to Summit Solution), right-click to add a new project. You will see a subcategory, Cloud.
-
In your new project, you can select the Web App + SQL combination and click OK. This will produce some code within your project that you can use to specify the necessary parameters.
-
In the Resources section that gives you all the resource that you will need. You will see that the code for a SQL Server and a website has automatically been added, but it also gives you predefined alerting rules.
-
Now that your template is set up with the necessary parameters, you can commit your template to Source Control and create a build pipeline or a release phase that uses this template to provision your infrastructure for you.
Once your template is set up, you need to link your template with your build definition. To do so, you can navigate to Visual Studio Team Services. You will now some steps to your build definitions:
-
First you can add the Azure Deploy Create task. This allows you to do Azure Cloud Service Deployment and Azure Resource Group Deployment. Azure Resource Group Deployment will allow you to allocate your template with resource definitions in it to your resource group.
-
Next you can add an Azure PowerShell script. This has all the Azure APIs and modules available to it so that you can run Azure specific commands using this PowerShell script.
-
The last task that you will add simply calls out the webserver name and the database name that it obtains from the script.
It is time to run the app and review the outputs.
Join the discussion
Have you tried to a new infrastructure project with Azure Resource Manager Templates? Have you gathered any insights from the outputs you reviewed after running the app?Use the discussion section below and let us know your thoughts. Try to respond to at least one other post and once you’re happy with your contribution, click the Mark as complete button to check the step off, then you can move to the next step where we will explore Databased Deployments.
This article is from the online course:
Microsoft Future Ready: Continuous Integration Implementation
Created by
This article is from the free online
Microsoft Future Ready: Continuous Integration Implementation
Reach your personal and professional goals
Unlock access to hundreds of expert online courses and degrees from top universities and educators to gain accredited qualifications and professional CV-building certificates.
Join over 18 million learners to launch, switch or build upon your career, all at your own pace, across a wide range of topic areas.
