Learn more about this course.

Provisioning Environments with Azure DevTest Labs

In this article, we’ll explore the key features of DevTest Labs and how DevTest Labs promotes scalability.

16.9

Hello, and welcome back to this video on Azure DevTest Labs. The latest offering for doing dev and tests within Azure. Very popular. Has had a lot of success. Was formally announced at the Connect event in 2016. That’s when it was made globally available. And since then, there have been so many features added to it. But before going into a lot of detail on DevTest Lab, what DevTest Lab is and what it can do for you and your organisation, my first question to you is, what is the first thing that you started doing in the cloud? And a lot of the CIOs answer by saying dev and test.

57.8

So for a lot of the companies, the first step into the cloud is usually dev and test. And it’s a fair idea, right? You kind of can test the waters. You can baseline the latency, the performance, see whether your teams have the skills to deal with cloud, and kind of do it at your own pace. It’s low risk. So Microsoft has definitely studied that space for a while, and they’ve seen that dev and test is probably the first set of workloads that people put in the cloud. Then how about making it easier for people to do it? Now it’s never been hard setting up virtual machines in the cloud.

91.3

It takes a couple of clicks and you have a virtual machine. What’s hard though is when you give that access to a lot of people and they start creating a lot of virtual machines, then you’re kind of worried, right? Because you could have runaway costs. You’re kind of concerned about security because these guys or people doing dev and test have actually admin access on the machine, so they can open up Firewall ports, which may just bring connectivity back into your on-prem environments. So security is a concern there. Runaway cost is usually a concern. And governance, right? You want someone to own that lab in a way where you kind of are assured that it’s managed.

129.9

At the same time, you don’t want to open up a new role within your organisation to bring someone just to manage the cloud for dev and test. So DevTest Labs is a hosted service provided within Azure which exactly addresses this issue. What it does is it gives you a security wrapper whereby IT administrators can lock down the network– can define the governance and security policies. It gives you a way of tracking cost spend, resource utilisation so that CIOs aren’t worried anymore about the runaway cost. But at the same time, within the lab it gives development teams full autonomy so they can do stuff fairly quickly. At the same time, what it also gives them is automation.

177.6

DevTest Labs is created ground up to enable automation. It basically gives you a way of accessing marketplace images and then being able to create VMs from them in a matter of minutes. And because these VMs are in the same network, they can communicate with each other quite easily, so you don’t have to work that out. At the same time as you develop software, you can share software between VMs by creating artefacts. And then once you get your VM into a certain position where you want to scale it to other people, you could just create a VHD from that virtual machine, right from the Azure portal without having to Sysprep it behind the scenes in PowerShell.

217.4

It does that for you just out of the box, so you don’t have to worry about it. And then at that point onward, you can use ARM templates to not only deploy VMs in DevTest Labs, but those ARM templates can be used for deploying more instances across Azure. So start by doing your automation within DevTest Labs, using all the features that it offers, and then very quickly scale it out across Azure. And that’s what DevTest Labs is built on. So we’re going to cover a lot of the features that DevTest Labs offer. In a walk through, we’ll do a whistle tour of some of the key features.

253

And in the following videos, we’ll dive a lot deeper into the ARM template’s capabilities within DevTest Labs, and how you can integrate that in your existing toolchain, like Team Services to automate the provisioning of infrastructure. OK, so like I talked about marketplace images, what if you’re an organisation that doesn’t want to allow the development teams to access marketplace images, because they haven’t been rubber stamped by the internal security team? Well, DevTest Labs allows you policies that can be used to lock down which image types your development teams have access to within DevTest labs. At the same time, you can select the specific marketplace images that people can access within the lab as well.

296.8

But once you have approved the marketplace images, you also get what’s called formulas. Now formula is not mathematical formulas, but in concept it’s the same thing. You can bring in some components of images. You can bring in some components of artefacts, and then you can specify how you want them assembled. And when you press a button, it just runs that set of components for you to provision a VM. Now let’s take an example. If I’m going to set up a dev machine, I need a Windows 10 image. On top of that, I need Visual Studio. I need .Net. I Need a web server.

339.4

I need PowerShell. I need Visual Studio code. And that put together is my development machine. So what DevTest Labs allows you to do is specify all of these components within a formula right from within a GUI, and then when you press that button, it just simply executes the formula to provision of VM for you. And don’t worry– we’ll go into the details of how it works by trying it out from within the Azure portal. Now as I mentioned, you get the resource and cost monitoring. This is great, because not only can you see the spend, you have the authority to action on it as well.

375.7

You can track how the DevTest Lab is consuming the resources, or how the resources are being consumed within DevTest Labs. At the same time, you can set up threshold limits, and you can set up action alerts on when those thresholds are met. For example, when you hit 50%, you could trigger a webhook which may send an email, or which may go off and run an Azure automation job to delete some of the VMs which may not be used. Whatever you can do with automation you can do. But by giving you the webhook, it enables that kind of automation workflow for you. As I said, it integrates with your existing toolchain.

413.6

So whether you’re using Git, you’re using Jenkins, Chef, Puppet– and the list goes on– it just plugs in and works out of the box there. Visual Studio Team Services also has great integration with DevTest labs. Again, we’ll go into a lot of detail on that. OK, the one thing that I do plan to cover as part of this course is show you a pipeline whereby you can simply trigger a deployment pipeline from within Team Services that will provision the machine for you, that will save the image of the machine for you, and then delete it. Now let’s put this in a context of a real IT scenario.

451.7

You know, you’re kind of finishing off development, and what you want to do is have a process where every night your automations run as a nightly job. The infrastructure is provisioned, the application is deployed, your regression or performance tests or whatever tests are run, and if everything goes well, then just delete the image of the environment and mark that a success. But if something fails, you want to preserve the image of the failure so that when you come back to investigate tomorrow, you can spin up a new environment right from that image. And by using DevTest Labs and this Team Services integration, you can absolutely enable scenarios like these. All right, a lot of talking. Let’s get into a demo.

497.7

I’m going to flip right into the Azure portal, and we’ll start off by creating a new DevTest Lab.

505.4

All right, so we’re right in the Azure portal now. Let’s click on the More Services section at the bottom here, type DevTest Labs. You click favourite. It will become part of the left navigation panel there. Let’s click on DevTest Labs. I already have a lab in here, but for demo, we’ll just walk through the steps for creating a new instance of a lab. Press Add. Give it a logical name. In this case, let’s call it IAC demo lab. Now you can select the subscription, you can select the location. The great option that we’re talking about is the auto shut down.

540.8

As part of creating the lab, you get the option to select what is the default shut down and start times for all the VMs that get provisioned as part of this DevTest Lab. Now optional, you can shut down the functionality if you don’t need it, but I would encourage you to use it. Now in this case, my developers are not

559.8

going to work beyond 7:00 PM, and they’re probably

562.2

not going to start before 8:00 in the morning. So I can set up a shut down and start interval based on that. The other option I have here is an option to trigger a webhook 15 minutes before the machines are being shut down. So let me talk a little bit about a scenario that I used on a customer site. So let’s click Yes here. If you go into Logic Apps within Azure, you can define a logic.

589.3

Now, if you define a logic which says every time this logic app gets triggered you want to send a tweet, or you want to send a message to a Slack channel, or do something creative, then you can define that workflow within a logic app. Now a workflow that I’ve defined within this logic app–

611.2

let’s see it on the designer here– simply accepts the request, looks at the request details, and then posts the tweet. You don’t have to post a tweet before switching off a VM, but I’m just showing you the art of possible. At this point, I’m going to take the end point or the callback URL or this logic app, and I’m going to put it in the webhook URL. And I’m going to click OK. So basically every time then the process of shutting down VMs will run, 15 minutes before that activity, it’s going to call this logic app and perform whatever action the logic app dictates needs to be done. Now of course, you’re tweeting.

648.9

That’s probably not the best thing to do, but you could make it more streamlined where an email goes out to the developer to confirm whether they want to shut down the VM. And they can obviously respond back to the email, and you can receive that message within logic app. And if the message uses a certain syntax to say no, don’t switch it off, then you could postpone the switch-off process of the VM. So the options of how you configure are absolutely up to you, but the semantics of making it available are available within the platform. So let’s click the Create button here. It will start provisioning the instance of DevTest Lab, which can take about three or four minutes.

693.8

So the deployment has started here.

698

If I click the Refresh icon here, I can see that the lab is starting to be provisioned. It’s not in the Ready status yet, but if I click on the lab, I can see that a resource group is being created. If I click on the resource group, I can see the individual resources that are being provisioned. So we’ll wait for the provisioning process to complete.

722.1

It looks like the provisioning process is close to completion now, so let’s refresh to see all the resources that get created within the DevTest Lab resource group as part of the provisioning process. Behind the scenes, it provisions a few storage accounts. It creates a network for you. It creates the lab itself, and it creates a keyboard resource as well. Now, the benefit is in the traditional IT, people kind of have passwords that they will save in Outlook or in text files, because they have too many environments– too many passwords.

757.4

Well, the keyboard just addresses that problem here, where your passwords or your SSH keys to log into your DevTest VMs can be just preserved within the keyboard, so you don’t have to manage them outside. And it’s kind of secure, and it scales really well when you’re dealing with a lot of environments. See, it provisions the network for you by default behind the scenes, which takes away a lot of the challenges that come with provisioning dev test traditionally. Because what would happen is you would have to involve network engineers to set up the network for you. But in this case, that’s all done and taken care of.

790.6

Again, if you have very specific requirements for how the network should be configured, where you can all by all means come in and customise it, or associate a different network to an existing lab as well. Now these storage accounts not only store the VHD of the DevTest machines, but they will also store the SysPrepped prepped images, should you decide to create those as part of the snap shotting process. So these are all the basic resources that get provision as part of DevTest Lab. So rather than walk through this IAC demo lab, I’m going to flip over to the lab I already have so that I can show you some of the resources and some of the features.

830.2

Because I’ve got a lot of VMs in here, it would make it more relevant to look at. So let’s scroll down to this section here called Settings, Configuration and Policies. Now this is the sweet spot for security, governance, resource management. You see, what you have is cost tracking. So right off the bat, I can see how much I’ve spent in this lab already this month. Well, that’s a lot of dev tests happening in there. But at the same time, it gives me a projection to say just following on from my current trend of spend this is probably what I will end at. But apart from just seeing the cost, you have the option of managing the target cost as well.

868.1

So you can decide a schedule– monthly, quarterly biannually– that you want to track the cost by. And then you could just set a target threshold. So you could say once this hits $1,500, I want to have certain actions performed. Now in this case it gives you the separate [INAUDIBLE] You could set up alerts at 25, 50, 75, 100, 125. At 125, you could decide to actually– sorry, at 100 you could actually decide to notify as well as display on chart. So you see a trend line there marking that. And what you also have is an option of webhook URL.

905.8

So when that threshold is hit, or if any of these are hit, you could trigger an action behind the scenes to call a webhook or logic app, which does some downward workflow processing within your organisation. OK, so let’s come right out from here. The other option is Cost by Resource. Now you may be spending a lot of money in the lab, but you would really like to know what you’re spending your money on. DevTest Labs gives you an option to see exactly what resources in your lab are hogging most of the spend. Like in this case, I can see well, this developer guy here, Surjeet, has spent about $21 on his machine due to the consumption.

945.4

But again, if you have certain developers creating 36 GB RAM VMs that are being underutilised but that are causing the actual spend, well, then that’s very easy to track within DevTest Labs. The other section here is the policy settings. So you can define the allowed virtual machine sizes. Now just the example of Surjeet, right, where he’s created a 36 GB RAM machine, you probably don’t need these high spec machines for dev and test. So as a lab owner, you can come in and say I only want to enable basic or standard machines or DV2 standard machines so that the developers within the lab wouldn’t see the option of these high spec machines.

983.5

Again, it’s a conversation to be had with your dev teams to understand what the requirements are from the lab, but the platform provides you the option to lock it down. If you give a developer the option to never delete a VM and always get more VMs, then they’ll probably go for that. If I as a developer have certain things running on a VM and I don’t want to switch it off, I don’t want to turn it off, I don’t want to delete it, I’m just going to come back and ask for another VM and then another VM. And that causes the runaway cost phenomena there. But DevTest labs gives you an option to lock it down.

1017.5

You could select the total number of limits within the lab to say this lab cannot have more than 40, 50, 60 VMs created within it, but at the same time, you can lock this limit down per developer as well. You can say a developer cannot have more than 5 VMs. If they need the sixth VM, they need to come back and delete what they already have to provision the next set. The other section here is Schedules. So like I talked about the option of auto start and auto shut down, it’s available and you can configure it per lab. You can even configure it per VM now. This bit here– the External Resources Repository– just park it for now.

1059

I will come back to it in a separate video. But it’s basically a way of sharing artefact within the lab. For example, if you’re creating a software, a nugget package, or a script that does some automation and you want to share that across all the VMs that are available within the lab, or you want it to be bootstrapped as part of the machine provisioning process, then repositories is a great way of doing it. At the bottom here you have the option of Creating Marketplace– selecting the marketplace images that your DevTest Lab consumers can consume. You have the option of creating custom images.

1092.9

So if you go within custom images, I’ve got two custom images that I created by selecting one of the VMs that I had set up with some bespoke software that I couldn’t automate the provisioning for. And then when I created a custom image it behind the scenes SysPrepped it and added it to my gallery. Now this model really works for consulting companies, because they usually have a Centre of Excellence teams that are creating accelerators for customer project deliveries. Now these COEs would create assets– would create custom images– that teams that were going out to customer sites to deliver projects could just use to help fast track the delivery of these projects.

1134.2

Formulas we talked about briefly– having the ability to create a recipe and a way to say how your machine gets provisioned. Well, we’ve got a few formulas here. As you can see, this formula basically selects the type of machine and specifies the list of artefacts that need to be executed to provision a dev VM for us. Again, you get an out of box library from DevTest Labs, which is open source– which the Microsoft product team is contributing to, which the open source is contributing to. But you also have the option of associating your own artefact libraries. Like in this case in my subscription, Avanade is also providing one of these that I’ve associated with the lab.

1179.6

So some of the internal stuff we’re creating, we’re internally consuming without making it public. All right, so take a step back. See if we have anything else worth covering here. You know, the bit at the bottom is just [INAUDIBLE] role-based access so you can define who has permissions to create stuff within the lab, administer stuff within the lab, or who’s just simply in a role to consume the stuff that’s been created. Again, for enterprise it’s quite important to have that fine-grained control within the lab.

1211.5

OK, there’s just one more thing I wanted to cover, which is called Claimable VMs. So let’s go back right into the lab and see what Claimable Virtual Machines are. In certain organisations, there is a central department that’s responsible for provisioning stuff, or setting stuff up. Claimable Virtual Machines just is a great idea where this central unit that’s provisioning stuff for you can mark the VMs that are being provisioned as claimable. Which means when you come into this section, you can see all the environments that have been pre-provisioned for you– maybe by someone else or by an automation tool– that still haven’t been picked up by someone else for consumption.

1257.2

So you can come in into this section, look for a workstation that fits your needs, and if it’s available you could just right click and say claim. And then it becomes your machine. And then you can start to see it within the My Machine section. You know, I kind of find it really useful. We at a customer site had a process that ran every night to provision dev VMs for development teams. And the developers would simply come back in the morning, right click, and claim those VMs. It’s also useful when you’re testing stuff. You have some three, four test environments that you create in the morning.

1287.5

And then as part of your testing activity throughout the day, you can claim some of those environments, do your testing, and then unclaim those environments so that they’re back in the pool again. So that was a quick whistle tour of DevTest Labs. It was just an introduction. We haven’t even looked at the deeper automation capabilities that it offers, but promise to cover that in the next video.

Up to this point in the DevOps Development Process Implementation course, you’ve learned that Azure offers automation solutions that can help you streamline your development and deployment processes.

By now, you can create and run templates in Azure instead of writing endless code and scripts. But what if you want to scale your development? In this article, we’ll explore the key features of DevTest Labs and how DevTest Labs promotes scalability.

What is DevTest Labs?

Although it’s always been easy to spin up virtual infrastructure in the cloud, to realise the benefits at scale, consumers need development and test labs that lend themselves to reuse, standard and secure setup, enterprise policies and proactive cost tracking. Azure DevTest Labs is an Azure service that addresses and accomplishes these goals.

Want to keep
learning?

This content is taken from
CloudSwyft Global Systems, Inc. online course,

Microsoft Future Ready: DevOps Development, Implementation and Azure Automation

View Course

DevTest Labs expedites the configuration of self-service Azure environments by allowing developers and testers to quickly provision Windows and Linux environments in Azure by using reusable templates and artefacts.

DevTest Labs is also capable of inheriting Azure marketplace images. The marketplace has a large collection of open-source product images that can be used to quickly provision infrastructure in the lab.

The automation assets and components created within the Azure DevTest Labs can be invoked by using out-of-the-box integration between Azure and Visual Studio Team Services (VSTS). Deployment pipelines within VSTS can be used to provision dev and test machines in the lab on Azure. The newly provisioned machines automatically inherit the permissions and the lab policies, such as shut-down and start-up routines.

You can create pre-provisioned environments by claiming the environment of the last good build of an application and using it immediately. Source control can be used to manage and share captured environment templates and artefacts. The API or preexisting plugins can be used to provision the environments directly from a preferred continuous integration tool or automated release pipeline.

Provisioning a New Lab

To create a new lab in Azure DevTest Labs:

Go the portal.
Select More services > DevTest Labs.
On the DevTest Labs blade, select Add.
On the Create a DevTest Lab blade, fill in the parameters and select Create:

The following resources are created when you provision a DevTest Lab:

Storage accounts are provisioned
The network is created
The lab is created
A Key Vault or Secret Vault is created.

Key Features

Key/Secret Vault

You’re often asked to enter a complex secret when using Azure DevTest Labs: The password for your Windows VM, a public SSH key for your Linux VM, or a personal access token to clone your Git repo through an artefact. Secrets are usually long and have random characters, so it can be difficult to ensure accuracy.

DevTest Labs provides a personal secret store for each lab user to keep your secrets safe. In the store, you can save your secret into the key vault with a name that DevTest Labs creates for you and use it later.

In the DevTest Labs portal, when a secret is needed to create a VM or to use an artefact, in addition to an input box asking for the secret, you’ll see a checkbox where you can select to use the secrets from the secret store. After a secret is selected, the input box will turn into a drop-down list where you can simply pick a secret:

To save your secret into your secret store, go to My secret store and enter a name and your secret as the value:

In addition to creating VMs and using artefacts, you can save your secret name with formulas and Azure Resource Manager templates. Here’s an example of how it’s used in a Resource Manager template to provision a new lab VM:

Cost Tracking

Your DevTest lab allows you to see how much you’ve spent in your lab and it gives you a projection of your current trend of spending. Apart from just seeing the cost, you have the option to manage your target cost as well.

You can schedule your tracking and set target thresholds for how much you want to spend. When your cost reaches the threshold, you can have certain actions performed within your lab such as notifications and displaying charts with trendlines. You also have the option of using a webhook URL. When the threshold is reached, you can trigger an action behind the scenes to call a webhook or a logic app, which does some downward workflow processing within your organisation.

Lab Policies

Azure DevTest Labs allows you to select policies to control cost and minimise waste in labs.

Allowed VM Sizes

The policy for setting the allowed virtual machine (VM) sizes helps to minimise lab waste by enabling you to specify which VM sizes are allowed in the lab. If this policy is activated, only VM sizes from this list can be used to create VMs.

VMs Per User

This policy allows you to specify the maximum number of VMs that can be created by an individual user. If a user attempts to create a VM when the user limit has been met, an error message indicates that the VM cannot be created.

VMs Per Lab

This policy allows you to specify the maximum number of VMs that can be created for the current lab. If a user attempts to create a VM when the lab limit has been reached, an error message indicates that the VM cannot be created.

Auto-Shutdown

This policy helps to minimise lab waste by allowing you to specify the time that the lab’s VMs shut down.

Auto-Start

This policy allows you to specify when the VMs in the current lab should be started.

External Resources Repository

With the external resources repository, you can share artefacts within your labs such as sharing automation scripts across all VMs in your lab or bootstrapping machines that are being provisioned.

Marketplace

You can create a marketplace where you can share your DevTest lab. Consumers can consume all your marketplace images and custom images.

Formulas

A formula is essentially a recipe of how your machines must be provisioned.

Role-Based Access

RBAC allows you to control who has permission to create VMs and artefacts within your lab, and who simply consumes the VMs and artefacts that were created.

Claimable VMs

You have the opportunity to set up claimable VMs for consumers in your environment to see which VMs are available for use.

Want to keep learning?

This content is taken from CloudSwyft Global Systems, Inc. online course

Microsoft Future Ready: DevOps Development, Implementation and Azure Automation

View Course

See other articles from this course

This article is from the free online

Microsoft Future Ready: DevOps Development, Implementation and Azure Automation

Created by

Join Now

Reach your personal and professional goals

Unlock access to hundreds of expert online courses and degrees from top universities and educators to gain accredited qualifications and professional CV-building certificates.

Join over 18 million learners to launch, switch or build upon your career, all at your own pace, across a wide range of topic areas.

Start Learning now

Short courses

ExpertTracks

Microcredentials

Online degrees

Learn more about this course.

Provisioning Environments with Azure DevTest Labs

What is DevTest Labs?

Want to keep
learning?

Microsoft Future Ready: DevOps Development, Implementation and Azure Automation

Provisioning a New Lab