Virtual machine scale sets are an Azure compute resource that you can use to deploy and manage a set of identical VMs. With all VMs configured the same, scale sets are designed to support true autoscale, and no pre-provisioning of VMs is required. So it’s easier to build large-scale services that target big compute, big data, and containerized workloads.
For applications that need to scale compute resources out and in, scale operations are implicitly balanced across fault and update domains.
A scale set can be optionally configured with autoscale settings when it’s created in the Azure portal. The number of VMs can then be increased or decreased based on average CPU usage.
Scale set performance and scale guidance
- A scale set supports up to 1,000 VMs. If you create and upload your own custom VM images, the limit is 100. For considerations in using large scale sets
- You do not have to pre-create Azure storage accounts to use scale sets. Scale sets support Azure managed disks, which negate performance concerns about the number of disks per storage account.
- Consider using Azure Premium Storage instead of Azure Storage for faster, more predictable VM provisioning times and improved I/O performance.
- The core quota in the region in which you are deploying limits the number of VMs you can create. You might need to contact Customer Support to increase your compute quota limit, even if you have a high limit of cores for use with Azure Cloud Services today.
Design Considerations For Scale Sets
Generally, scale sets are useful for deploying highly available infrastructure where a set of machines have similar configuration. However, some features are only available in scale sets while other features are only available in VMs. In order to make an informed decision about when to use each technology, we should first take a look at some of the commonly used features that are available in scale sets but not VMs:
Scale set-specific features
- Once you specify the scale set configuration, you can simply update the “capacity” property to deploy more VMs in parallel. This is much simpler than writing a script to orchestrate deploying many individual VMs in parallel.
- You can use Azure Autoscale to automatically scale a scale set but not individual VMs.
- You can reimage scale set VMs but not individual VMs.
- You can overprovision scale set VMs for increased reliability and quicker deployment times. You cannot do this with individual VMs unless you write custom code to do this.
- You can specify an upgrade policy to make it easy to roll out upgrades across VMs in your scale set. With individual VMs, you must orchestrate updates yourself.
On the other hand, some features are only available in VMs (at least for the time being):
- You can attach data disks to specific individual VMs, but attached data disks are configured for all VMs in a scale set.
- You can attach non-empty data disks to individual VMs but not VMs in a scale set.
- You can snapshot an individual VM but not a VM in a scale set.
- You can capture an image from an individual VM but not from a VM in a scale set.
- You can migrate an individual VM from native disks to managed disks, but you cannot do this for VMs in a scale set.
- You can assign IPv6 public IP addresses to individual VM nics but cannot do so for VMs in a scale set. Note that you can assign IPv6 public IP addresses to load balancers in front of either individual VMs or scale set VMs.
Scale sets with Azure Managed Disks
Scale sets can be created with Azure Managed Disks instead of traditional Azure storage accounts. Managed Disks provide the following benefits:
- You do not have to pre-create a set of Azure storage accounts for the scale set VMs.
- You can define attached data disks for the VMs in your scale set.
- Scale sets can be configured to support up to 1,000 VMs in a set.
If you have an existing template, you can also update the template to use Managed Disks.
A scale set that is not defined with Azure Managed Disks relies on user-created storage accounts to store the OS disks of the VMs in the set. A ratio of 20 VMs per storage account or less is recommended to achieve maximum IO and also take advantage of overprovisioning (see below). It is also recommended that you spread the beginning characters of the storage account names across the alphabet. Doing so helps spread load across different internal systems.
Scale sets currently default to “overprovisioning” VMs. With overprovisioning turned on, the scale set actually spins up more VMs than you asked for, then deletes the extra VMs once the requested number of VMs are successfully provisioned. Overprovisioning improves provisioning success rates and reduces deployment time. You are not billed for the extra VMs, and they do not count toward your quota limits.
While overprovisioning does improve provisioning success rates, it can cause confusing behavior for an application that is not designed to handle extra VMs appearing and then disappearing.
If your scale set uses user-managed storage, and you turn off overprovisioning, you can have more than 20 VMs per storage account, but it is not recommended to go above 40 for IO performance reasons.
A scale set built on a Marketplace image (also known as a platform image) and configured to use Azure Managed Disks supports a capacity of up to 1,000 VMs. If you configure your scale set to support more than 100 VMs, not all scenarios work the same (for example load balancing). For more information, see Working with large virtual machine scale sets.
A scale set configured with user-managed storage accounts is currently limited to 100 VMs (and 5 storage accounts are recommended for this scale).
A scale set built on a custom image (one built by you) can have a capacity of up to 100 VMs when configured with Azure Managed disks. If the scale set is configured with user-managed storage accounts, it must create all OS disk VHDs within one storage account. As a result, the maximum recommended number of VMs in a scale set built on a custom image and user-managed storage is 20. If you turn off overprovisioning, you can go up to 40.