January 2, 2024 Marco

The power of Azure Policy DeployIfNotExists

In a nutshell, Azure Policy is a powerful tool for maintaining control and governance over Azure resources, ensuring they meet organizational standards, security, and compliance requirements. For example, you could deny any creation of Azure resources with public endpoints or summarize all resources that do not meet certain requirements or settings.

Azure policies can be bundled into initiatives, similar to GPO’s, and assigned to one of the following scopes:

  • Management group
  • Subscription
  • Resource group
  • Resource

Each policy definition has a single effect. That effect determines what happens when the policy rule is evaluated to match. Effects behave differently if they are for a new resource, an updated resource, or an existing resource. As of today, the following policy effects are available and supported:

  • AddToNetworkGroup
  • Append
  • Audit
  • AuditIfNotExists
  • Deny
  • DenyAction
  • DeployIfNotExists
  • Disabled
  • Manual
  • Modify
  • Mutate

You’ll find more information about each effect on Microsoft Learn: Understand how effects work

The DeployIfNotExists policy effect

In this blog post, we’ll lay our focus on the DeployIfNotExists policy effect. This effect is especially powerful, because, as the name suggests, it allows you to run an ARM deployment depending on certain conditions, like a specific setting or tag of the scoped resource.

Real-life use cases:

  • Automatically configure backup for blob and/or file storage, except for certain tags and values
  • Automatically configure diagnostic settings on specific resources
  • Automatically deploy a private endpoint for certain resource types
  • Automatically configure a route table for scoped virtual networks
  • Automatically enable purge protection on Key Vaults
  • and many more …

How it works

After a new resource is deployed, the policy evaluation process starts if the resource falls into the scope of the policy assignment. During the evaluation, conditions defined in the policy definition are checked, i.e. the ExistenceCondition. If the conditions are not met, a deployment using a managed identity is triggered. The managed identity will run deployment after a certain delay, the progress can be followed in the activity logs.

Example policy structure

Let’s take a look at a DeployIfNotExists policy (this is a built-in policy that can be found in the Azure portal under “Policy”). Our goal is to automatically enable backup for storage accounts (blob) into a specific backup vault. Since the onboarding of a storage account into a backup vault is a small deployment on the backup vault side, we need to use a DeployIfNotExists policy effect.

A policy definition consists of several sections, starting with general information like the name or type and the policy parameters. These are the parameters you need to specify when assigning the policy, i.e. the ID of the backup policy which in turn contains the destination vault.

{
  "properties": {
    "displayName": "[Preview]: Configure blob backup for all storage accounts that do not contain a given tag to a backup vault in the same region",
    "policyType": "BuiltIn",
    "mode": "Indexed",
    "description": "Enforce backup for blobs on all storage accounts that do not contain a given tag to a central backup vault. Doing this can help you manage backup of blobs contained across multiple storage accounts at scale. For more details, refer to https://aka.ms/AB-BlobBackupAzPolicies",
    "metadata": {
      "version": "2.0.0-preview",
      "preview": true,
      "category": "Backup"
    },
    "parameters": {
      "vaultLocation": {
        "type": "String",
        "metadata": {
          "displayName": "Location (Specify the location of the storage accounts that you want to protect)",
          "description": "Specify the location of the storage accounts that you want to protect. Blobs in the storage accounts should be backed up to a vault in the same location. For example - CanadaCentral",
          "strongType": "location"
        }
      },
      "backupPolicyId": {
        "type": "String",
        "metadata": {
          "displayName": "Backup Policy (of type Azure Blobs (Azure Storage) from a vault in the location chosen above)",
          "description": "Specify the ID of the backup policy to be used for configuring backup for blobs. The selected Azure Backup policy should be of type Azure Blobs (Azure Storage). This policy needs to be in a vault that is present in the location chosen above. For example - /subscriptions/<SubscriptionId>/resourceGroups/<resourceGroupName>/providers/Microsoft.DataProtection/vaults/<VaultName>/backupPolicies/<BackupPolicyName>. Also, make sure that this Backup vault's managed identity has the Storage Account Backup Contributor role assigned on the storage accounts for which backup is to be configured."
        }
      },
      "exclusionTagName": {
        "type": "String",
        "metadata": {
          "displayName": "Exclusion Tag Name",
          "description": "Name of the tag to use for excluding storage accounts in the scope of this policy. This should be used along with the Exclusion Tag Value parameter. Learn more at https://aka.ms/AB-BlobBackupAzPolicies"
        }
      },
      "exclusionTagValues": {
        "type": "Array",
        "metadata": {
          "displayName": "Exclusion Tag Values",
          "description": "Value of the tag to use for excluding storage accounts in the scope of this policy (in case of multiple values, use a comma-separated list). This should be used along with the Exclusion Tag Name parameter. Learn more at https://aka.ms/AB-BlobBackupAzPolicies."
        }
      },
      "effect": {
        "type": "String",
        "metadata": {
          "displayName": "Effect",
          "description": "Enable or disable the execution of the policy"
        },
        "allowedValues": [
          "DeployIfNotExists",
          "AuditIfNotExists",
          "Disabled"
        ],
        "defaultValue": "DeployIfNotExists"
      }
    },
    "policyRule": {
      "if": {
        "allOf": [
          {
            "field": "type",
            "equals": "Microsoft.Storage/StorageAccounts"
          },
          {
            "field": "kind",
            "equals": "StorageV2"
          },
          {
            "field": "Microsoft.Storage/storageAccounts/sku.name",
            "contains": "Standard"
          },
          {
            "field": "Microsoft.Storage/storageAccounts/isHnsEnabled",
            "notEquals": "true"
          },
          {
            "field": "Microsoft.Storage/storageAccounts/isNfsV3Enabled",
            "notEquals": "true"
          },
          {
            "field": "location",
            "equals": "[parameters('vaultLocation')]"
          },
          {
            "anyOf": [
              {
                "not": {
                  "field": "[concat('tags[', parameters('exclusionTagName'), ']')]",
                  "in": "[parameters('exclusionTagValues')]"
                }
              },
              {
                "value": "[empty(parameters('exclusionTagValues'))]",
                "equals": "true"
              },
              {
                "value": "[empty(parameters('exclusionTagName'))]",
                "equals": "true"
              }
            ]
          }
        ]
      },
      "then": {
        "effect": "[parameters('effect')]",
        "details": {
          "type": "Microsoft.Storage/storageAccounts/blobServices",
          "name": "default",
          "existenceCondition": {
            "field": "Microsoft.Storage/storageAccounts/blobServices/default.restorePolicy.enabled",
            "equals": true
          },
          "roleDefinitionIds": [
            "/providers/Microsoft.Authorization/roleDefinitions/5e467623-bb1f-42f4-a55d-6e525e11384b"
          ],
          "deployment": {
            "properties": {
              "mode": "incremental",
              "template": {
                "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
                "contentVersion": "1.0.0.0",
                "parameters": {
                  "backupPolicyId": {
                    "type": "string",
                    "metadata": {
                      "description": "Backup Policy Id"
                    }
                  },
                  "storageAccountResourceId": {
                    "type": "string",
                    "metadata": {
                      "description": "ResourceId of the Storage Account"
                    }
                  },
                  "location": {
                    "type": "string",
                    "metadata": {
                      "description": "Location for all resources"
                    }
                  }
                },
                "variables": {
                  "storageAccountName": "[first(skip(split(parameters('storageAccountResourceId'), '/'), 8))]",
                  "dataSourceType": "Microsoft.Storage/storageAccounts/blobServices",
                  "resourceType": "Microsoft.Storage/storageAccounts",
                  "backupPolicyName": "[first(skip(split(parameters('backupPolicyId'), '/'), 10))]",
                  "vaultName": "[first(skip(split(parameters('backupPolicyId'), '/'), 8))]",
                  "vaultResourceGroup": "[first(skip(split(parameters('backupPolicyId'), '/'), 4))]",
                  "vaultSubscriptionId": "[first(skip(split(parameters('backupPolicyId'), '/'), 2))]"
                },
                "resources": [
                  {
                    "type": "Microsoft.Resources/deployments",
                    "apiVersion": "2021-04-01",
                    "resourceGroup": "[variables('vaultResourceGroup')]",
                    "subscriptionId": "[variables('vaultSubscriptionId')]",
                    "name": "[concat('DeployProtection-',uniqueString(variables('storageAccountName')))]",
                    "properties": {
                      "mode": "Incremental",
                      "template": {
                        "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
                        "contentVersion": "1.0.0.0",
                        "parameters": {},
                        "resources": [
                          {
                            "type": "Microsoft.DataProtection/backupvaults/backupInstances",
                            "apiVersion": "2021-01-01",
                            "name": "[concat(variables('vaultName'), '/', variables('storageAccountName'))]",
                            "properties": {
                              "objectType": "BackupInstance",
                              "dataSourceInfo": {
                                "objectType": "Datasource",
                                "resourceID": "[parameters('storageAccountResourceId')]",
                                "resourceName": "[variables('storageAccountName')]",
                                "resourceType": "[variables('resourceType')]",
                                "resourceUri": "[parameters('storageAccountResourceId')]",
                                "resourceLocation": "[parameters('location')]",
                                "datasourceType": "[variables('dataSourceType')]"
                              },
                              "policyInfo": {
                                "policyId": "[parameters('backupPolicyId')]",
                                "name": "[variables('backupPolicyName')]"
                              }
                            }
                          }
                        ]
                      }
                    }
                  }
                ]
              },
              "parameters": {
                "storageAccountResourceId": {
                  "value": "[field('id')]"
                },
                "backupPolicyId": {
                  "value": "[parameters('backupPolicyId')]"
                },
                "location": {
                  "value": "[field('location')]"
                }
              }
            }
          }
        }
      }
    }
  },
  "id": "/providers/Microsoft.Authorization/policyDefinitions/958dbd4e-0e20-4385-a082-d3f20c2a6ad8",
  "type": "Microsoft.Authorization/policyDefinitions",
  "name": "958dbd4e-0e20-4385-a082-d3f20c2a6ad8"
}

You can already see that an Azure policy with this effect follows nothing other than an if/then logic inside the policyRule section. Let’s take a look at the if section, which defines what values of a certain resource must match. In this case, the deployment will only trigger if the scoped resource is of the type Microsoft.Storage/StorageAccounts in Standard sku and so on.

"if": {
  "allOf": [
    {
      "field": "type",
      "equals": "Microsoft.Storage/StorageAccounts"
    },
    {
      "field": "kind",
      "equals": "StorageV2"
    },
    {
      "field": "Microsoft.Storage/storageAccounts/sku.name",
      "contains": "Standard"
    },
    {
      "field": "Microsoft.Storage/storageAccounts/isHnsEnabled",
      "notEquals": "true"
    },
    {
      "field": "Microsoft.Storage/storageAccounts/isNfsV3Enabled",
      "notEquals": "true"
    },
    {
      "field": "location",
      "equals": "[parameters('vaultLocation')]"
    },
    {
      "anyOf": [
        {
          "not": {
            "field": "[concat('tags[', parameters('exclusionTagName'), ']')]",
            "in": "[parameters('exclusionTagValues')]"
          }
        },
        {
          "value": "[empty(parameters('exclusionTagValues'))]",
          "equals": "true"
        },
        {
          "value": "[empty(parameters('exclusionTagName'))]",
          "equals": "true"
        }
      ]
    }
  ]
},

If all the required values match, the deployment defined in the then section will trigger as a result. Take a look at lines 115 – 203. This section might be familiar to you as it is a generic ARM deployment, which means you can dive deep into whatever you like to deploy.

"then": {
  "effect": "[parameters('effect')]",
  "details": {
    "type": "Microsoft.Storage/storageAccounts/blobServices",
    "name": "default",
    "existenceCondition": {
      "field": "Microsoft.Storage/storageAccounts/blobServices/default.restorePolicy.enabled",
      "equals": true
    },
    "roleDefinitionIds": [
      "/providers/Microsoft.Authorization/roleDefinitions/5e467623-bb1f-42f4-a55d-6e525e11384b"
    ],
    "deployment": {
      "properties": {
        "mode": "incremental",
        "template": {
          "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
          "contentVersion": "1.0.0.0",
          "parameters": {
            "backupPolicyId": {
              "type": "string",
              "metadata": {
                "description": "Backup Policy Id"
              }
            },
            "storageAccountResourceId": {
              "type": "string",
              "metadata": {
                "description": "ResourceId of the Storage Account"
              }
            },
            "location": {
              "type": "string",
              "metadata": {
                "description": "Location for all resources"
              }
            }
          },
          "variables": {
            "storageAccountName": "[first(skip(split(parameters('storageAccountResourceId'), '/'), 8))]",
            "dataSourceType": "Microsoft.Storage/storageAccounts/blobServices",
            "resourceType": "Microsoft.Storage/storageAccounts",
            "backupPolicyName": "[first(skip(split(parameters('backupPolicyId'), '/'), 10))]",
            "vaultName": "[first(skip(split(parameters('backupPolicyId'), '/'), 8))]",
            "vaultResourceGroup": "[first(skip(split(parameters('backupPolicyId'), '/'), 4))]",
            "vaultSubscriptionId": "[first(skip(split(parameters('backupPolicyId'), '/'), 2))]"
          },
          "resources": [
            {
              "type": "Microsoft.Resources/deployments",
              "apiVersion": "2021-04-01",
              "resourceGroup": "[variables('vaultResourceGroup')]",
              "subscriptionId": "[variables('vaultSubscriptionId')]",
              "name": "[concat('DeployProtection-',uniqueString(variables('storageAccountName')))]",
              "properties": {
                "mode": "Incremental",
                "template": {
                  "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
                  "contentVersion": "1.0.0.0",
                  "parameters": {},
                  "resources": [
                    {
                      "type": "Microsoft.DataProtection/backupvaults/backupInstances",
                      "apiVersion": "2021-01-01",
                      "name": "[concat(variables('vaultName'), '/', variables('storageAccountName'))]",
                      "properties": {
                        "objectType": "BackupInstance",
                        "dataSourceInfo": {
                          "objectType": "Datasource",
                          "resourceID": "[parameters('storageAccountResourceId')]",
                          "resourceName": "[variables('storageAccountName')]",
                          "resourceType": "[variables('resourceType')]",
                          "resourceUri": "[parameters('storageAccountResourceId')]",
                          "resourceLocation": "[parameters('location')]",
                          "datasourceType": "[variables('dataSourceType')]"
                        },
                        "policyInfo": {
                          "policyId": "[parameters('backupPolicyId')]",
                          "name": "[variables('backupPolicyName')]"
                        }
                      }
                    }
                  ]
                }
              }
            }
          ]
        },
        "parameters": {
          "storageAccountResourceId": {
            "value": "[field('id')]"
          },
          "backupPolicyId": {
            "value": "[parameters('backupPolicyId')]"
          },
          "location": {
            "value": "[field('location')]"
          }
        }
      }
    }
  }
}

Remediations

You might ask yourself, who exactly runs the deployment in the background? What about permissions?

This is the time at which remediations and managed identities come into play. When you assign a DeployIfNotExists Azure policy at a scope, the assignment needs a managed identity to remediate non-compliant resources, in other words, you need a service principal that the policy can use to run the deployments. You can either use system-assigned or user-assigned managed identities for that.

Take a quick look at lines 112 – 114, where roleDefinitionIds are defined. These are the Azure RBAC roles, i.e. Backup Contributor in this case, which the managed identity needs to successfully run the deployment. These roles are automatically granted if you use a system-assigned managed identity via the Azure portal. If you use user-assigned managed identities, you need to manually grant these permissions beforehand. So what’s the benefit of using either one of those managed identities? Take a look at the tips below.

Tips

Here are a few last tips for your policy journey:

  • Take a look at the DenyAction to prevent deletion of resources. This brings some advantages over resource locks.
  • You need to manually trigger a remediation task for existing resources after you create a DeployIfNotExists assignment. Existing resources will not be remediated automatically.
  • System-assigned managed identities are automatically created for each policy assignment, a new one is created each time you re-create the assignment which leaves you with a lot of stale permissions. If you want to centralize managed identities used by policy remediations, use user-assigned managed identities. This is also beneficial if another higher privileged user or process needs to grant permissions beforehand.

Useful links

Conclusion

Here you have it, I hope this helps in the general understanding of Azure policies and encourages you to develop your own policies too. Azure Policy is a powerful tool to keep your environment secure, compliant, and automated. Cheers!

, ,
Marco Gerber

Marco

Senior Cloud Engineer, keen on Azure and cloud technologies.

Leave a Reply

Your email address will not be published. Required fields are marked *