Exploring the Azure Control Plane vs Data Plane & What It Means For Your Terraform Modules (Plus, A Free Mastercard Shoutout)

A couple of months ago while working on automating some Azure AI deployments, specifically Azure AI Search, we were going through the usual routine - pull up the azurerm Terraform docs, locate the search service resource, and... notice a surprisingly short set of input variables. "Ah, this one is going to be fun." The realization hit quickly: the Control Plane isn't enough to get this done. The Data Plane needs to be in the mix as well.

If both of those phrases seem foreign to you, here's the TLDR. The control plane is Azure Resource Manager (ARM) territory. This is where you manage the lifecycle of Azure resources themselves - creating them, updating their configurations, scaling them up or down, destroying them. When you provision an Azure resource with the azurerm provider in Terraform, you're working with the control plane. In fact, most interactions you have with Azure on a day-to-day, be it in the CLI, the ARM Rest API, or via IaC, are all through the control plane.

The data plane is where you interact with what the resource actually does. For Azure AI Search, this means everything that makes it a search service - the indexes that define your data structure, the indexers that populate them, the skillsets that enrich your data, the datasources that feed the whole thing. These operations don't go through ARM. They go through the resource's REST API directly. Different endpoint, different authentication, different permission model.

Once you start looking for this control/data plane split, it shows up all over Azure. Storage Accounts, Key Vault, Cosmos DB, Azure Load Testing - they all follow this pattern. Provision the infrastructure through ARM, configure the functionality through the service's REST API.

The azurerm provider does have some resources that create data plane objects - like azurerm_key_vault_secret for writing secrets or azurerm_storage_blob for uploading blobs. But this coverage is selective. For something like Azure AI Search, the data plane resources simply don't exist in the provider.

Here's another wrinkle: the permission models are completely different. You might have Contributor rights (control plane) but not be able to create indexes (data plane). So the next time you’re wondering why you still can’t do the things you expect to do after being granted Contributor (please use a custom role with the actual permissions you need instead), it may be that you don’t have the right Data Plane role assignment.

So, what does that mean for your Terraform modules?

The short answer: you manage data plane items via the REST API for the specific resource. Every Azure service with a data plane has its own REST API documentation. For Azure AI Search, that means using the Search REST API to create indexes, indexers, skillsets, and datasources.

You could handle these REST calls outside of Terraform entirely - Azure CLI scripts, PowerShell, whatever works. But if you're committed to infrastructure-as-code (and you should be), here’s one I really like: the Mastercard REST API provider.

It’s a generic Terraform provider that lets you make authenticated REST calls as first-class Terraform resources. It supports full CRUD operations, maintains state just like any other resource, and works with the standard plan/apply workflow. Honestly, this is one of the best open source providers we've come across.

Using this approach means everything stays in Terraform - single workflow, single state file, single source of truth. No manual portal updates, no environment drift, no "wait, which configuration is correct?" conversations. The upfront effort is worth it when your resources ends up version-controlled, peer-reviewed, and consistently deployed across all environments.

So, next time you're looking at a resource in Terraform that doesn't have all the variables you need, you may need to see if you need to interact with the Data Plane.

As always, if any of this is interesting, or your team is working through a similar set of issues, reach out! Would love to have a chat.