Terraform vs Databricks Asset Bundles (DABs)

azure3 min read

This comparison defines ownership boundaries for Terraform and Declarative Automation Bundles (DABs). Databricks previously called DABs "Databricks Asset Bundles."

For the related platform guidance, read Databricks platform lessons. Use the Entra ID and SCIM guide to check account groups and workload identities before deployment.

Current deployment model#

Use the direct deployment engine for bundles. It calls Databricks APIs through the Go SDK and does not depend on Terraform. Databricks plans to disable the old Terraform deployment engine.

The direct engine supports more resources than the tables below assign to DABs. The boundary is an ownership choice based on change risk and repository access. It is not a product limitation.

bundle plan -o json reports field changes. bundle deploy --plan plan.json replays an approved plan.

Move an existing bundle to the direct engine#

CLI 1.14.0 and later can automatically migrate a Terraform-engine bundle during deployment after a clean conversion check. Check the CLI version and current engine before the next deployment. To migrate manually:

  1. Deploy the current configuration with its existing engine.
  2. Run databricks bundle deployment migrate -t <target>.
  3. Inspect databricks bundle plan -t <target> for unexpected changes.
  4. Resolve differences before the next deployment.
  5. Deploy the bundle to synchronize the converted state to the workspace.

With the direct engine, a removed field returns to the resource default on the next deployment. Keep an explicit value in the bundle when it must persist. Check the plan for changes to schedules, notifications, and other operating settings. See engine migration and field behavior.

The Boundary Rule#

Terraform = "What does the workspace look like?" (infra, identity, catalog structure, security) DABs = "What runs inside the workspace?" (jobs, notebooks, pipelines, workload compute, secrets)

Keep in Terraform (infrastructure/platform layer)#

Resource Type Why Terraform
Azure VNet, Subnets, NSGs, NAT Cloud networking
Azure Resource Groups Cloud infra
Azure Storage Account (ADLS Gen2) Cloud storage
Databricks Workspace Platform provisioning
Access Connector Azure IAM bridge
Azure Role Assignments Cloud IAM
Entra ID Groups Identity provider
Databricks Account Groups Account-level identity
Unity Catalog Metastore Account-level UC
Storage Credential Account-level UC
Catalog Platform-level UC object
Schemas + External Locations Platform-level UC objects
Grants (metastore, catalog, schema, external location) Security and governance. Read Unity Catalog grants first.
Volumes (external, managed) Platform storage
Shared Compute Clusters + Permissions Centrally managed interactive compute
SQL Warehouses Centrally managed SQL compute
Cluster Policies + Permissions Shared compute guardrails

Catalog grants need one owner. A bundle changes grants only when its resource declares them. Terraform databricks_grants is authoritative for all grants on one securable. databricks_grant is authoritative for one principal. Do not let both tools manage the same principal on the same object. Read Unity Catalog grants for the full rule.

Move to DABs (application/project layer)#

Resource Type Why DABs
Jobs Workflow orchestration tied to application code
Source files and artifacts Notebooks, Python wheels, and JAR files deployed with the workload
Secret Scopes Application secrets tied to specific pipelines or jobs
Job Permissions Tied to job lifecycle
Lakeflow Spark Declarative Pipelines (formerly DLT) Data engineering workflows
ML Experiments / Models Data science workflows

Bundles define workload compute. They use lookups to reference shared clusters, SQL warehouses, and cluster policies that Terraform owns.

Ownership rules#

  1. One resource, one owner — do not define the same remote resource in Terraform and a bundle.
  2. Repository access affects the boundary — keep sensitive resources in a restricted Terraform repository.
  3. Workload autonomy — bundles let workload teams manage schedules and workload compute.
  4. Use each tool for its assigned scope — use bundles for workloads and Terraform for the platform and cloud infrastructure.

The community migration case study also covers how to bind existing resources to a deployment. Assign one deployment owner before you bind a resource.

Official sources#