Databricks ABAC: governed tags, row filters and PII masks

Use Databricks ABAC to apply row filters and PII column masks through Unity Catalog governed tags. Compare the access controls below, then use the SQL examples and rollout checks to test your policy. Keep stable governance controls in Terraform.

This guide covers row filter and column mask policies. ABAC also supports GRANT policies and DENY policies (Beta) for supported securable types. See the ABAC overview. The ownership split and quarterly review schedule below are recommendations, not product requirements.

Read the identity guide and the Terraform and DABs guide before you apply this governance model.

The governance layers#

1. Classification, tagging, and masking#

Use Databricks Data Classification (GA 2026-04-20) to detect sensitive data in Unity Catalog.
Use governed tags to standardize labels such as classification=restricted, pii=email, pii=phone, pii=ssn, row_security=region.
Require data-steward review before broad auto-tagging in production.
Use ABAC policies to apply masks or row filters based on governed tags.
Use DABs or SQL migrations to apply workload-specific tags when pipelines create new tables.
Use Terraform for the stable governance plane: tag policies, ABAC policies, Entra groups, grants, and stable tag assignments.

Use Unity Catalog, Entra-backed groups, and group grants as the foundation. Add governed tags, Data Classification, steward review, and ABAC policies for PII governance.

2. ABAC vs governed tags vs tag policies#

Each object has a specific role:

Governed tags describe the data.
Tag policies define allowed tag keys and values.
ABAC policies enforce row filters or column masks based on those tags.
Entra-backed groups decide who is exempt or allowed to see unmasked data.

Governed tag:
  pii=email on prod_catalog.gold.customer.email_address

ABAC policy:
  Any column tagged pii=* is masked for account users
  except approved PII-reader groups

Entra ID:
  Membership in a "PII Unmasked Readers" group
  determines whether the policy exempts the reader through EXCEPT

Rows are not the governance target. Do not tag rows. Tables or columns are tagged, and row-filter UDFs evaluate row values such as region_code, tenant_id, or business_unit.

3. Row filters and column masks#

RBAC gets a user to the catalog/schema/table.
Column masks decide whether sensitive column values are raw or masked.
Row filters decide which rows are returned.
ABAC applies those controls centrally when matching governed tags are present.

Table:   prod_catalog.gold.customer
Column:  region_code
Tags:    table row_security=region
         column security_key=region_code
Policy:  Pass region_code to governance.allow_region(region_code)
UDF:     Return true only if current user is in the matching Entra-backed group

Do not create one policy for each table. Define a small governed tag taxonomy, apply tags at the highest useful Unity Catalog object level, tag sensitive columns specifically, and use catalog-level ABAC policies so the same mask/filter logic follows the tags.

4. Terraform's role at scale#

Use Terraform for stable, repeatable governance objects:

Entra security groups and Databricks account groups.
Unity Catalog grants and service principal permissions.
Governed tag definitions and allowed values.
Catalog/schema/table/column tag assignments for stable, high-value assets.
ABAC policy resources (databricks_policy_info).
External locations, storage credentials, and managed-identity access paths.

Do not use Terraform for frequently changing tables or generated columns. These resources create noisy plans and stale governance metadata.

Terraform:            platform controls and stable governance
DABs / pipeline SQL:  workload-owned table creation and workload-specific tags
Data Classification:  sensitive-data discovery and optional auto-tagging
Data stewards:        review false positives, approve tag activation, exceptions

5. DABs in the governance model#

DABs are not a replacement for Terraform-owned ABAC.

Terraform owns the platform and control plane.
DABs own jobs, pipelines, notebooks, SQL tasks, dashboards, and deployment targets.
DABs run production jobs as service principals that Terraform created and granted.
Pipelines can apply tags to the tables they produce, but central ABAC policies stay in Terraform or a restricted governance repository.

Terraform defines the guardrails; DABs deploy the workloads inside them. A bundle can write a gold customer table. The central governance team controls Unity Catalog grants, governed tags, ABAC policies, and the service principal's access.

6. Periodic access reviews#

Make access reviews identity-led, not workspace-led:

Entra ID is the source of truth for group membership.
Databricks account groups are synced or mapped from Entra.
Terraform grants permissions to groups, not individual users.
A quarterly review exports group membership, UC grants, service principals, PAT usage, and job ownership.
Business data owners approve or remove access.
Remove access in Entra first. Then, let the change propagate to Databricks.

Review Entra group membership and service principal ownership first, then validate Unity Catalog grants and workspace permissions against those approved groups.

7. PAT tokens, service principals, and secrets#

The modern direction is workload identity federation and service principals instead of long-lived secrets:

Prefer OAuth / service principal auth for automation.
Avoid human PATs for production automation.
Restrict PAT creation and lifetime where PATs are still allowed.
Store secrets in Azure Key Vault-backed secret scopes or an external secret system.
Use managed identities for Azure resource access where possible.
Rotate secrets and tokens, audit usage, and alert on stale credentials.

Feature status (GA dates)#

The following features reached GA in April 2026, according to the Azure Databricks release notes:

Governed tags GA: 2026-04-02
Databricks Data Classification GA: 2026-04-20
ABAC GA: 2026-04-28
SQL DDL for governed tags (CREATE/ALTER/DROP/DESCRIBE GOVERNED TAG, SHOW GOVERNED TAGS) documented 2026-04-13

The ABAC GA release introduced a breaking change: row filters and column masks on underlying tables now evaluate as the session user when accessed through views or functions, not the view/function owner. Only potentially affected customers contacted by Azure Databricks received a three-month grace period. Other customers received the new behavior by default. Do not assume that grace period remains available.

ABAC compute requirements: serverless, standard compute on DBR 16.4+, or dedicated compute on DBR 16.4+ with fine-grained access control enabled. Older runtimes cannot read ABAC-protected tables.
Governed tags SQL DDL (CREATE GOVERNED TAG, ALTER GOVERNED TAG) requires DBR 18.1+ or Databricks SQL.
Use non-sensitive metadata for governed tags and ABAC. Do not put actual PII values in tag keys or values.

Tag inheritance#

Tags applied to a catalog inherit to schemas and tables underneath.
Tags applied to a schema inherit to tables underneath.
Tags applied to a table do NOT inherit to its columns.
Column tags must always be applied directly.

Classify schemas with data_classification. Then, tag individual PII columns. Schema tags do not propagate to columns.

SQL DDL reference#

Built-in functions for ABAC WHEN and MATCH COLUMNS clauses are snake_case: has_tag('pii'), has_tag_value('pii', 'ssn'). Not hasTag().

These are SQL patterns, not a complete deployment script. Create the catalogs, governance schemas, groups, and target tables first. Apply the matching governed tags before you test each policy. Use DBR 18.1+ or Databricks SQL for CREATE GOVERNED TAG. Prefer TO and EXCEPT for group targeting, as the ABAC best practices recommend.

-- 1. Create the governed tag taxonomy (account-level, one time)
CREATE GOVERNED TAG pii
  DESCRIPTION 'Type of PII contained in the column'
  VALUES ('ssn', 'email', 'phone', 'dob', 'address');

CREATE GOVERNED TAG data_classification
  VALUES ('public', 'internal', 'confidential', 'restricted');

-- 2. Define the masking UDF (one per masking strategy)
CREATE FUNCTION prod_catalog.governance.mask_ssn(val STRING)
  RETURNS STRING
  RETURN '***-**-' || right(val, 4);

-- 3. Define the ABAC policy at the highest useful scope (catalog)
CREATE POLICY mask_ssn_columns
ON CATALOG prod_catalog
COLUMN MASK prod_catalog.governance.mask_ssn
TO `account users`
EXCEPT `pii_readers`
FOR TABLES
MATCH COLUMNS has_tag_value('pii', 'ssn') AS ssn_col
ON COLUMN ssn_col;

-- 4. Row filter pattern (uses USING COLUMNS, not ON COLUMN)
CREATE FUNCTION sales.governance.allow_region(region STRING, allowed STRING)
  RETURNS BOOLEAN
  RETURN region = allowed;

CREATE POLICY regional_access_emea
ON CATALOG sales
ROW FILTER sales.governance.allow_region
TO `emea team`
FOR TABLES
MATCH COLUMNS has_tag('region') AS rgn
USING COLUMNS (rgn, 'EMEA');

Terraform resources — exact names#

The Databricks Terraform provider documents these resources and data sources. Use the unified Unity Catalog tag assignment resource for schemas and columns.

Concern	Resource
Define a governed tag key + allowed values	`databricks_tag_policy`
Look up an existing tag policy	`databricks_tag_policy` (data source)
List all tag policies	`databricks_tag_policies` (data source)
Assign a tag to a UC entity (catalog, schema, table, column, volume)	`databricks_entity_tag_assignment`
Assign a tag to a workspace entity (dashboard, notebook, app, Genie space)	`databricks_workspace_entity_tag_assignment`
Create an ABAC row-filter or column-mask policy	`databricks_policy_info`
List policies on a securable	`databricks_policy_infos` (data source)
Manage who can assign or manage tag policies (`roles/tagPolicy.manager`, `roles/tagPolicy.assigner`)	`databricks_access_control_rule_set`

databricks_entity_tag_assignment takes entity_type (catalogs, schemas, tables, columns, volumes) and entity_name (fully qualified, e.g. prod_catalog.customers.users.email_address).

Quotas and limits#

Limit	Value
Governed tags per account	1,000
Allowed values per governed tag	500
Policies per metastore, all objects	10,000
Policies attached directly to a metastore (Beta)	100
Policies per catalog or schema	100
Policies per table	50
Principals per policy (TO + EXCEPT combined)	20
Column conditions per MATCH COLUMNS clause	3
Tag key/value max length	256 chars, UTF-8, case-sensitive

The design uses a small set of policies with many tag assignments. A metastore that approaches the policy limit needs a design review.

Separation of duties — who does what#

Use this four-role operating model. Base its permissions on the official governed tag permissions and policy requirements.

Tag taxonomy owner (security or governance team) — CREATE on tags at account level. Defines pii, data_classification, etc.
Data stewards / classifiers — ASSIGN on the tag + APPLY TAG on the object. They stamp tags on data.
Governance admins — MANAGE on the catalog/schema + EXECUTE on UDFs. They write the ABAC policies.
Data consumers — query through the enforced policies.

Tagging is a security boundary. If a user can change tags, they can change which policies apply. Audit tag changes.

ABAC limitations#

Review these limitations before you enable ABAC:

ABAC policy inventory has a dedicated information schema view (Public Preview). Use information_schema.abac_policy_definitions, or system.information_schema.abac_policy_definitions across catalogs. The reference lists READ METADATA, MANAGE, or ownership as visibility requirements. The policy guide still lists only MANAGE; verify visibility with your actual reviewer identity. Use SHOW EFFECTIVE POLICIES for inherited policy scope and system.access.audit for operation history.
Pipeline identity affects stored results. If the owner or run-as identity is subject to a policy, the materialized view or streaming table permanently contains masked or filtered data. Exempt a trusted pipeline identity only when the output must retain the original values. Protect that output separately; an exemption gives the pipeline unmasked access.
Time travel and clones fail on ABAC-protected tables unless the principal is in EXCEPT. Trusted service principals that run time travel queries or clone operations need explicit exemption.
AI Search indexes (formerly Vector Search) do not inherit source-table ABAC. The index syncs all rows from the source and serves them without applying filters/masks. For PII columns, use the "columns to sync" setting to exclude masked columns — otherwise PII leaks out the vector search path.
OpenSharing requires the share owner in EXCEPT. Tables with ABAC policies (or views over them) can only be shared if the share owner is exempt. The policy does not govern the recipient. Recipients enforce their own ABAC.
Multiple conflicting policies on the same column for the same user → query blocked. Only one row filter resolves per (table, user) and one column mask per (column, user). If two distinct policies resolve, Databricks blocks access. Same UDF + same args is fine; different masks for the same column for the same user is not.
Standard or dedicated compute on DBR versions before 16.4 cannot read ABAC tables. The escape hatch is to scope the policy to a specific group via TO and put the legacy workload's principal in EXCEPT. Useful migration pattern.

Review each exemption as access to unfiltered, unmasked data. The current ABAC requirements describe pipeline, OpenSharing, and AI Search behavior. Column exclusion does not restore row filters in an index. Do not index rows that its readers must not see.

Rollout checks before production#

Use a test table with synthetic PII before you apply a policy to production data. Record the expected result for each identity before you run the tests.

Test identity or change	Check
Reader without a PII exception	Confirm that protected values remain masked and excluded rows do not appear.
Approved PII reader	Confirm the approved columns are visible without additional table access.
Reader without effective table privileges	Query the table directly and confirm that a mask exception does not grant access.
Production job identity	Run the actual workload with its service principal, not an administrator account.
New tagged column	Confirm that the intended mask applies before readers receive access.
Removed group membership	Confirm the identity change reaches Databricks, then repeat the access test.
Conflicting policy	Confirm that the query fails instead of exposing protected data.

Use the Entra ID and SCIM guide to verify identity synchronization before policy tests. Record the principal, compute, query, expected result, and actual result as release evidence.

If a test exposes protected values, stop the rollout and restrict access to the affected test scope. Do not remove a production mask merely to restore a failed workload. Ask the data owner to approve any exception before you repeat the tests.

Operating principles#

Do not tag at the row level. Row filters use column values. Tag schemas with classification, then columns with PII type only where it matters. Policies are written against tags, so a new PII table does not change policy code. A few PII-type policies cover the whole platform, and a CI check against information_schema.column_tags identifies columns that do not have a required tag.
Confirm all production compute is on DBR 16.4+ before adopting ABAC, otherwise ABAC-protected tables are unreadable. If a team has dynamic views built around owner identity, revisit them for the session-user GA change.