# Databricks Wiki A knowledge base for people who build, govern, and operate Databricks. The guides follow the seven pillars of the Databricks Well-Architected Framework, product updates arrive by month, and community lessons are labeled as community lessons. Examples use Azure. The framework is the same on AWS and Google Cloud. Read the [Azure Databricks release notes](azure-release-notes.md) by month, or start with the [Entra ID and SCIM guide](best-practices/identity-entra-scim.md) for identity and access. ## Guides {#guides} One collection per pillar of the [Databricks Well-Architected Framework](best-practices/well-architected-framework.md). Open a guide, or [see the full list](best-practices/index.md). ### [The framework](best-practices/index.md#the-framework) Start with the seven pillars, then the community view across all of them. - [Databricks well-architected framework](best-practices/well-architected-framework.md) - [Community best practices](best-practices/community-best-practices.md) [All 2 guides](best-practices/index.md#the-framework) ### [Data and AI governance](best-practices/index.md#data-and-ai-governance) Unify data and AI management and security, and set data quality standards. - [Unity Catalog grants](best-practices/unity-catalog-grants.md) - [PII and ABAC governance](best-practices/governance-pii-abac.md) - [AI and ML platform](best-practices/ai-ml-platform.md) [All 3 guides](best-practices/index.md#data-and-ai-governance) ### [Interoperability and usability](best-practices/index.md#interoperability-and-usability) Integration standards, open formats, and simple paths to new use cases. - [SQL and BI](best-practices/sql-bi.md) - [Databricks Agent Skills](best-practices/databricks-agent-skills.md) [All 2 guides](best-practices/index.md#interoperability-and-usability) ### [Operational excellence](best-practices/index.md#operational-excellence) Build and release processes, automated deployments, capacity, and monitoring. - [Platform lessons](lessons-learned/platform-lessons.md) - [Terraform vs DABs](best-practices/terraform-vs-dabs.md) - [Lakeflow data engineering](best-practices/lakeflow-data-engineering.md) [All 3 guides](best-practices/index.md#operational-excellence) ### [Security, compliance, and privacy](best-practices/index.md#security-compliance-and-privacy) Least-privilege identity, data protection, network controls, and compliance. - [Identity with Entra ID and SCIM](best-practices/identity-entra-scim.md) [All 1 guide](best-practices/index.md#security-compliance-and-privacy) ### [Reliability](best-practices/index.md#reliability) Design for failure, data quality, autoscaling, and tested recovery. No dedicated guide yet. [Operations and reliability topic](topic/operations.md) ### [Performance efficiency](best-practices/index.md#performance-efficiency) Scaling, serverless, workload design, and performance testing. - [Delta and storage](best-practices/delta-storage.md) [All 1 guide](best-practices/index.md#performance-efficiency) ### [Cost optimization](best-practices/index.md#cost-optimization) Optimal resources, dynamic allocation, and cost monitoring. No dedicated guide yet. [Cost and FinOps topic](topic/cost.md) ## Browse by topic {#topics} Topics are broad areas. Tags are single concepts, like Terraform or serverless, that cut across them. The number is the page count. **Topics** - [Databricks platform architecture](topic/platform) (10 pages) - [Identity and access](topic/identity) (3 pages) - [Governance and Unity Catalog](topic/governance) (6 pages) - [Security and networking](topic/security-networking) (3 pages) - [Data engineering and Lakeflow](topic/data-engineering) (4 pages) - [Delta and storage](topic/delta-storage) (2 pages) - [Compute](topic/compute) (2 pages) - [SQL and BI](topic/sql-bi) (2 pages) - [Operations and reliability](topic/operations) (6 pages) - [Cost and FinOps](topic/cost) (5 pages) - [Developer workflow and IaC](topic/devops) (6 pages) - [Migration](topic/migration) (4 pages) - [AI and ML](topic/ai-ml) (2 pages) - [Product watch](topic/product-watch) (11 pages) **Tags** - [Unity Catalog](tags/unity-catalog) (12 pages) - [Serverless](tags/serverless) (8 pages) - [Compute](tags/compute) (6 pages) - [Terraform](tags/terraform) (6 pages) - [AI and ML](tags/ai-ml) (5 pages) - [Delta Lake](tags/delta-lake) (5 pages) - [FinOps](tags/finops) (5 pages) - [Identity and access](tags/identity) (5 pages) - [Networking](tags/networking) (5 pages) - [Security](tags/security) (5 pages) - [Declarative Automation Bundles](tags/dabs) (4 pages) - [Lakeflow](tags/lakeflow) (4 pages) - [Migration](tags/migration) (4 pages) - [Observability](tags/observability) (4 pages) - [Databricks well-architected framework](tags/well-architected-framework) (3 pages) - [Microsoft Entra ID](tags/entra-id) (3 pages) - [MLflow](tags/mlflow) (2 pages) - [SQL and BI](tags/sql-bi) (2 pages) - [System tables](tags/system-tables) (2 pages) ## Migrating to Databricks? {#migration} Moving off SQL Server or Synapse? Start with the [migration overview](migrations/overview.md), then the guide for your source platform. - [Databricks migration guide](migrations/migration-playbook.md) - [SQL Server to Databricks](migrations/sql-server-to-databricks.md) - [Azure Synapse to Databricks](migrations/azure-synapse-to-databricks.md) ## What changed {#changes} - 2026-09-25 Corrected access-control, identity refresh, network migration, runtime, and pipeline monitoring guidance. - 2026-09-25 Added [Lakeflow recovery procedures](best-practices/lakeflow-data-engineering.md), [serverless cost attribution](lessons-learned/platform-lessons.md#attribute-serverless-costs-to-workloads-and-teams), and [Photon fallback diagnosis](best-practices/community-best-practices.md). - 2026-09-25 Added bundle migration details, integrated SQL Server CDC, and selected September feature changes. For product changes, read the [Azure release notes](azure-release-notes.md) by month or the full [product updates](topic/product-watch.md) list. The [changelog](changelog.md) covers changes to this wiki. [536tech](https://536tech.com/) maintains this wiki. A product claim gets checked against the official documentation before a page counts as reviewed. [About this wiki](about.md) explains how that works. The [coverage report](coverage.md) shows the gaps.