# New and recent Databricks features to watch Generated 2026-08-08 from . What shipped in the last 90 days that I would raise with a client. Grouped by theme, not by date. I keep this list so I can answer "what changed?" without reading the whole feed, and so I can tell a client the difference between something they can build on and something they should only test. Source is the official Databricks release notes. I do not add a feature that the notes do not name. Conventions: - **GA** — generally available. Safe to design client work around. - **Public Preview** — usable and documented. Keep it off the critical path. - **Beta** — early. Test it, do not sell it. - **announced** — it appears in "What's coming". It is not usable yet. The date in parentheses is when Databricks announced it, plus the effective date when the notes give one. - **(status: verify)** — I could not pin the status. Check the live docs before you quote it. - Dates are the date the item appeared in the docs feed, in `YYYY-MM-DD`. - **The feed is the AWS docs feed. Azure can lag, and some items are AWS-only.** Confirm the Azure page before you give a client a date. I drop AWS-only items where I can identify them. ## Data engineering and orchestration - **Declarative Automation Bundles default to the direct deployment engine** — bundles deploy without the Terraform deployment engine. Bundles still on the Terraform engine migrate automatically. Status: announced (2026-06-03; effective 2026-07-24). Why it matters: this changes how every DABs deployment runs, and it is the first thing to check when a client's bundle deploy starts behaving differently. - **Lakeflow Designer** — a drag-and-drop canvas for preparing and transforming data. The output is production code governed by Unity Catalog. Status: GA (2026-06-16). Why it matters: gives analysts a supported path that does not end in an ungoverned notebook. - **Unit testing for pipelines** — write Python unit tests for Spark Declarative Pipelines in the Lakeflow Pipelines Editor, against mock data, with catalog-table redirection. Status: Beta (2026-07-26). Why it matters: pipeline tests were the weakest part of every CI story I have built. Watch this one. - **Real-time mode in Spark Declarative Pipelines** — end-to-end latency as low as five milliseconds for operational streaming. Status: Public Preview (2026-05-27). Why it matters: changes the answer when a client asks whether they need a separate streaming stack. - **REPLACE WHERE flows** — replace only the rows that match a predicate on each refresh, without reprocessing table history. Status: GA (2026-07-07). Why it matters: the standard fix for a full-refresh pipeline that costs too much. - **REPLACE USING flows** — for sources that arrive as partial snapshots keyed by column. Each update replaces the matching rows and leaves the rest. Status: Beta (2026-08-05). Why it matters: covers the snapshot-export pattern that many ERP and finance sources still use. - **AUTO CDC flows for streaming tables in Databricks SQL** — `FLOW AUTO CDC` on `CREATE STREAMING TABLE`, handling out-of-order records as SCD type 1 or type 2. Status: GA (2026-06-15). Why it matters: SCD2 without hand-written merge logic, from SQL. - **Query-based connectors in Lakeflow Connect** — ingest from a database with a cursor column, with no CDC configuration and no ingestion gateway. Status: GA (2026-05-29). Why it matters: removes the CDC prerequisite that blocks many first ingestion projects. - **Microsoft Dynamics 365 connector** — managed ingestion of CRM and ERP data through Azure Synapse Link. Status: GA (2026-06-26). Why it matters: an Azure-native path for the most common Microsoft-shop source. - **SharePoint connector** — ingest files and structured data from SharePoint sites. Status: GA (2026-08-04). Why it matters: pairs with the Excel and CSV sprawl that sits in every client tenant. - **Lakeflow Connect row filtering** — apply a `WHERE`-style condition during initial load and incremental sync. Status: GA (2026-07-02). Why it matters: cuts volume and duplication at the source instead of after landing. - **MySQL integrated CDC pipeline** — one pipeline for extraction and apply, without the gateway-based architecture. Status: Beta (2026-07-01). Why it matters: fewer moving parts than the gateway pattern, if it holds up. - **Run a subset of job tasks with their dependencies** — the `+` modifier runs a task with its upstream (`+my_task`), downstream (`my_task+`), or both. Status: GA (2026-06-30). Why it matters: makes partial reruns after a failure much less painful. - **SQL alert task in Lakeflow Jobs** — evaluate a Databricks SQL alert as a job task and branch downstream on the result. Status: GA (2026-06-29). Why it matters: puts a data-quality gate in the job graph instead of in a side channel. - **Table update triggers on OpenSharing and system tables** — trigger a job when a provider updates a shared table or when new rows land in a system table. Status: Beta (2026-06-03). Why it matters: event-driven operational jobs, such as reacting to a billing or audit table. - **Behavior changes for continuous jobs and pipelines** — configure a continuous schedule from the pipeline page, including job-level settings such as performance mode. Status: announced (2026-07-08; effective early Aug 2026). Why it matters: a configuration change that can alter the cost of an existing continuous pipeline. - **Lakeflow Pipelines Editor becomes the default on compliance-profile workspaces** — the legacy editor is removed at the same time. Status: announced (2026-07-09; effective Aug 2026). Why it matters: tell regulated clients before their team opens a pipeline and finds a new UI. ## Storage and table formats - **Unity Catalog managed Apache Iceberg tables, foreign Iceberg tables, and Iceberg v3** — create and manage Iceberg tables in Unity Catalog, and read Iceberg tables that live elsewhere. Status: GA (2026-05-21). Why it matters: the format question is now a choice, not a lock-in argument. Expect it in every competitive evaluation. - **Variant and Variant shredding** — a data type for semi-structured data with structured-like query performance. Status: GA (2026-07-22). Why it matters: the right default for Event Hub and REST payloads, instead of a string column plus `from_json`. - **Transactions across Unity Catalog managed Delta tables** — group statements with `BEGIN ATOMIC ... END` or `BEGIN TRANSACTION ... COMMIT`. Status: GA (2026-07-22). Why it matters: removes a real objection from teams migrating from a relational warehouse. - **Auto time-to-live (auto-TTL)** — delete rows automatically from managed Delta tables, managed Iceberg tables, and streaming tables, based on a `DATE` or `TIMESTAMP` column. Status: GA (2026-06-12). Why it matters: a retention control that does not need a scheduled job. Useful in a data-protection conversation. - **Convert a partitioned table to liquid clustering** — `ALTER TABLE ... REPLACE PARTITIONED BY WITH CLUSTER BY`, with minimal reader and writer downtime. Status: GA (2026-06-12). Why it matters: the migration path for every over-partitioned legacy table you inherit. - **Automatic upgrades roll out row tracking and Checkpoint V2 to existing tables** — the rollout applies to existing Unity Catalog managed tables, not only new ones, and is gradual. Status: announced (2026-07-29). Why it matters: a table property changes on tables nobody touched. Know about it before a client's external reader complains. - **Parquet v2 for Delta Lake tables** — advanced encodings, v2 page headers, and INT64 timestamps, set with the `delta.parquet.format.version` table property. Status: GA (2026-06-29, Databricks Runtime 18.1 and above). Why it matters: smaller tables and faster reads, but it is opt-in and it affects external readers. - **Configurable recovery period for dropped managed tables** — set `UNDROP` recovery to 0 hours or to between 7 and 30 days. Default stays 7 days. Status: Public Preview (2026-06-15). Why it matters: answers the "prove the data is gone" question in a data-deletion policy. - **Managed Iceberg materialized views** — a materialized view that external Iceberg readers can read. Requires account team enablement. Status: Public Preview (2026-07-07). Why it matters: serve gold to an outside engine without a copy. - **External data access for pipeline streaming tables and materialized views** — external Delta and Iceberg clients read pipeline-managed tables through the Unity Catalog and Iceberg REST APIs. Status: Public Preview (2026-06-05). Why it matters: same argument, one layer down. - **Change data feed on materialized views** — read a CDF from a materialized view created in a Lakeflow pipeline or in Databricks SQL. Status: Beta (2026-08-07, Databricks Runtime 18 LTS and above). Why it matters: replicate gold changes outward without rebuilding the whole view. - **UNIQUE constraint support** — informational `UNIQUE` constraints on Unity Catalog Delta tables. `RELY` enables join elimination and `DISTINCT` simplification on Photon. Status: Public Preview (2026-06-05). Why it matters: a real query-plan win for a modeled star schema. - **Delta Sharing is now OpenSharing** — a rename that follows the open-source release. Status: GA (2026-06-10). Why it matters: update the word in every deck and doc. The product is unchanged. ## Governance and Unity Catalog - **Secrets in Unity Catalog** — store and govern secrets as securable objects in the three-level `catalog.schema.secret` namespace. Status: GA (2026-08-03). Why it matters: this changes my standing advice on secret scopes. Secrets read inside a job can now sit under the same grants and audit as the data. - **Unity AI Gateway** — governance for enterprise AI: control which AI services teams use, route traffic across providers, and govern MCP servers. Status: GA (2026-08-04). Why it matters: the control plane a client needs before they let teams call models freely. - **MANAGE no longer requires USE CATALOG or USE SCHEMA on the same object** — `MANAGE` now behaves more like ownership. Status: GA (2026-08-03). Why it matters: a privilege-model change. Re-read any Terraform grant module that worked around the old rule. - **Role-based access control (RBAC)** — a user assumes a role and uses only that role's permissions for the session, instead of their accumulated permissions. Status: Public Preview (2026-07-22). Why it matters: the first credible answer to "how do we give an admin exclusive, time-boxed access?". - **ABAC GRANT policies** — dynamically grant Unity Catalog privileges to objects whose governed tags match a condition, instead of granting object by object. Status: Beta (2026-06-11). Why it matters: tag-driven grants scale where per-object grants do not. See [PII and ABAC governance](governance-pii-abac.md). - **Cross-engine attribute-based access controls** — external engines read Unity Catalog managed Delta and Iceberg tables with ABAC, row filters, and column masks enforced server-side. Status: Beta (2026-05-28). Why it matters: masking that survives the trip to an outside engine. - **Tag automations** — assign or remove governed tags on tables and volumes that match conditions you define. Status: Beta (2026-08-07). Why it matters: governed tags only work if they stay accurate. This is the missing half of a tag-based ABAC design. - **Governed tags system table** — `system.tags.governed_tags` holds a record for each governed tag key in the account. Status: Beta (2026-07-24). Why it matters: lets you audit tag coverage from SQL instead of from the UI. - **Governance Hub** — an account-level console view of data health and coverage, AI usage and spend, and cost drivers. Status: Beta (2026-07-22). Why it matters: a ready-made governance report for a steering committee. - **External lineage** — register assets outside Databricks, such as a Power BI dashboard or a MySQL source, so one lineage graph spans the full flow. Status: GA (2026-06-08). Why it matters: lineage that stops at the workspace boundary does not satisfy an auditor. - **Discover page** — a curated browsing experience over Unity Catalog assets, organized by domains and subdomains. Status: Public Preview (2026-06-22). Why it matters: gives a data product a front door that is not the catalog tree. - **Access requests across the metastore** — enable access requests at the metastore level and set a destination that catalogs and schemas inherit. Status: Public Preview (2026-06-22). Why it matters: one routing rule instead of a per-object setting. - **Request for Access enabled by default for all workspaces** — requests route to the owner of the Unity Catalog object by email. Status: announced (2026-07-14). Why it matters: object owners start receiving email. Set the destination before the change lands. - **Refresh materialized views and streaming tables under ABAC** — the pipeline owner or run-as identity no longer needs an exemption from an ABAC policy. Status: GA (2026-08-03). Why it matters: removes the exemption that used to punch a hole in an otherwise clean ABAC design. ## Identity, networking, and security - **Automatic identity management with Microsoft Entra ID** — sync users, service principals, and groups from Entra ID to Databricks without SCIM provisioning. Status: GA (2026-05-26). Why it matters: this is now the default identity design on Azure. Use SCIM only where AIM does not apply. Read [Databricks platform lessons](../lessons-learned/platform-lessons.md). - **Automatic identity management for Okta** — the same, for Okta tenants. Status: GA (2026-07-03). Why it matters: covers the clients who run Okta in front of Entra ID. - **Identity federation enabled by default for new workspaces** — including workspaces not attached to a Unity Catalog metastore. Existing workspaces are unaffected. Status: GA (2026-06-09). Why it matters: new workspaces behave differently from the ones a client already has. Say so during a rollout. - **Explicit entitlements when adding principals to workspaces** — entitlements are granted explicitly instead of inherited from the `users` system group. Admins can opt in from 2026-06-15. Status: announced (2026-06-08). Why it matters: a client who relied on inheritance will find new users have no entitlements. Plan the group design first. - **Email notifications for expiring service principal tokens** — Databricks emails workspace admins when a service principal token is within seven days of expiry. Status: GA (2026-07-22). Why it matters: the failure mode this prevents is a silent production job outage. - **Authenticate as a service principal using OAuth U2M** — obtain tokens as the service principal's identity when you hold the Service Principal Manager role. Status: Beta (2026-05-20). Why it matters: lets an engineer reproduce a job's permissions without holding the secret. - **Context-based ingress control for workspace public access** — allow and deny rules that combine identity, network source, and request type. Status: GA (2026-07-21). Why it matters: a finer instrument than an IP access list, and it does not need Private Link. - **Partner platform IPs in context-based ingress control** — allowlist the IP ranges that Power BI, Tableau Cloud, and dbt platform use. Databricks maintains the lists. Status: Beta (2026-07-14). Why it matters: ends the hand-maintained Power BI IP list. - **Serverless compute access control** — govern who runs serverless workloads through two built-in objects: Default Interactive Compute and Default Automated Compute. Status: GA (2026-08-03). Why it matters: serverless previously escaped the cluster-policy story. Now it does not. - **Block specific internet destinations in network policies** — block destinations from serverless workloads through the network policies REST API, regardless of the network access mode. Status: Public Preview (2026-06-23). Why it matters: egress control for serverless, which is the gap a security team always finds. - **Private Link for account-level resources** — reach the account console privately through a private endpoint instead of the public internet. Status: Beta (2026-07-06). Why it matters: closes the last public leg in a fully private design. Confirm Azure availability before you promise it. - **Inbound Private Link for performance-intensive services** — covers Lakebase Autoscaling and Zerobus Ingest. Status: GA (2026-06-08). Why it matters: OLTP and push ingestion can now sit behind Private Link. - **Custom URLs for the account** — a single branded entry point such as `acme.databricks.com`. Status: Public Preview (2026-07-15). Why it matters: small, but it removes the workspace-URL confusion that plagues large tenants. - **Microsoft Entra ID SSO for Power BI** — configure Entra ID as the identity provider for SSO from Power BI to Databricks, through an account-level federation policy. Status: Public Preview (2026-05-20). Why it matters: the Azure BI chain finally uses one identity end to end. ## Resilience, cost, and operations - **Managed disaster recovery** — replicates Unity Catalog metadata, managed table data, and workspace assets to a secondary region, with a stable URL that survives failover. Status: Public Preview (2026-06-12). Why it matters: replaces the hand-built DR runbook that most clients never finish. Watch this closely. - **Mission Critical workspace add-on** — bundles managed disaster recovery with the existing Enhanced Security and Compliance capabilities at a single compute rate, enabled per workspace. Status: Public Preview (2026-06-12). Why it matters: a new SKU to price before a client asks. - **Budgets, including budgets for Unity AI Gateway** — spending thresholds, with per-user thresholds and overrides for requests managed through Unity AI Gateway. Status: GA (2026-07-06). Why it matters: the control that makes a self-service AI rollout defensible. - **Automatic cost attribution for materialized views and streaming tables in Databricks SQL** — refreshes inherit custom tags from the enclosing SQL warehouse and propagate them to `system.billing.usage`. Status: GA (2026-08-03). Why it matters: closes a known blind spot in showback. Read the [system tables documentation](https://learn.microsoft.com/azure/databricks/admin/system-tables/). - **Standard performance mode for MV and ST refreshes in jobs** — a refresh orchestrated by a SQL task now respects the job's Performance optimized setting, and consumes fewer DBUs when it is off. Status: GA (2026-07-06). Why it matters: a cost lever that costs nothing to pull. - **Instance events and instance pools system tables** — `system.compute.instance_events` and `system.compute.instance_pools` track state transitions and pool configuration history. Status: Public Preview (2026-05-21). Why it matters: the missing piece for explaining a cluster that will not start. - **Legacy `compute_preview` system schema deprecated** — use `system.compute` instead. Status: announced (2026-06-22). Why it matters: check every dashboard and alert that queries the old schema. - **View warehouse activity details** — color-coded annotations on the Running clusters chart that explain why a warehouse stays active. Status: GA (2026-07-13). Why it matters: answers "why is this warehouse still billing?" without a query. - **Lakehouse Replay** — replays a sample of read-only serverless workloads against upcoming runtime releases to catch regressions before they reach production. No setup required. Status: Beta (2026-06-10). Why it matters: removes the strongest argument against serverless, which is loss of version control. - **Deferred compute policy enforcement for all-purpose compute** — apply a policy update the next time the compute terminates or restarts, instead of restarting it now. Status: GA (2026-06-10). Why it matters: lets you tighten a policy without interrupting a running analyst. ## Compute and runtime - **Databricks Runtime 19** — powered by Apache Spark 4.2.0. JDK 21 only; the JDK 17 fallback is removed. Arrow-optimized Python UDFs are on by default, which can change type coercion. Status: GA (2026-07-23). Why it matters: two behavior changes that break real code. Test the UDFs before you upgrade a client. - **Databricks Runtime 18 moves to LTS** — three years of stability and security fixes. Status: GA (2026-07-23). Why it matters: DBR 18 LTS is the version to standardize on in a cluster policy right now. - **JAR tasks on serverless compute** — run a Scala or Java JAR as a Lakeflow Job task on serverless. The JAR must match the serverless environment's Scala, JDK, and Databricks Connect versions. Status: GA (2026-08-05). Why it matters: removes the last common reason to keep a classic cluster for a JVM workload. - **Databricks Container Services for standard compute** — specify a Docker image on standard compute, Databricks Runtime 18.3 or later. Status: Beta (2026-05-19). Why it matters: custom containers without dropping to a single-user cluster. - **Workspace admin setting for the serverless notebook execution timeout** — set the default under Settings > Compute, instead of asking for a manual override. Status: GA (2026-05-27). Why it matters: the serverless analogue of the auto-termination default I always override. - **Improved workspace file read and write performance** — prefetching for reads on serverless and Databricks Runtime 18 and above, and asynchronous flushing for writes on classic compute at Databricks Runtime 17.1 and above. Status: GA (2026-06-16 and 2026-06-04). Why it matters: repos with many small files, and Git operations in the workspace, get noticeably faster. - **Dedicated group clusters on compliance-profile workspaces** — assign compute to a group with Dedicated access mode, so members share compute and still run languages securely. Status: announced (2026-07-24; effective Sept 2026). Why it matters: gives regulated clients a shared cluster option they do not have today. ## SQL and analytics - **Lakehouse Real-Time (Lakehouse//RT)** — a serverless SQL warehouse type for low-latency, high-concurrency reads, with sub-second queries against Unity Catalog tables. Status: Beta (2026-06-30). Why it matters: serving analytical data to an application, without a separate serving database. - **Warehouse-level statement timeouts** — set `statement_timeout` per warehouse through the Create or Update warehouse APIs. API-only. Status: Beta (2026-07-21). Why it matters: a runaway query cap you can vary by workload, instead of one workspace-wide value. - **Metric view parameters** — bind values at query time and serve many query variants from one definition. Status: Public Preview (2026-06-26). Why it matters: one metric view instead of a copy per region or per business unit. - **Window measures on a numeric index column in metric views** — offsets and trailing or leading ranges over a consecutive integer order column. Status: GA (2026-08-05). Why it matters: fiscal weeks and 4-4-5 periods, which no calendar-date function handles. - **MATCH_RECOGNIZE** — the SQL:2016 row pattern recognition clause. Enable the preview in workspace settings. Status: Beta (2026-06-30). Why it matters: sessionization and funnel logic in SQL rather than in PySpark. - **Excel file format support by default** — read and query `.xls`, `.xlsx`, and `.xlsm` with no external library. Status: GA (2026-07-07). Why it matters: every client has an Excel source somewhere in bronze. - **Standalone pipeline materialized views support expectations** — add `CONSTRAINT ... EXPECT (...)` to a materialized view created in Databricks SQL. Status: GA (2026-06-23). Why it matters: data quality checks for SQL-authored assets, not just pipelines. - **Change the owner of standalone streaming tables and materialized views** — `ALTER STREAMING TABLE` and `ALTER MATERIALIZED VIEW` with `SET OWNER TO`. Status: GA (2026-06-23). Why it matters: fixes the orphaned asset an analyst leaves behind when they change teams. - **Refresh policies and EXPLAIN CREATE MATERIALIZED VIEW** — control when a refresh is incremental or full, and see why a plan chose one. Status: GA (2026-05-27). Why it matters: the cost of a materialized view is decided by this setting. ## AI/BI, Genie, and the Microsoft surface - **Genie renamed across the product** — Genie is now Genie One, Genie Spaces are now Genie Agents, and Genie Code is the coding surface. Capabilities are unchanged. Status: GA (2026-06-09 and 2026-07-08). Why it matters: fix the names in every deck before a client does it for you. - **Genie pricing moved to pay-as-you-go** — 150 DBUs of free LLM usage per user per month, about $10.50 in US East. Genie One and Genie Agents usage is free through 2027-01-31, and budget controls do not apply during the promotion. Service principals are excluded. Genie Code is billed. Status: GA (2026-07-08, promotion extended 2026-08-03). Why it matters: a client will plan around "free" and then get a bill in February. Say the end date out loud. - **Databricks Genie app in Microsoft Teams** — ask data questions in a direct message or by mentioning the app in a channel. Status: Public Preview (2026-07-17). Why it matters: the fastest way to put governed data in front of a Microsoft-shop business user. - **Genie One in Microsoft Excel** — query governed data in natural language from the Databricks Excel Add-in and import results as native rows. Status: GA (2026-08-05). Why it matters: it meets finance users where they already work. - **Write data from Microsoft Excel back to Databricks** — create or overwrite a Unity Catalog table from the Excel Add-in. Status: GA (2026-06-04). Why it matters: raise the governance question early. This is a write path into the lakehouse from a spreadsheet. - **Databricks connector for Microsoft Power Platform** — plus OAuth authentication as an alternative to a personal access token. Status: GA (2026-06-23; OAuth 2026-06-15). Why it matters: Power Apps and Power Automate against governed data, without a long-lived PAT. - **Connect Genie to Microsoft Copilot Cowork** — through the Genie managed MCP server. Status: Beta (2026-06-16). Why it matters: worth watching if a client has already standardized on Copilot. - **Import Power BI and Tableau files as AI/BI dashboards** — Genie Code rebuilds the visualizations, reuses metric views, and creates dashboard relationships where it detects them. Status: Public Preview (2026-07-09). Why it matters: turns a BI migration estimate from weeks into a spike. - **Agentic code converter** — converts T-SQL, Snowflake, Redshift, Oracle, BigQuery, and Teradata SQL to ANSI SQL, validating syntax as it goes. Status: Beta (2026-07-16). Why it matters: relevant to the [Azure Synapse migration](../migrations/azure-synapse-to-databricks.md). Test it before you scope the next migration by hand. - **Git folders for dashboards** — version control an AI/BI dashboard in Git. Status: GA (2026-07-16). Why it matters: dashboards can finally follow the same promotion path as code. - **Dashboard relationships** — multi-fact data models scoped to a dashboard. Status: Public Preview (2026-07-01). Why it matters: removes a real gap against Power BI's model layer. ## Mosaic AI and model serving - **ai_query** — call any supported model from SQL or Python. Status: GA (2026-06-15). Why it matters: the general-purpose entry point. Everything else in this section builds on it. - **ai_extract and ai_classify** — extract structured data against a schema, with citations and confidence scores, and classify text against custom labels. Status: GA (2026-06-11). Why it matters: two very common client asks, now with no endpoint to manage. - **ai_search** — retrieve from one or more AI Search indexes with a natural-language query, generating and reranking searches across sources. Status: Beta (2026-08-07). Why it matters: RAG retrieval as a SQL function. - **Vector Search is now AI Search** — the rename comes with full-text indexes that need no vectors or embeddings. Status: GA (2026-06-01). Why it matters: keyword search without an embedding pipeline changes the cost of a simple search feature. - **AI Search High QPS** — set a target QPS on standard endpoints for high-throughput workloads. Status: GA (2026-07-15). Why it matters: makes search bars and recommendations viable on a standard endpoint. - **MLflow trace storage in Unity Catalog** — unlimited storage, OpenTelemetry format, access governed by Unity Catalog schema permissions. Databricks recommends it for production. Status: GA (2026-07-16). Why it matters: agent observability that an audit can accept. - **Endpoint telemetry and faster inference tables for custom model serving** — persist OpenTelemetry logs, traces, and metrics to Unity Catalog, and deliver request and response payloads to an inference table view within seconds. Status: GA (2026-06-11) and GA (2026-06-12). Why it matters: closes the monitoring gap on custom endpoints. - **Model services in Unity Catalog** — a securable that represents a governed LLM endpoint, defined once and shared across workspaces. Status: Beta (2026-06-29). Why it matters: the cleanest way to stop each team from wiring its own endpoint. - **Foundation model Unity Catalog permissions** — control which Databricks-hosted models the organization can use, through permissions on the `system.ai` schema. Requires account team enablement. Status: GA (2026-06-05). Why it matters: the model allowlist a security review will ask for. - **AI Functions Unity Catalog permissions** — restrict which task-specific AI Functions are available, independent of model access. Requires account team enablement. Status: Public Preview (2026-07-08). Why it matters: a narrower control than blocking the model outright. - **Managed agent memory** — long-term memory across conversations, stored in Unity Catalog memory stores and governed as securables. Status: Beta (2026-06-23). Why it matters: removes the bespoke memory store from every agent project. - **Databricks-managed MCP connectors move under Unity AI Gateway** — connectors for Genie One and Genie Code are now governed, access-controlled, and visible alongside other MCP servers. Status: Beta (2026-08-06). Why it matters: MCP stops being a shadow integration path. - **Model retirements** — Google Gemini 2.5 Flash and 2.5 Pro retire on 2026-10-02. Anthropic Claude Sonnet 4 retires on 2026-10-09. Google Gemini 3 Pro is already retired. Status: announced (2026-07-08 and 2026-07-16). Why it matters: pin the model name in a client's code and put the retirement date in their calendar. ## OLTP and apps - **Lakebase supports Postgres 18** — select it when you create a new project. Postgres 17 stays the default. Status: GA (2026-06-18). Why it matters: version parity matters when a team is migrating an existing Postgres application. - **Customer-managed keys for Lakebase in all AWS and Azure regions** — CMK encrypts Autoscaling project data at rest with a key in your own cloud KMS, for new projects. Status: GA (2026-05-20). Why it matters: Key Vault-backed encryption removes a common Azure blocker. - **Lakebase snapshot storage is now billed** — the size appears on the Backup & Restore page. Status: GA (2026-06-01). Why it matters: a live cost change on an existing deployment. Check a client's snapshot retention. - **Postgres password authentication off by default for new Lakebase projects** — existing projects are unaffected. Status: GA (2026-05-21). Why it matters: a connection string that worked in a proof of concept will fail in a new project. - **Lakebase Autoscaling up to 64 CU (128 GB)** — up from 32 CU. Status: GA (2026-05-20). Why it matters: raises the ceiling on what you can serve from Lakebase instead of from Azure Database for PostgreSQL. - **Lakebase is SOC 2 Type 2 compliant** — reports come from the account team. Status: GA (2026-07-14). Why it matters: the first question a client's security team asks about a new data store. - **Databricks Apps horizontal scaling** — run an app across multiple instances behind one URL, with zero-downtime deployment and session affinity. Status: Beta (2026-05-29). Why it matters: the availability story that makes Apps viable for something more than a demo. - **Third-party apps from Databricks Marketplace** — discover, install, and run partner apps in your own Unity Catalog-governed workspace. The provider's code stays closed. Status: GA (2026-06-16). Why it matters: a new procurement and review path a client's security team has not seen yet. - **Databricks Apps on by default for compliance-profile workspaces** — Status: announced (2026-07-31; effective early Sept 2026). Why it matters: regulated clients get a new capability they did not ask for. Decide the policy first. ## Sources - - - - - - - - - - - - Azure equivalents, for the date and availability check: