Product watch

Azure Databricks release notes · September 2025

azure12 min read

View all azure databricks release notes

September 2025#

  • Pipeline update timeline table is now available (Public Preview) — Sep 30 · Azure · Product The system.lakeflow.pipeline_update_timeline table provides complete historical tracking of pipeline updates and supports fine-grained analysis of update activity, triggers, results, and compute usage. See Pipeline update timeline table schema.
  • Data classification system table Beta — Sep 30 · Azure · Product A new system table, system.data_classification.results, is now available in Beta. This table captures detections of sensitive data at the column level across all enabled catalogs in your metastore.
  • Anthropic Claude Opus 4.1 now available as a Databricks-hosted foundation model — Sep 29 · Azure · Product Mosaic AI Model Serving now supports Anthropic's Claude Opus 4.1 a Databricks-hosted foundation model. You can access this model using Foundation Model APIs pay-per-token.
  • Lakeflow Pipelines Editor is now in public preview — Sep 29 · Azure · Product The Lakeflow Pipelines Editor (previously called the multi-file editor) is now in public preview. The Lakeflow Pipelines Editor shows a pipeline as a set of files in the pipeline asset browser. You can edit the files and control the configuration of the pipeline and which files t
  • New requirement to create connections for Salesforce ingestion — Sep 29 · Azure · Product In early September 2025, Salesforce began restricting the use of uninstalled connected apps. This restriction does not break existing Unity Catalog connections to Salesforce, but it prevents you from creating a connection to a new Salesforce instance without the Databricks connec
  • AI/BI dashboard global filters panel improvements — Sep 25 · Azure · AI/BI Global filters panel expanded by default: The global filters panel now opens by default when users first visit a dashboard to improve discoverability. On subsequent visits, the panel preserves the user's last setting.
  • Genie benchmarks and JOIN relationship enhancements — Sep 25 · Azure · AI/BI Benchmarks run in the background: Benchmarks now run as background tasks. Users can start a run without blocking their workflow. Checkmark for accepted JOIN relationships: Accepting extracted JOIN relationships now displays a checkmark to confirm acceptance. Improved benchmarks a
  • Databricks SQL version 2025.30 is now available in Preview — Sep 25 · Azure · SQL Databricks SQL version 2025.30 is now available in the Preview channel. Review the following section to learn about new features, behavioral changes, and bug fixes. UTF8 based collations now support LIKE operator You can now use LIKE with columns that have one of the following co
  • Migrate Spark Declarative Pipelines on Lakeflow pipelines from legacy publishing mode is generally available — Sep 24 · Azure · Product Spark Declarative Pipelines on Lakeflow has a legacy publishing mode that only allowed publishing to a single catalog and schema. The default publishing mode enables publishing to multiple catalogs and schemas. Migration from the legacy publishing mode to the default publishing m
  • Databricks Runtime maintenance updates — Sep 24 · Azure · Product New maintenance updates are available for supported Databricks Runtime versions. These updates include bug fixes, security patches, and performance improvements. For details, see Databricks Runtime maintenance updates.
  • Mosaic AI Agent Framework supports automatic authentication passthrough for Lakebase resources — Sep 23 · Azure · Product Mosaic AI Agent Framework now supports automatic authentication passthrough for Lakebase resources. This requires MLFlow 3.3.2 or above. See Automatic authentication passthrough.
  • Route-optimized endpoints now require route-optimized URL path for querying — Sep 22 · Azure · Product All newly created route-optimized endpoints must be queried using the route-optimized URL. Queries using the standard workspace URL path are not supported for route-optimized endpoints created after September 22, 2025. See Query route-optimized serving endpoints.
  • Explore table data using an LLM (Public Preview) — Sep 22 · Azure · Product You can now ask natural language questions about sample data using Catalog Explorer. The Assistant uses metadata context and table usage patterns to generate a SQL query. You can then validate the query and run it against the underlying table. See Explore table data using an LLM.
  • AI/BI dashboard Genie integration and publish dialog improvements — Sep 18 · Azure · AI/BI Genie credential handling for embedded dashboards: Genie integrated with dashboards published with embedded credentials now runs generated queries using the publisher's data credentials. Dashboards published without embedded credentials continue running using the viewer's data cr
  • Genie SQL improvements, API enhancements, and bug fixes — Sep 18 · Azure · AI/BI Upgraded large language models (LLMs): The LLMs used by Genie to author SQL have been upgraded for improved accuracy and performance. SQL autocomplete: Autocomplete is now available when editing Genie SQL to help users write queries more efficiently. Feedback API endpoints: Thumb
  • Genie One Public Preview — Sep 17 · Azure · Product Genie One, a simplified user interface designed for business users, is now in Public Preview. Genie One provides a single, intuitive entry point to interact with data and AI in Databricks, without requiring technical knowledge of compute resources, queries, models, or notebooks.
  • Discover files in Auto Loader efficiently using file events without enrollment (Public Preview) — Sep 16 · Azure · Product The cloudFiles.useManagedFileEvents option with Auto Loader is now in ungated Public Preview. This option allows you to discover files efficiently. For details, see the following: (Pre-requisite) Enable file events for an external location File notification mode with and without
  • Databricks Runtime 17.2 is now is now GA — Sep 16 · Azure · Product Databricks Runtime 17.2 is now generally available. See Databricks Runtime 17.2 (EoS).
  • Delta Sharing on Lakehouse Federation is in Beta — Sep 16 · Azure · Product You can now use Delta Sharing to share foreign schemas and tables created with query federation in Databricks-to-Databricks sharing and open sharing. See Add foreign schemas or tables to a share and Read data in a shared foreign table or foreign schema.
  • Mount Delta shares to an existing shared catalog — Sep 12 · Azure · Product Delta Sharing recipients can now mount shares received from their Delta Sharing provider to an existing shared catalog. Previously, recipients needed to create a new catalog for each new share. See Create a catalog from a share.
  • AI/BI dashboard text widget and performance improvements — Sep 11 · Azure · AI/BI Improved text widget editor: The dashboard text widget now has an enhanced UI for formatting text, adding links, and inserting images. Fixed performance regressions: Resolved issues that caused performance slowdowns, including pages failing to load and URL parameters applying inc
  • Genie chart fixes and feature enhancements — Sep 11 · Azure · AI/BI Fixed thin bars on binned date charts: Resolved an issue where very thin bars appeared in bar charts with binned dates. Fixed single-point maps: Point maps now render correctly even when only a single point is present. Increased entity matching limit: Spaces can now include up to
  • Serverless jobs and pipelines configured in the UI are now performance optimized by default — Sep 11 · Azure · Product Serverless jobs and pipelines configured in the UI are now set to Performance optimized by default. This aligns the UI with existing defaults across APIs, Terraform, and SDKs. For more information, see Performance mode in serverless jobs and Performance mode in serverless pipelin
  • Google Analytics Raw Data connector GA — Sep 10 · Azure · Product The Google Analytics Raw Data connector in Lakeflow Connect is now generally available. See Set up Google Analytics 4 and Google BigQuery for Azure Databricks ingestion.
  • Python custom data sources can be used with Spark Declarative Pipelines on Lakeflow — Sep 10 · Azure · Product You can use Python custom data sources and sinks in your pipeline definitions in Spark Declarative Pipelines on Lakeflow. For information about Python custom data sources, see the following: Load data from a Python custom data source. Create a sink. PySpark custom data sources.
  • Automatic identity management is generally available — Sep 10 · Azure · Product Automatic identity management enables you to sync users, service principals, and groups from Microsoft Entra ID into Azure Databricks without configuring an application in Microsoft Entra ID. When enabled, you can directly search in identity federated workspaces for Microsoft Ent
  • Spark Declarative Pipelines on Lakeflow now supports stream progress metrics in Public Preview — Sep 10 · Azure · Product Spark Declarative Pipelines on Lakeflow now supports querying the event log for metrics about the progress of a stream. See Monitor pipeline streaming metrics.
  • Databricks Runtime maintenance update — Sep 08 · Azure · Product New maintenance updates are available for supported Databricks Runtime versions. These updates include bug fixes, security patches, and performance improvements. For details, see Databricks Runtime maintenance updates.
  • Databricks Apps support for Genie resources — Sep 08 · Azure · Product Databricks Apps now supports adding a Genie Space as an app resource to enable natural language querying over curated datasets. See Add a Genie Agent resource to a Databricks app.
  • Databricks Online Feature Stores (Public Preview) — Sep 05 · Azure · Product Databricks Online Feature Stores, powered by Lakebase, provide highly-scalable low-latency access to feature data while maintaining consistency with your offline feature tables. Native integrations with Unity Catalog, MLflow, and Mosaic AI Model Serving help you productionize you
  • MLflow metadata is now available in system tables (Public Preview) — Sep 05 · Azure · Product MLflow metadata is now available in system tables. View metadata managed within the MLflow tracking service from the entire workspace in one central location, taking advantage of all the lakehouse tooling Databricks offers, such as building custom AI/BI dashboards, SQL alerts, an
  • AI/BI dashboard active filter bar and visualization improvements — Sep 04 · Azure · AI/BI Active filter bar: A new filter bar shows all active filters on a dashboard. Any filter (global or page-level) not set to All, as well as any cross-filtering, appears in this bar. Minimum and maximum aggregations for date fields: Date fields now support minimum and maximum aggreg
  • New Genie API endpoints and improved benchmarks — Sep 04 · Azure · AI/BI New API endpoints: The following new endpoints are available for managing and analyzing conversations: List conversation messages Delete conversation message List all conversations For more guidance, see Best practices and limits. Prompt matching is generally available: Prompt ma
  • Databricks Assistant Agent mode: Data Science Agent is in Beta — Sep 03 · Azure · Product Agent mode for Databricks Assistant is now in Beta. In Agent mode, the Assistant can orchestrate multi-step workflows from a single prompt. The Data Science Agent is custom-built for data science workflows and can build an entire notebook for tasks like EDA, forecasting, and mach
  • Tables backed by default storage can be Delta shared to any recipient (Beta) — Sep 02 · Azure · Product Delta Sharing providers can now share tables backed by default storage with any recipient, including both open and Azure Databricks recipients—even if the recipient is using classic compute. Tables with partitioning enabled are an exception. See Limitations.
  • Migrate Spark Declarative Pipelines on Lakeflow from legacy publishing mode is rolled back to Public Preview — Sep 02 · Azure · Product Spark Declarative Pipelines on Lakeflow includes a legacy publishing mode that previously limited publishing to a single catalog and schema. Default publishing mode enables publishing to multiple catalogs and schemas. A feature, recently released as generally available, can help
  • AI agents: Authorize on-behalf-of-user Public Preview — Sep 02 · Azure · Product AI agents deployed to Model Serving endpoints can use on-behalf-of-user authorization. This lets an agent act as the Databricks user who runs the query for added security and fine-grained access to sensitive data.
  • SQL Server connector supports SCD type 2 — Sep 01 · Azure · Product The Microsoft SQL Server connector in Lakeflow Connect now supports SCD type 2. This setting, known as history tracking or slowly changing dimensions (SCD), determines how to handle changes in your data over time. With history tracking off (SCD type 1), outdated records are overw