Databricks blog watch
Platform and governance posts from the Databricks blog. The newest entries appear first.
Added 2026-08-08#
- Ingest semi-structured data faster and more efficiently with Variant - Now Generally Available — Aug 03, 2026 For years, ingesting semi-structured data like JSON, XML, or CSV meant a difficult...
Added 2026-08-08#
- Simplify AI agent orchestration with Lakebase Postgres — Jul 22, 2026
- Why R&D Data Belongs in the Lakehouse - and Why Agents Need It There — Jul 21, 2026
- Navigating a Synapse Migration to Databricks — Jul 08, 2026
- From monolith to Lakebase to LTAP: rethinking the database from storage up — Jun 30, 2026
- A Decision Framework for ETL Migration to Databricks — Jun 26, 2026
- From test bench to lakehouse: how AVL modernizes measurement data analytics with Impulse — Jun 25, 2026
- Ingesting the Milky Way: Petabyte-Scale with Zerobus Ingest — Jun 11, 2026
- How the lakebase architecture stays resilient to cloud failures — May 27, 2026
- Observability for any agent, anywhere: Production-ready tracing with OpenTelemetry & Unity Catalog on Databricks — May 22, 2026
- How to Build Real-Time Fraud Detection using Spark Real-Time Mode and Lakebase — May 19, 2026
- How lakebase architecture delivers 5x faster Postgres writes — May 07, 2026
- Building with Databricks Document Intelligence and Lakeflow — Apr 16, 2026
- The next era of the open lakehouse: Apache Iceberg™ v3 in Public Preview on Databricks — Apr 09, 2026
- Announcing General Availability of Real-Time Mode for Apache Spark Structured Streaming on Databricks — Mar 19, 2026
- Building a near real-time application with Zerobus Ingest and Lakebase — Mar 06, 2026
- Announcing General Availability of Zerobus Ingest — Feb 23, 2026
- Introducing Python User-Defined Table Functions (UDTFs) in Unity Catalog — Nov 17, 2025
- Announcing the Public Preview of Zerobus Ingest — Oct 30, 2025
Added 2026-08-14#
- Databricks Network Configuration delivery to Tens of Millions of Serverless VMs — Aug 12, 2026 SummaryDatabricks' serverless platform launches tens of millions of VMs daily, and...
Added 2026-08-27#
- Choosing Data Governance Tools for Enterprise Data Governance — Aug 25, 2026 Data governance tools are software platforms that help organizations catalog, secure,...
- Data Mesh vs. Data Fabric: Key Differences and How the Lakehouse Resolves the Debate — Aug 25, 2026 Executive Verdict: Organization vs. TechnologyData mesh vs. data fabric hinges on...
- Modernizing SQL ETL in Lakehouse with Declarative Patterns — Aug 25, 2026 Databricks is bringing declarative ETL to data warehousing workflows in Lakehouse...
- Open Table Formats Explained: Iceberg vs. Delta vs. Hudi — Aug 25, 2026 Open table formats are metadata layers that sit on top of data files in object storage, adding ACID transactions...
Added 2026-08-30#
- Building for the AI Era: Lakebase, Streaming, and Lakehouse Innovations at VLDB 2026 — Aug 27, 2026 We are headed to VLDB 2026 to share multiple innovations that power the Databricks platform...
- Vertical Advantage: Transforming Industries with Lakebase and Agentic AI — Aug 27, 2026 In the first blog of this series, we looked at how Lakebase Postgres is rewriting...
- Object Storage + WAL: Lakebase Postgres for the agentic era — Aug 27, 2026 Agents that interact with a traditional OLTP database often create bottlenecks at the storage layer. New deployments...
Added 2026-08-31#
- Autoscaling Lakebase Postgres — Aug 31, 2026 Choosing a database instance size before you know the workload is an old building pattern...
Added 2026-09-05#
- Governance beyond security: knowledge, context & ontology on the lakehouse — Sep 03, 2026 Ask most organizations what data governance for AI means, and you’ll hear a security...
Added 2026-09-09#
- Build durable agents with Temporal and Lakebase — Sep 08, 2026 A personal-loan underwriting agent gathers evidence, applies policy, and may wait...
Added 2026-09-11#
- Unifying governance across engines and catalogs in the Open Lakehouse — Sep 10, 2026 In our previous posts, we showed how open table formats, open APIs and unified governance...
- Improving Lakebase Postgres compute cache — Sep 10, 2026 The disaggregated storage model of Lakebase Postgres provides a feature rich, flexible...
Added 2026-09-13#
- Unify your marketing data with Lakeflow Connect — Sep 11, 2026 This is the first post in a new series exploring how Lakeflow Connect brings fully...
Added 2026-09-15#
- Managed Postgres: What Lakebase Actually Takes Off Your Plate — Sep 14, 2026 Every Postgres vendor calls itself "managed." Few of them agree on what that word...
- Announcing On-Demand State Repartitioning for Apache Spark™ Structured Streaming on Databricks — Sep 14, 2026 Anyone running stateful Apache Spark™ Structured Streaming queries in production...
Added 2026-09-19#
- Database for AI Agents: 5 Evaluation Criteria — Sep 17, 2026 The five criteria for evaluating a database for AI agents are branch isolation, serverless...
Added 2026-09-23#
- Unity Catalog Pages: a governed home for your business knowledge in Genie Ontology — Sep 22, 2026 Every AI agent is only as good as the context it is grounded in. Ask an agent a question...
Added 2026-09-29#
- Lakebase Search: State-of-the-art full text and vector search for Postgres — Sep 28, 2026 Traditional OLTP systems weren't built for the search demands of AI agents. They...
Added 2026-10-01#
- IP Functions are Generally Available, bringing high-performance network analytics to the Lakehouse — Oct 01, 2026 IP data is warehousing dataEvery firewall, load balancer, VPN, CDN edge, DNS resolver,...
- A practical guide to cost optimization with Lakebase Postgres — Sep 30, 2026 Lakebase is a fully managed Postgres database, built for the operational realities...
- Your data, your storage, your rules: a 2026 guide to storing Unity Catalog managed tables — Sep 29, 2026 Unity catalog managed tables allow you to control the placement of your data when...
Added 2026-10-03#
- Real-Time Retail Intelligence: Building E-Commerce Recommendations with Lakebase and AI Search on Databricks — Oct 02, 2026 The opportunity: Personalization as a revenue engineEvery second a shopper spends...
- Lakebase Postgres branch-based restores for fast recovery at scale — Oct 01, 2026 In managed OLTP, restores have always been painfully slow and they get slower at...
Added 2026-10-05#
- Unlocking Data Portability: Preventing Catalog Lock-in with REGISTER and UNREGISTER APIs — Oct 05, 2026 As organizations embrace the lakehouse architecture, data is shifting from proprietary...
Added 2026-10-07#
- The data lakehouse is a better data warehouse: 2026 benchmarks and proof — Oct 06, 2026 The warehouse was the right answer for 20 yearsThe data warehouse earned its place. For two decades...
Added 2026-10-09#
- Announcing Workday Data Connect federation in Unity Catalog — Oct 08, 2026 We're excited to announce the publicly available Beta of the Workday Data Connect federation connector for Unity Catalog...
- Introducing Funke: Native HL7v2 Parsing on Databricks — Oct 08, 2026 The HL7v2 problem in the lakehouseHL7v2 is the messaging standard that quietly runs healthcare...
- Lakebase and Agentic SDLC: Branching Databases for Coding Agents — Oct 08, 2026 AI has changed how software gets built. As coding agents take on a growing share...
- Load terabytes of data in minutes into Lakebase Postgres — Oct 08, 2026 Operational databases like Postgres are built to reliably execute highly concurrent...
- How to build governed enterprise apps on Databricks with Replit and Lakebase — Oct 08, 2026 The combination of Replit & Databricks gives enterprises a complete development-to-deployment platform...
- How to Repoint dbt ETL Pipelines to Databricks — Oct 06, 2026 More teams are running their dbt transformations on Databricks Lakehouse: an open...
Added 2026-10-15#
- Sizing Lakebase Search: lessons from a real app and 4 million passages — Oct 15, 2026 Slide Hub is an internal Databricks app. Field engineers use it to find and reuse...
- Introducing Private Network Gateway — Oct 14, 2026 Databricks serverless compute is fully managed, intelligent, and versionless, allowing...
- What's New in Lakeflow Designer - October 2026 Roundup — Oct 13, 2026 We launched Lakeflow Designer to general availability in June, and since then, over...
Added 2026-10-17#
- Transactions are Generally Available, advancing warehousing on the Lakehouse — Oct 16, 2026 Multi-Statement Transactions are Generally Available, enabling organizations to run...
- Stripe Data Pipeline now syncs real-time Stripe data to Lakebase Postgres — Oct 15, 2026 If your business runs on Stripe, your application needs access to Stripe data at...
- Reducing the noise: How Atlassian built an OCSF security lakehouse on Databricks — Oct 15, 2026 The challenge: Two decades of logging noise at global scaleAtlassian is a global...
- Geospatial queries are up to 38X faster with Lakehouse//RT — Oct 15, 2026 Spatial SQL and spatial joins now run on Lakehouse//RT and they're up to 38X faster...
Added 2026-10-21#
- Deploy RFD4-Proteina on Databricks — Oct 21, 2026 Bring RFD4-Proteina to your research data with AI-assisted development and serverless NVIDIA GPU compute....
- How Hamlet uses Databricks Lakebase Postgres to test changes against production data — Oct 20, 2026 Hamlet, an AI-powered platform that makes local government decision-making accessible...
- What makes a catalog open? Unity Catalog Advances the Open Lakehouse — Oct 19, 2026 While almost every platform now supports open formats, what makes a catalog truly...
Added 2026-10-27#
- Unify your productivity and AI data with Lakeflow Connect — Oct 26, 2026 This is the second post in our series on how Lakeflow Connect brings fully managed...
- Building live intelligence: Databricks AI Search supports Streaming Tables, Materialized Views, and Managed tables using Iceberg — Oct 26, 2026 AI Search now natively supports Streaming Tables, Materialized Views, and Managed...
Added 2026-10-29#
- Smarter cost controls for serverless — Oct 29, 2026 At Databricks, our serverless compute usage has grown more than 3x in the past year,...
- Unlocking Investment Insights Without Migrating Your Data: Lakehouse Federation — Oct 27, 2026 Lakehouse Federation: Query Anywhere, Move NothingIn today's enterprise landscape,...
Added 2026-11-01#
- Lakebase branching for CI/CD: lessons from 100 PR databases — Oct 31, 2026 The problem: testing against production data shouldn't take over 30 minutesSnorkel...
Added 2026-11-03#
- Lakebase Postgres is now generally available on Google Cloud — Nov 02, 2026 We're excited to announce that Lakebase Postgres is generally available on Google Cloud...
Added 2026-11-05#
- Push-based CDC into Unity Catalog with Zerobus Ingest and Debezium — Nov 05, 2026 Data in operational databases drives business decisions, and much of that analysis...
Added 2026-11-07#
- Read, don't copy: replacing file and API feeds with OpenSharing - Part 1 (The Why) — Nov 06, 2026 If you ingest data from your customers over SFTP or APIs, you have probably asked...
Added 2026-11-09#
- Accelerate GPU training on Unity Catalog tables with UCDataset — Nov 09, 2026 GPUs are usually the most expensive part of any training run, and most teams can't...
Added 2026-11-11#
- Unity Catalog Metric Views, now in Power BI: one unified, governed semantic layer for all your reporting and AI — Nov 10, 2026 Organizations want a single, trusted definition of their business semantics. Revenue,...
Added 2026-11-13#
- Dispelling LTAP myths — Nov 12, 2026 Reynold Xin's post "From monolith to Lakebase to LTAP" explained what LTAP is and...
- Eclipse Aerospace simplifies flight-data ingestion with Databricks — Nov 12, 2026 As organizations in aerospace, manufacturing, life sciences, and other data-intensive...
- How H6.ai builds AI-powered financial automation with Lakebase Postgres — Nov 12, 2026 Traditional accounts payable and invoice processing tools often force companies into a trade-off...
- Lakehouse//RT: Scale real-time analytics, not your bill — Nov 12, 2026 AI agents are pushing the limits of data warehouses. A human analyst might run a...
Added 2026-11-22#
- Interactive Databricks Reference Architectures Across 70 Industries — Nov 20, 2026 Why we built itMost Databricks architecture talks used to start with the same slide: ingestion on the left...
- Introducing Genie Migrate for Serverless — Nov 19, 2026 Serverless compute in Databricks offers a simpler, faster way to run your data and...
- PostgreSQL 19 is available on Lakebase — Nov 19, 2026 PostgreSQL 19 reached GA today and it's already available on Lakebase. New major...