Databricks Community watch

Platform and governance posts from the Databricks Community Technical Blog, filtered and accumulated. Community posts are practitioner guidance, not product documentation; verify product claims against the linked official docs before relying on them.

Source: Databricks Community Technical Blog.

Added 2026-08-18#

• Triggered vs. Continuous Mode: A Deep Dive into Serverless Lakeflow Spark Declarative Pipelines — Aug 10, 2026 One pipeline woke up every five minutes. The other stayed running. Using the same e-commerce streaming workload, median end-to-end data freshness improved from 4.0 minutes to 33 seconds, while P95 latency dropped from 9.3 minutes to just 60 seconds. Those resu
• How to Build, Test, and Ship SQL Pipelines on Databricks — Aug 10, 2026 Spark Declarative Pipelines (SDP) give SQL developers a first-class ETL experience. It is purpose-built for SQL, and it handles batch as well as streaming. There is now a full dev loop for SQL pipelines: you can either author in the new Databricks pipelines ed
• Building a Production LangGraph Agent on Databricks - NorthStar Brand Copilot — Aug 10, 2026 We built NorthStar Brand Copilot, an AI assistant for brand managers and field-sales reps at a fictional multi-category CPG company (Snacks, Beverages, Personal Care). It's a LangGraph custom agent that routes each question to the right Databricks-native capab
• Building Custom Apps on Lakehouse — Aug 07, 2026
• Plug & Play: Zerobus Ingest Now Supports Apache Kafka® Compatible APIs (Beta) — Aug 07, 2026 Learn how to use your existing Apache Kafka producer clients to ingest data directly into your Delta lakehouse with new beta support for Kafka-compatible APIs.
• Load Testing Databricks SQL Warehouses with JMeter — Part 3: Running and Analyzing Results — Aug 05, 2026
• Load Testing Databricks SQL Warehouses with JMeter — Part 2: Concurrency and the Test Plan — Aug 05, 2026
• Load Testing Databricks SQL Warehouses with JMeter — Part 1: Inputs and Configuration — Aug 05, 2026
• Try the new Zerobus Rescue Column: Choose-Your-Own-Adventure Schema Management (Beta) — Jul 29, 2026 It is Friday afternoon. An upstream team ships a harmless-looking change: they add a region field to the events your service already emits, and somewhere along the way, a sensor starts reporting temperature as "72F" instead of 72. You did not write that produc
• From Experiment to Prod: LakeFlow Spark Declarative Pipelines Testing Blueprint — Jul 28, 2026 Part 2 of the testing blueprint: how to test SDP pipelines end-to-end. Decouple your logic for fast unit tests, then use pytest, the Databricks SDK, and Databricks Connect to run the real pipeline against ephemeral, isolated Unity Catalog schemas, all from you
• Serve Tableau Reports Directly from Databricks SQL — Jul 28, 2026 Connect Tableau to Databricks SQL to eliminate data warehouse sync pipelines, cut compute costs, and serve dashboards directly from your Lakehouse.
• When Milliseconds Matter: Databricks Powers the Next Generation of Operational Workloads — Jul 27, 2026 Databricks redefined real-time data processing with Real-Time Mode (RTM) in Spark Structured Streaming, hitting sub-300ms P99 latency for mission-critical operational workloads. Lakeflow Spark Declarative Pipelines let both pro-coders and SQL/Python developers
• One Pipeline, Any Destination: The ForEachBatch Sink in Spark Declarative Pipelines is now GA — Jul 24, 2026 A Spark Declarative Pipeline used to stop at the edge of the lakehouse. Streaming to a system with no first-party connector meant bolting a standalone Structured Streaming job onto the declarative one, and operating a second framework to do it. The ForEachBatc
• Azure DevOps with Databricks DABs: Step-by-Step Integration Guide — Apr 16, 2026 A practical CI/CD walkthrough using Declarative Automation Bundles, Databricks Git folders, Azure DevOps service connections, staged deployment, and production approvals.