From Stream to Table: Building a Real-Time Architecture with Redpanda and Databricks

Apache Iceberg has emerged as the foundational table format for the open data lakehouse era. But getting fresh, real-time data into your lakehouse remains painfully complex—and costly.

In this live Tech Talk, Redpanda and Databricks will share how they’ve partnered to remove this bottleneck through a first-principles engineering approach. Redpanda’s Iceberg Topics and Databricks’ Unity Catalog create a sustaining architecture for stream-to-table agility—one that’s performant, open-standard, and production-ready. No batch jobs means no orchestration tools or processes.

You’ll learn:

• Why Iceberg is becoming the universal format for cloud-native analytics
• How Redpanda built an integrated, declarative ingestion layer that respects Kafka semantics while writing directly to Unity-managed tables
• How to eliminate operational drag by unifying streaming, storage, and table governance in one real-time architecture
• See it in action: a live demo of high-throughput Redpanda data landing in Databricks and queried instantly