Redpanda 25.1: Iceberg Topics now generally available

The 25.1 release of Redpanda brings powerful upgrades for organizations operating at the intersection of streaming data, analytics, and AI using a modern data lakehouse. Iceberg Topics transform how businesses can manage real-time data used for a mix of operational and analytics workloads, delivering the same data sets to applications in both the Apache Kafka^® and SQL lakehouse ecosystems transparently and automatically — in a single environment. 

Redpanda is the first product in the industry to offer a Kafka-Iceberg streaming data solution that is generally available for use on multiple clouds, AWS, Azure, and GCP.

In addition, Redpanda 25.1 introduces a range of important updates:

• Unified identity and fine-grained RBAC bridging Redpanda and Redpanda Console
• Expanded Kafka authentication options with SASL/PLAIN
• Improved schema compatibility with support for normalization of Protobuf schemas
• Better observability with native consumer group lag metrics
• FluxCD removal and introduction of platform-centric versioning for Redpanda on Kubernetes 

Together, these features reduce operational complexity, improve governance, and make real-time data more useful and accessible across the enterprise. Below, we break down what’s new and why it matters. 

Iceberg Topics — now generally available

With Iceberg Topics, Redpanda brings to market a game-changing advancement in how enterprises unify real-time and analytical data. Organizations no longer need to maintain separate infrastructure for managing data used in streaming, SQL queries, and batch processing—Iceberg Topics let you work with streams in a transactional, open table format, instantly queryable by engines like Snowflake, Spark SQL, Databricks SQL, BigQuery, AWS Athena, Trino, and more. 

This approach significantly reduces data duplication, eliminates mundane data engineering work, slashes compute costs (compared to alternatives like Kafka Connect), minimizes latency between event capture and insight, and simplifies your overall data stack.

As this capability becomes generally available, Iceberg Topics adds support for:

• Custom, hierarchical, bucketed partitioning for fine-tuned table layouts and faster, more efficient queries
• Built-in dead-letter queues to redirect and re-process invalid records, improving data quality, reliability, and end-user trust in data
• Seamless schema evolution with full support for all changes allowed by the Iceberg spec, allowing safe field additions, renames, and deletions over time
• Snapshot expiry that performs automatic housekeeping of old snapshot information within table metadata to manage metadata size over time

Instant querying through seamless catalog integration

A key benefit of Iceberg Topics in Redpanda 25.1 is how tightly they integrate with Apache Iceberg-compatible catalogs, including Snowflake’s Open Catalog, Databricks Unity, and AWS Glue. When a topic is configured for Iceberg, Redpanda can automatically register the corresponding table in the Iceberg catalog of your choice. 

That means the data produced to that topic is instantly queryable by all data lake and lakehouse users. No custom ETL, no knowledge of the underlying data’s location, no credentials, and no manual registration needed. As data streams in, tables are automatically kept up to date, and Redpanda’s tunable workload management ensures the table never lags too far behind the topic.

Redpanda’s catalog integration supports:

• Secure REST catalog sync, supporting Snowflake Open Catalog, Databricks Unity, AWS Glue, and others; using OIDC and TLS for secure, seamless interoperability with enterprise analytics platforms
• Transactional writes to Iceberg tables, allowing safe simultaneous use by other clients 
• Automatic table discovery, so newly created Iceberg-enabled topics & data appear instantly in any analytics platform connected to the catalog
• A built-in object store catalog, suitable for ad hoc access by data engineers when no REST catalog is available

Once registered, you can run SQL queries against the Iceberg tables in data platforms like Snowflake, Databricks SQL, Google BigQuery, ClickHouse, AWS Athena, and leverage a broad range of tooling and engines in the Iceberg ecosystem such as Spark SQL, Flink, Trino and others to access data as tables, or directly as Parquet files managed by Redpanda. 

For Snowflake users, data written to Iceberg Topics in cloud storage can be queried via Snowflake’s external Iceberg table integration, leveraging their Open Catalog managed service (based on Apache Polaris™).

Creating this end-to-end pipeline—from producing to a Redpanda topic to querying in Snowflake or Spark SQL—now takes minutes instead of hours or days, unlocking instant analytics on real-time, streaming data.

🤿 Deep dive into Iceberg Topics →

Unified user identity across Redpanda Console and clusters

Managing access in distributed systems can become a source of friction, especially in enterprises with strict governance requirements. In 25.1, Redpanda introduces unified user identity across the Redpanda Console and Redpanda clusters, giving administrators a single, consistent way to manage users and permissions.

Previously, separate access control configuration was required for Console and API access—now, identities and roles propagate across both environments, and access is restricted in Console using fine-grained Access Control Lists and tailored role membership. This update simplifies onboarding, ensures consistent policy enforcement, and reduces the chance of misconfigurations or access drift between the UI and API surfaces. Furthermore, Redpanda’s audit logs now always reflect the true end user’s identity, regardless of how they access Redpanda resources.

Whether you're scaling Redpanda across teams or ensuring auditability in regulated industries, unified identity delivers enterprise-grade governance.

🔐 Learn more about configuring unified identity →

SASL/PLAIN Authentication for Kafka API

Security in event streaming must balance protection with usability. Redpanda offers robust authentication like OIDC, Kerberos, and SASL/SCRAM. However, we recognize that some environments—legacy systems with existing applications and CI/CD workflows—need a simpler approach.

With Redpanda 25.1, SASL/PLAIN authentication makes the integration and migration of existing applications easy, allowing Kafka clients to authenticate using plaintext credentials over TLS. SASL/PLAIN was an original authentication mechanism for early Kafka deployments and is ideal for rapid prototyping, testing, and constrained environments where setup speed matters.

When ready, upgrading to SASL/SCRAM is seamless: just update the authentication method without changing accounts, permissions, or credentials. Redpanda gives teams the flexibility to start simple and scale security as needed.

🔧 Configuration guide →

Protobuf normalization in Schema Registry

For teams relying on Protobuf to define data contracts across services, schema compatibility can become a source of friction as systems evolve. Redpanda 25.1 introduces support for Protobuf normalization in its schema registry, aligning its behavior with existing support for Avro and JSON.

With this update, Protobuf schemas may be normalized during registration and lookup, ensuring syntactic variations in field ordering or structure don’t break compatibility. This results in smoother developer experiences and greater interoperability across microservices, pipelines, and analytics workflows that speak Protobuf. By handling the complexities of minor syntactic differences in logically equivalent Protobuf variations behind the scenes, Redpanda helps teams stay focused on building value—not fighting format quirks.

🧬 More on Redpanda Schema Registry and normalization →

[CTA_MODULE]

New in observability: consumer group lag metrics

As streaming pipelines scale, understanding how far behind your consumers are becomes critical to diagnosing performance bottlenecks, preventing data loss, and maintaining SLAs for end-to-end latency.

With 25.1, Redpanda introduces native consumer group lag metrics, bringing observability in line with what modern Kafka users expect. This feature is a new native (pre-calculated) metric that replaces the previously documented query, rounding out Redpanda’s observability story for enterprises that require transparent monitoring of consumer health and throughput.

These metrics, exposed via Prometheus and visible in observability tools like Redpanda Console, Grafana, or Datadog, enable platform and SRE teams to:

• Monitor lag per consumer group
• Alert on stuck or underperforming consumers
• Correlate lag spikes with downstream processing delays or ingestion bursts

Lag metrics are a foundational signal for any real-time system. With this addition, Redpanda makes it easier for teams running critical Kafka workloads to get the observability they depend on.

📈 Learn more about monitoring consumers in Redpanda →

Platform-centric versioning for Redpanda on Kubernetes 

Redpanda is releasing two new major versions of the Redpanda Helm chart and Redpanda Operator that no longer rely on FluxCD, and is adopting a new Platform-centric versioning scheme, so that Redpanda administrators can easily understand what platform version is tied to each Kubernetes component version.

Starting with version v25.1-k8s-beta1, the Operator and the Redpanda Helm Chart will provide releases in which the Redpanda platform version prefixes all Kubernetes components. This beta release is meant to allow early feedback on the new components. 

Concurrently, Redpanda is also releasing Operator v2.4.1 and Helm Chart v5.10.1, which do not rely on FluxCD by default, reducing the chance of conflicts with a customer’s own use of FluxCD in K8s deployments. 

We recommend that for production environments, the Redpanda platform 25.1 be deployed with the Redpanda Helm Chart v5.10.1 or the Redpanda Operator v2.4.x.   

🤖 Read more on what's new in the Redpanda Operator and Helm Chart 

SELECT * FROM release_25_1; -- closing stream

That’s everything you need to know about the 25.1 release of Redpanda! Push-button analytics in your data lakehouse, unified security model, better observability, and more—all combined with flexible deployment options like Redpanda Bring Your Own Cloud (BYOC). 

The big picture summary: with Iceberg Topics now GA—and now queryable via Snowflake, Databricks, BigQuery, Athena, ClickHouse, Trino, Spark SQL, or any Iceberg-compatible catalog—Redpanda becomes a first-class solution for companies looking to unify batch and stream workloads. With improvements to identity, security, and schema handling, managing data across teams and tools just got a whole lot easier.

If your organization is exploring real-time data architectures—or looking to simplify what you already have—Redpanda 25.1 brings the capabilities to make that shift possible with fewer moving parts. To learn more, contact your Redpanda representative or sign up for Redpanda Cloud, where Iceberg Topics are coming soon to a cluster near you!

‍

[CTA_MODULE]

‍