Redpanda Cloud improves cost-efficiency by 50% with new ARM processor-based instances

The latest update to Redpanda Cloud brings you improved performance, simpler schema management, a new Iceberg connector, and lower costs.

By
on
November 27, 2023

Today, we’re excited to announce upgrades to Redpanda Cloud that bring greater hardware performance and cost-efficiency, including up to a 50% reduction in VM cost, and a 2x increase in partition counts supported.

Additionally, the latest update to Redpanda Cloud brings a managed connector for Apache Iceberg for easier data lake integration, a simple user interface for schema registry management (that even non-experts can use), and customizable maintenance windows for greater flexibility in planning upgrades.

Here’s a breakdown of what’s new.

ARM processor-based instances: 50% more cost-efficient

We’re always finding new ways to optimize Redpanda Cloud for better performance and cost-efficiency. That’s why we’re introducing ARM processor-based instances in Redpanda Cloud, starting with support for the AWS Graviton family of Amazon EC2 instances.

For those unfamiliar with ARM, it is a family of CPUs based on the RISC (Reduced Instruction Set Computing) architecture, and known for their low cost and energy efficiency. ARM processors are used everywhere from mobile devices to supercomputers, and power cost-efficient cloud computing services, such as AWS Graviton.

With our new ARM instances for Redpanda Cloud on AWS, you can expect as much as a 50% reduction in TCO compared to standard x86 instances when running Redpanda BYOC (Bring Your Own Cloud) clusters.

The new optimized tiers also scale to higher partition counts, and can support up to 2X the number of partitions at the same cost as previous generation instances, for both BYOC and Dedicated clusters. By paying less per throughput and per partition, customers can now unlock a broader range of streaming data use cases in the cloud.

Below is an example of monthly TCO savings with the new Redpanda Cloud optimized tiers versus previous generation instances:

AWS Tiers Throughput Previous Generation VM + local storage Cost Estimate New Optimized Tiers VM + local storage Cost Estimate TCO Savings
AWS-Tier-1 Ingress: up to 20 MB/s Egress: up to 60 MB/s $10,971 $5,392 50.85%
AWS-Tier-2 Ingress: up to 50 MB/s Egress: up to 150 MB/s $20,353 $9,168 54.96%
AWS-Tier-3 Ingress: up to 100 MB/s Egress: up to 200 MB/s $39,091 $16,721 57.25%
AWS-Tier-4 Ingress: up to 200 MB/s Egress: up to 400 MB/s $57,802 $31,827 44.94%
AWS-Tier-5 Ingress: up to 400 MB/s Egress: up to 800 MB/s $85,896 $62,037 27.78%

These enhancements add to the cost-efficiencies many organizations are already seeing from Redpanda BYOC, which allows you to leverage the cloud provider discounts and commitments you may already have with your infrastructure provider.

Apache Iceberg managed connector: easier data stream analysis

Increasingly, data engineers and data analysts want to perform analytics on streaming data with evolving schemas. Today, this usually requires building and maintaining ETL / ELT pipelines to consume data from streaming data platforms like Redpanda, and transcode it into a suitable format for analytics like Apache Iceberg—all while juggling changing schemas.

With our new fully managed connector for Apache Iceberg, you now have a simple way to analyze Redpanda data streams using tools that support the open-source Iceberg format — including Snowflake, Databricks, Google BigQuery, Amazon Athena, ClickHouse, Tabular, Starburst, Dremio, Spark, Hive, and Presto.

With the new connector, you can store Redpanda data as Iceberg files in object storage (like S3 and GCS) to be used ad hoc for analytics, without having to pay for importing and storing data in multiple analytics systems.

Diagram of how the Iceberg connector plays with Redpanda

Note: the managed Iceberg connector is the first stage of our Apache Iceberg journey, and should not be confused with our bigger project to have Iceberg tables stored directly in Redpanda tiered storage — stay tuned!

Schema registry management: simpler schema operations (even for non-experts)

Managing schemas is a frequent operation for practitioners working with Kafka-compatible systems like Redpanda. It requires knowledge of the Schema Registry REST API, which enables automated, programmatic access (but isn’t always well suited for non-experts and human operators). Troubleshooting typed messages can require writing an ad-hoc application that writes/reads the record in the proper wire format. This is all extra work for engineers who just want to start getting more value from their data streams.

To reduce this complexity, we’ve added schema management features to our web console, bringing visual tooling that helps even non-experts run typical schema operations with ease. Human users can now perform schema operations directly in the UI, including creation, versioning, browsing/searching, interacting with typed topics (topics with messages with schema), and import/export. And, because the Redpanda Console already knows the identity and role of the user, we can combine multiple API calls in the UI to provide an even better UX.

Simple schema registry management finds it way directly into the Redpanda UI
Simple schema registry management finds its way directly into the Redpanda UI

Maintenance window selection: upgrade at your convenience

Upgrades are one of the most expensive operations for IT teams today which has led customers to adopt fully managed services. Vendors will manage all the updates needed to run your cloud service both at the infrastructure and application level.

However, most vendors apply these updates to your environment at a time of their choosing when it is convenient for their teams. Redpanda Cloud provides the flexibility to select a Maintenance window to apply updates during off-hours at a time that best suits your business. Through your Redpanda Cloud cluster setting, customers are able to specify a Maintenance window during which Redpanda services can be upgraded or patched to a new version.

All upgrades and maintenance on Redpanda Cloud clusters are implemented in a rolling fashion, accompanied by a series of health checks—so there'll be no disruption to the availability of your service. Redpanda Cloud’s top priority is always to provide you with the best possible experience as part of a fully managed service.

Redpanda Cloud: flexible, powerful, connected

That’s the latest for Redpanda Cloud, but we’re not slowing down: expect regular updates that ensure we offer the most flexible, powerful, and connected streaming data cloud platform.

Welcome to the next step in the future of cloud.

To get started, sign up for a free trial of Redpanda Cloud or grab the free Community Edition from our Redpanda GitHub repo. If you have questions about the latest updates or just want to chat with our team, join our Redpanda Community on Slack.

Graphic for downloading streaming data report
Save Your Spot
Redpanda Connect for Cloud
Christina Lin
&
&
&
September 12, 2024
Text Link
New AI connectors and GPU runtime support for AI models
Tyler Rockwood
&
&
&
September 12, 2024
Text Link
Cloud Topics: Efficiently stream data through object storage
Noah Watkins
&
Matt Schumpert
&
&
September 12, 2024
Text Link