Akamai sharpens threat intelligence and future-proofs streaming pipelines

Challenge

Escalating costs and latency in critical data pipelines

With an edge network spanning over 4,100 locations in more than 135 countries, Akamai is the steward of one of the world’s largest Content Delivery Networks (CDNs), blocking billions of cyberattacks daily and offering advanced protection for web apps, APIs, and critical infrastructure to thousands of customers.

"30% of the internet bandwidth is going through Akamai networks”, says Idan Harel, Infrastructure Team Lead in Akamai’s Cloud Security Intelligence (CSI).

Consequently, the CSI division ingests massive volumes of security event data from its global infrastructure, with its largest feed handling over three million events per second. The team was using Confluent Cloud and object storage, which had a growing number of challenges:

• Escalating costs of Confluent Cloud as data volumes exploded
• High latency caused by batch processing via object storage
• The existing solution had object storage scaling issues with Linode

“We had a big data platform that processed events from our security products and made them accessible to other security systems. But our Kafka just referenced files stored in object storage. Latency was an issue, and the costs were growing,” explained Yaniv Kunda, Senior Software Architect at Akamai.

Why Redpanda

Cost and resource efficiency, with private cloud compatibility

Looking to simplify its architecture, reduce infrastructure spending, and future-proof its data pipeline, Akamai discovered Redpanda. 

“Confluent Cloud had become one of our top third-party expenses,” said Yaniv. “When we ran the numbers, Redpanda came in at about 55% lower costs, and that includes both cloud and self-hosted alternatives.”

The team initially adopted Redpanda as a Kafka-compatible, self-hosted replacement for Confluent Cloud. But after validating its performance and ease of operation, they realized they could aim higher, and began shifting more data directly into Kafka topics and moving toward a true, real-time streaming architecture. 

To support the transformation, Akamai deployed Redpanda Self-Managed Enterprise, powering two of its most critical application security systems: Cloud Security Intelligence (CSI) and Client-Side Protection and Compliance (CPC). While the initial deployment was on Azure, Akamai set a strategic goal to bring more workloads in-house to Linode, its recently acquired cloud platform.

“Redpanda’s performance, support, and deployment simplicity made it easy to expand. No other streaming solution could keep up with our needs, especially on Linode.”

Results

55% cost reduction and accelerated development cycles

Akamai's shift to Redpanda enhanced performance and resource efficiency, further advancing real-time insights and cybersecurity at the edge for its customers. The internal impact was undeniable, giving Akamai’s team the following advantages: 

• Cost reduction: 55% savings compared to Confluent Cloud and self-hosted alternatives
• Low latency: enables near real-time processing across global data streams
• Scalability: replaced object storage limitations with a cleaner, more efficient architecture
• Developer agility: Redpanda Console gives developers instant insights into their topics
• Strategic alignment: runs on Akamai-owned Linode infrastructure, giving them full visibility and control
• Smooth upgrades: despite running on Akamai's private Linode cloud, three major upgrades were performed with zero downtime.

Redpanda also delivered what the DevOps team at Akamai needed: greater control and fewer restraints. Previously, running a cloud-based service meant they couldn’t fully tune or customize the platform. With Redpanda, they regained ownership of their environment. 

Meanwhile, all existing DevOps tooling, such as Terraform scripts, automation, and user/topic management processes, continued working unchanged. The team “didn’t have to change a thing.”

Based on Redpanda’s success, Akamai’s CSI division is already planning its next architectural move. Today, much of their long-term event data is stored in flat files accessed with Jupyter-based tools. Now with Redpanda’s support for Apache Iceberg™, they plan to shift toward a modern, open data lakehouse format with lower operational overhead and more flexible querying. 

Furthermore, as Akamai infuses AI-powered features into its threat intelligence products, Redpanda serves as a reliable streaming backbone, delivering the low-latency data pipelines that AI models need to perform in real time while also accelerating product development.