
The convergence of AI and data streaming - Part 1: The coming brick walls
A realistic look at where AI is now and where it’s headed
A new storage engine built from the ground up for Kafka-API compatibility with Kernel bypass technologies for 10x lower tail latencies.
On Dec 1st, I gave a talk at Performance Summit titled "Redpanda: A new storage engine built from the ground up for Kafka-API compatibility with Kernel bypass technologies for 10x lower tail latencies."
The talk covers how we built a new storage engine from scratch using a thread per core architecture for predictable tail latencies. We dive deep into the challenges of adapting a well known protocol (Kafka-API) on top of this new storage engine and the performance gains of optimizing for modern hardware.
Here for your viewing pleasure is the video of that talk:
Sign up for our Community Slack (here!) and engage with us on Twitter via @redpandadata or personally at @emaxerrno
Chat with our team, ask industry experts, and meet fellow data streaming enthusiasts.

A realistic look at where AI is now and where it’s headed

Putting governed agents to work, plus a refreshed onboarding UX

Highlights from the second day of Redpanda Streamfest 2025
Subscribe and never miss another blog post, announcement, or community event. We hate spam and will never sell your contact information.