Poolside stays ahead in the competitive world of AI with Redpanda BYOC

Challenge

Choosing the right building blocks to automate pipelines

poolside understands the importance of choosing the right building blocks for its DevOps environment, leveraging familiar tools when appropriate, and carefully adopting new ones as needed. Founded in 2023, poolside is building the world’s most capable AI-driven software development tools to enhance developers’ efficiency and capabilities. The company trains large language models (LLMs) to understand and execute coding tasks, improving and streamlining software development processes.

From the outset, poolside chose the Apache Kafka® ecosystem to connect the real-time data pipelines needed to train AI models that completed millions of tasks in actual software projects. The Kafka API addresses the challenges of discoverability and resilience to errors—both of which are crucial when analyzing real-time data streams.

The poolside team leverages its predefined patterns instead of building a custom pipeline. They also use the Kafka API to implement batches and checkpoint streams, automating time-consuming processes while freeing engineers to build the best AI model.

They were confident about the platform but knew it wouldn’t be enough. Kafka, released in 2011, isn’t optimized for composable architecture, the cloud, edge computing, or the high throughput and massive processing requirements of LLMs and AI applications.

“We wanted the power of the Apache Kafka API without the limitations of an aging platform optimized for obsolete hardware,” says a poolside engineer.

So, for another one of its building blocks, poolside chose Redpanda, a highly performant, cost-effective streaming data platform.

“The AI space is highly competitive. To launch quickly, you must offload routine tasks like managing data streaming pipelines to off-the-shelf components like Redpanda.”

Solution

Redpanda’s cloud-ready Kafka-compatible streaming data platform

The poolside team deployed Redpanda’s bring-your-own-cloud (BYOC) clusters on its on-prem cloud. This fully managed solution gives the team complete control over data movement while also controlling costs. It offers GDPR compliance and full data sovereignty, ensuring the data remains inside the company’s virtual private cloud (VPC) while being accessible to poolside’s customers. It’s also fully Kafka-compliant, enabling the company to manipulate datasets using an improved version of a familiar tool.

Redpanda gives poolside a place to store datasets, perform primitive operations, build software, and efficiently read and write data. They can store intermediate states of transformations, run experiments, and revert to previous iterations if necessary. Engineers have full visibility and control while offloading the time-consuming work of cluster management to Redpanda.

“We love this trade-off where we own the infrastructure, but Redpanda handles the day-to-day management of our clusters. It’s the best of both possible worlds,” the poolside engineer explains.

Redpanda support helps configure poolside’s clusters and assists with scaling as necessary. When poolside occasionally encounters a bottleneck or a limitation, they will often resolve the problem on their own. However, if all else fails, they know they can rely on the Redpanda support team for additional assistance.

Redpanda sits at the abstraction level, so the poolside team can build new solutions on top of it, but it’s high-level enough so they don’t have to keep reinventing the wheel. Redpanda connects poolside’s data pipelines across different teams. The minimalistic UI enables the company’s engineers to sample data, catch and correct mistakes, and seamlessly integrate different streaming jobs, making it easier to process data.

“Everybody knows where the data is and how it flows from one place to another,” says the poolside engineer. “Engineers have a fixed set of protocols and can figure out whether they’re looking at the right datasets. Redpanda does many little things that add up to a great workflow.”

Speed and flexibility are essential when training LLMs. As poolside relies on Redpanda’s streaming capabilities to integrate and update external data sources easily, they’ve improved the accuracy and relevance of the AI-generated output using retrieval-augmented generation (RAG) tools and techniques. This capability helps the company achieve its goals faster.

The engineer explains that Redpanda’s streaming capacity allows them to keep their RAGs up to date with a constant stream of updated web crawls that fit into Kafka topics. Everything plays together so well that they can move more data faster.

The platform runs so smoothly that poolside CTO Eiso Kant says they don’t give it a second thought. “We never have to think about our infrastructure, and having visibility into our compute and network resources is massively valuable. We can move it around freely on AWS, and that’s not the same when you go with a managed product, where you don’t have any insight.”

Results

Moving faster without reinventing the wheel

With Redpanda in their stack, poolside is iterating rapidly, addressing mistakes, avoiding delays and missed deadlines, and launching new features faster in an increasingly competitive space.

This speed is crucial, as the poolside engineer emphasizes: “Mistakes happen all the time. If it takes five days to find a bug in your code, you’re two weeks behind the competition by the time you fix it. Redpanda moves our data faster, so we can undo the damage sooner and shorten the time to get back on track.”

Redpanda automates some of the most challenging and time-consuming aspects of building streaming data pipelines, including writing data safely and consistently, so poolside engineers have more time to bring new product ideas to life. With AWS powering its infrastructure, poolside uses Redpanda to train its ML models at massive scale and speed. It used to take weeks for these training jobs to complete. Now, it only takes a day.

“Partnering with Redpanda was the right decision. We’re happy with the support and the quality of the tool. It’s a valuable component in our toolset that we use daily,” the poolside engineer says.

Any company with complex data stacks in the AI space needs to find the right abstractions that work for them. By introducing Redpanda, poolside has gained a critical building block in its technology stack and can move at the speed of AI.