Poolside stays ahead inthe competitive world of AI with Redpanda BYOC

Challenge

Choosing the right building blocks toautomate pipelines

poolside understands the importance of choosing the right building blocksfor its DevOps environment, leveraging familiar tools when appropriate, andcarefully adopting new ones as needed. Founded in 2023, poolside is buildingthe world’s most capable AI-driven software development tools to enhancedevelopers’ efficiency and capabilities. The company trains large languagemodels (LLMs) to understand and execute coding tasks, improving andstreamlining software development processes.

From the onset, poolside chose the Apache Kafka® ecosystem to connect thereal-time data pipelines needed to train AI models that completed millionsof tasks in actual software projects. The Kafka API addresses the challengesof discoverability and resilience to errors—both of which are crucial whenanalyzing real-time data streams.

The poolside team leverages its predefined patterns instead of building acustom pipeline. They also use the Kafka API to implement batches andcheckpoint streams, automating time-consuming processes while freeingengineers to build the best AI model.

They were confident about the platform but knew it wouldn’t be enough.Kafka, released in 2011, isn’t optimized for composable architecture, thecloud, edge computing, or the high throughput and massive processingrequirements of LLMs and AI applications.

“We wanted the power of the Apache Kafka API without the limitations of anaging platform optimized for obsolete hardware,” says a poolside engineer.

So, for another one of its building blocks, poolside chose Redpanda, a highlyperformant, cost-effective streaming data platform.

“The AI space is highly competitive. To launch quickly, you mustoffload routine tasks like managing data streaming pipelines to offthe-shelf components like Redpanda.” — a poolside engineer

Solution

Redpanda’s cloud-ready Kafka-compatible streamingdata platform

The poolside team deployed Redpanda’s bring-your-own-cloud (BYOC) clusters on its on-prem cloud. This fully-managedsolution gives the team complete control over data movement while also controlling costs. It offers GDPR compliance andfull data sovereignty, ensuring the data remains inside the company’s virtual private cloud (VPC) while being accessible topoolside’s customers. It’s also fully Kafka-compliant, enabling the company to manipulate datasets using an improved versionof a familiar tool.

Redpanda gives poolside a place to store datasets, perform primitive operations, build software, and efficiently read and writedata. They can store intermediate states of transformations, run experiments, and revert to previous iterations if necessary.Engineers have full visibility and control while offloading the time-consuming work of cluster management to Redpanda.

“We love this trade-off where we own the infrastructure, but Redpanda handles the day-to-day management of our clusters. It’sthe best of both possible worlds,” the poolside engineer explains.

Redpanda support helps configure poolside’s clusters and assists with scaling as necessary. When poolside occasionallyencounters a bottleneck or a limitation, they will often resolve the problem on their own. However, if all else fails, they knowthey can rely on the Redpanda support team for additional assistance.

Redpanda sits at the abstraction level, so the poolside team can build new solutions on top of it, but it’s high-level enoughso they don’t have to keep reinventing the wheel. Redpanda connects poolside’s data pipelines across different teams. Theminimalistic UI enables the company’s engineers to sample data, catch and correct mistakes, and seamlessly integrate differentstreaming jobs, making it easier to process data.

“Everybody knows where the data is and how it flows from one place to another,” says the poolside engineer. “Engineers havea fixed set of protocols and can figure out whether they’re looking at the right data sets. Redpanda does many little things thatadd up to a great workflow.”

Speed and flexibility are essential when training LLMs. As poolside relies on Redpanda’s streaming capabilities to integrate andupdate external data sources easily, they’ve improved the accuracy and relevance of the AI-generated output using retrievalaugmented generation (RAG) tools and techniques. This capability helps the company achieve its goals faster.

The engineer explains that Redpanda’s streaming capacity allows them to keep their RAGs up to date with a constant stream ofupdated web crawls that fit into Kafka topics. Everything plays together so well that they can move more data faster.

The platform runs so smoothly that poolside CTO Eiso Kant says they don’t give it a second thought. “We never have to thinkabout our infrastructure, and having visibility into our compute and network resources is massively valuable. We can move itaround freely on AWS, and that’s not the same when you go with a managed product, where you don’t have any insight.”

Results

Moving faster without reinventing the wheel

With Redpanda in their stack, poolside is iterating rapidly, addressing mistakes, avoiding delays and missed deadlines, andlaunching new features faster in an increasingly competitive space.

This speed is crucial, as the poolside engineer emphasizes: “Mistakes happen all the time. If it takes five days to find a bug inyour code, you’re two weeks behind the competition by the time you fix it. Redpanda moves our data faster, so we can undo thedamage sooner and shorten the time to get back on track.”

Redpanda automates some of the most challenging and time-consuming aspects of building streaming data pipelines, includingwriting data safely and consistently, so poolside engineers have more time to bring new product ideas to life. With AWSpowering its infrastructure, poolside uses Redpanda to train its ML models at massive scale and speed. It used to take weeksfor these training jobs to complete. Now, it only takes a day

“Partnering with Redpanda was the right decision. We’re happy with the support and the quality of the tool. It’s a valuablecomponent in our toolset that we use daily,” the poolside engineer says.

Any company with complex data stacks in the AI space needs to find the right abstractions that work for them. By introducingRedpanda, poolside has gained a critical building block in its technology stack and can move at the speed of AI.