Why agentic AI pilots stall before production

We’ve said it before: building a demo agent is the easy part. Where we see teams struggle is ensuring that agent works in production when it’s up against messy data, infrastructure that sometimes glitches, and complex regulatory and governance needs.

In a recent Data & AI Weekly episode, “From Pilot to Production: What It Actually Takes to Scale Agentic AI,” Redpanda CMO Melissa Czapiga discussed the realities of taking agents to production with Jimmy Astle, a cybersecurity veteran and co-founder of AI startup Onebee.  They chatted about what separates a flashy pilot agent from a production workhorse that people trust—and why the technology shift is only part of the journey.

Below we cover the conversation’s key takeaways for engineers and technical teams who own or are planning agentic projects. Be sure to watch the full 30-minute video to catch all of Jimmy’s insights.

{{featured-event}}

Why do so many agentic AI projects stall before production?

Scaling agents isn’t primarily a model problem. According to Jimmy, the real challenges are getting your data in order and shifting how your people work.

According to Gartner, 60% of AI projects will be abandoned due to data issues, because an organization’s data isn’t “AI-ready.” Before you can build out agentic systems, you need answers to basic questions about your data:

• Where is your data stored, and how accessible is it?
• Is there a common schema or map of where everything lives?
• Who controls access to your data systems?

Then you need to get your entire team on board. Technology tends to outpace a workforce’s ability to adopt it, so you want to hand your people an easy-to-follow adoption process.

How do people build trust in agents?

Jimmy shares a 4-step process to help employees re-learn how to work with help from AI—and warns that this is a lengthy process. Expect the journey to take a year or more, as people acclimate to performing daily tasks using agents.

According to Jimmy, most projects stall between steps 2 and 3 and boil down to issues with building trust or building habits. 

Watch the full Data & AI Weekly episode to hear all of Jimmy’s thoughts on how an organization can successfully follow this process, including why you should treat agents as an extension of your workforce instead of another tool in your tech stack.

What changes between a demo and production?

Demos are straightforward, and the stakes are relatively simple, while agents in production must be able to navigate imperfect conditions and have the power to impact live systems. Here’s a quick look at how the environments compare:

If a pilot looks good in a demo but disappoints in real-world circumstances, Jimmy suggests starting smaller and narrowing the agent’s scope.

What does a production-ready agent require?

When you’re building on a laptop, it’s easy to overlook requirements you’d need for a production-ready agent. These include:

• Durability: An agent can save progress, pause when something goes wrong, and resume once the issue is resolved
• Governance: Agents should be granted the lowest level of access by default, log all actions for auditability, and provide visibility into usage and spend

The other big piece to keep in mind here is who owns the infrastructure. As Jimmy points out, building and managing all of these components in-house requires a lot of engineering hours—and can end up taking time away from customer-facing product features and fixes. Evaluate what makes more sense for your team: building or buying.

Govern agents safely with the Redpanda Agentic Data Plane

Looking for help scaling and managing agents safely? The Redpanda Agentic Data Plane provides a single governed gateway between your apps and agents, with built-in identity, auditing, scope limits, and kill switches. Plus it works with any model, API, database, and cloud.

Learn more from our Head of AI about what an agentic data plane is, and what makes the Redpanda Agentic Data Plane different.

Watch the full Data & AI Weekly episode for more

If your team is struggling to take agents from demo to production, this Data & AI Weekly episode is a must-watch. Check out the full episode on From Pilot to Production: What It Actually Takes to Scale Agentic AI to learn:

• Why the journey to agentic AI is really a data and people journey
• How to move employees from asking agents to delegating, and where projects tend to stall
• Where agents fit into your workforce, and why you shouldn’t see them as just a tool in your tech stack
• What to keep in mind before you start building

And, if you’re ready to build out your governance infrastructure for enterprise agents, book a demo to see the Redpanda Agentic Data Plane in action.