4 FAQs about designing agentic systems for production

While your leadership team is salivating over the possibilities of agentic systems, you’re in the trenches trying to architect an infrastructure that will actually support autonomous agents. To get you on track, this post is for the architects and the builders, and we’re diving into the practicalities of designing and running enterprise AI agents.

This is based on a Tech Talk, Inside the Agentic Data Plane for Architects and Builders, hosted by Redpanda Solutions Engineer Garrett Raska, where he discussed the technical challenges of running agents in production—and why the ideal solution is a separate architectural layer for governance, policy enforcement, and observability.

Below we cover some of the most commonly asked questions about designing and operating an agentic system, and the must-have capabilities in an AI governance framework. Watch the full Tech Talk to hear all of Garrett’s insights and see a live demo of the Redpanda Agentic Data Plane.

{{featured-event}}

1. What are the operational challenges of moving agents from prototype to production?

As Garrett points out, prototypes don’t typically have security requirements. You can sandbox them, controlling how you engage and where they access data. 

Moving agents to production requires far more rigor to ensure they operate safely and consistently. You must consider how agents should act (and what they shouldn’t do), taking into account:

• Governance: You need to meet enterprise protocols across Service Control Policies (SCPs), networking requirements, organizational data practices, classified data requirements, and all other security or company standards.
• Control: You must be able to closely monitor everything that happens (all inputs and outputs) to ensure the agent is doing its job appropriately and has the right behavioral requirements. And equally important, you need the ability to quickly review if something goes wrong.
• Data access: You need to make sure agents have access to the right tools, set the right level of permissions for each agent, and limit the scope of their data access and tool calls.

2. How can you ensure agents behave how you want them to?

First, you need an immutable log that records all agent actions—inputs, outputs, and tool calls—distinguishable by specific agent identities. That log gives you the ability to oversee everything your agents are doing, and transcripts to audit if an agent does something wrong. And speaking of agent errors, you also need a way to quickly and easily stop agents in case one goes off the rails.

You also want to be able to set guardrails: policies that your agents follow, from data access to token or budget limits, that keep your agents in line.

But perhaps the most important part of the puzzle is retaining governance policies in infrastructure that sits separate from your agents, so the agents can’t revise or override said policies.

For more information, we recommend checking out our handy cheat sheet on a reference architecture for AI governance that works on any stack. 

{{featured-resource}}

3. What are the benefits of having a control layer between agents and models?

A common question we hear from teams building agentic systems is, “why should we add a control layer between agents and models?” 

Directly connecting your agents and AI models seems simpler, but doing so gives agents unfettered access to your LLMs and frontier models. A control layer that sits between your agents and your LLMs offers several advantages:

• Ability to condition agent responses
• Ensure agents are accessing the right data and not using sensitive information
• Monitor agent token consumption and set limits (by cost or time frame)
• Natural observation point to collect and inspect traffic

4. How can you introduce governance without slowing down development?

Garrett recommends establishing a solid architecture for where you’ll build your agents and how you’ll host all of the accompanying services, like Model Context Protocols (MCPs). This setup should also comply with company-wide policies.

‍Think about it the same way you would approach a traditional enterprise system: How are you going to observe agent behaviors? Oversee authorization and policy compliance? Set budgetary limits? These are the questions you should be answering from an organizational perspective to set your agents up for success while ensuring governance.

Garrett answers these questions in greater detail and covers additional FAQs in the Tech Talk, so be sure to check it out!

Enterprise AI governance with the Redpanda Agentic Data Plane

The Redpanda Agentic Data Plane is AI governance infrastructure built to address the unique needs of enterprise agents. As a separate control layer, the Agentic Data Plane acts as a built-in mediator between your agents and everything they access—data, tools, identities, and models. Safely run and scale agents with a centralized set of rules built on globally-trusted data infrastructure for unified agent governance and real-time streaming.

The capabilities of Redpanda Agentic Data Plane will give you the confidence to trust agents in production:

• Every agent gets a verified identity. No anonymous agents acting on your data, so you always know who did what—and on behalf of which human.
• Every action is recorded and replayable. The audit trail is tamper-proof by design, not a separate system you maintain. No agent can alter the log.
• Every agent is scoped to only what it needs. Set task limits and least-privilege access so a compromised or confused agent can't reach beyond its job.
• Every agent can be stopped and capped. Cap spend, access, and instantly deactivate any agent if it goes off the rails. 
• A single governed gateway. Connect agents to any model, cloud, or database through one gateway. One clear path in and out, not a sprawl of point integrations.
• One pane of glass. The Governance Dashboard shows spend and activity for every agent and server in one view, whether Redpanda-hosted or your own.

Learn more from our Head of AI about what an agentic data plane is, and what makes the Redpanda Agentic Data Plane different. 

Watch the full Tech Talk for more

If your organization is struggling to safely and consistently deploy AI agents, then this talk is a must-watch. Garrett shares how architects and builders can design and run agentic systems in practice. Watch the full Tech Talk on Inside the Agentic Data Plane for Architects and Builders to learn:

• Common questions architects and builders are asking about how to control agents
• How enterprises can scale agentic AI safely
• The capabilities you need to consider for effective agent governance
• How the Redpanda Agentic Data Plane differs from other governance systems

Ready to build out your governance infrastructure for enterprise agents? Book a demo to see the Redpanda Agentic Data Plane in action.