
6 questions to ask before choosing an AI agent governance platform
Make sure you’re buying actual agent governance, not lock-in

Why AI agent accountability starts with observability, and how to mitigate risk with AI governance
So you’ve started deploying agents in your organization. Maybe they’re automating tedious workflows or even providing external customer support. But are you prepared for what happens when one fails? Say an agent makes a change it shouldn’t have or acts without the right permissions. Who responds? Who’s accountable? And how do you prevent it from happening again?
Those are the questions we help answer in episode 10 of our Data & AI Weekly series, “When AI Governance Fails: Who’s Accountable When Agents Get It Wrong.” Daniel Takabayashi, Staff Forward Deployed Engineer at Redpanda, works directly with customers deploying agentic systems. In this episode, he chats about where and why these projects fail, why accountability should be discussed in the planning stage of an AI project, and how to mitigate risk with proper AI governance.
Read on for highlights from the talk, or watch the full episode to catch all of Daniel’s insights.
By 2028, more than 50% of failed AI projects will occur not from model performance problems but from weak accountability, according to a recent Gartner report titled When AI Governance Fails, Revenue and Executives Take the Hit.
AI governance is quickly shifting from an optional initiative to a must-have. Procurement and security reviews are now starting to test for AI governance the same way they do SOC II compliance. Companies are being pressed to meet the regulatory requirements of the EU AI Act, whether they’re in the EU or not—much like GDPR started impacting organizations globally.
That means an annual review and a spreadsheet inventory of your AI systems won’t cut it. As Daniel points out, the more complex your agentic systems become, the harder they will be to observe and the harder it will be to mitigate damage.
Over the years, AI project worries have shifted from “Is it good enough for production?” to “Do we have the skills to train it?” and then to “Is it scalable?”
Now the open question is observability: how do you monitor nondeterministic agents when you don’t know the end result? According to Daniel, too many companies forget to map risk and accountability at the start. When something goes wrong, suddenly people are pointing fingers and asking questions like:
Gartner outlines this scenario in the report: a customer flags unfair pricing from an AI-driven quote engine, legal asks who approved it, sales asks whether quotes can still go out, and the board asks why no one ran a risk assessment.
Daniel shares best practices for implementing AI accountability in the Data & AI Weekly episode, including when and how to map risks.
Daniel shares several use cases based on real customer projects to show how quickly complexity (and risk) snowballs when an organization deploys multiple agents and AI tools.
His examples start with a relatively simple use case (an IT agent) and then demonstrate how a company could continue building a complex web of agents based on that original use case. Here’s a quick diagram of how the IT agent could function:

As a company continues developing its network of agents, these straightforward observability requirements quickly shift to a complex web of systems, agents, and permissions. Accountability becomes much harder to apply after the fact, which is why Daniel recommends discussing these topics early in your AI projects.
Below is a summary of the use cases Daniel discusses and their expanding skills, integrations, and observability requirements. (Check out the full Data & AI Weekly episode for a more detailed walkthrough of each use case.)
Regardless of where you’re at in your agentic journey, you should prioritize assessing observability needs if you haven’t already. Daniel suggested two high-priority action items to focus on:
Models are rarely the weak link. The bigger potential problem is whether you know what your agents are doing, what systems they touch, and who owns the outcome when something goes wrong. Get observability in place first and name your owners and risks early, so the controls that come after (alerts, evals, kill switches) have something to stand on.
If you're looking to put that into practice, Redpanda Agentic Data Plane provides a single governed gateway between your apps and agents, with built-in identity, auditing, scope limits, and kill switches. The best part is it works with any model, API, database, and cloud. See the Agentic Data Plane in action.

If your team is facing post-deployment AI observability challenges, watch the full episode on When AI Governance Fails: Who’s Accountable When Agents Get It Wrong to learn:
If you’re looking for a unified AI governance platform for enterprise agents, ask our team for a custom walkthrough of Redpanda Agentic Data Plane.

Make sure you’re buying actual agent governance, not lock-in

Why the demo worked but the rollout didn’t—and what you need to successfully scale agents

Good intentions don’t work. Mechanisms do.
Subscribe to our VIP (very important panda) mailing list to pounce on the latest blogs, surprise announcements, and community events!
Opt out anytime.