Soft pastel gradient blending from warm peach to cool blue
Data & AI Weekly
On-demand video from Data & AI Weekly

Why Your AI Agents Need Performance Reviews, Too

Data & AI Weekly · Ep. 9
Weekly | Tuesday at 9AM PT
Why Your AI Agents Need Performance Reviews, Too

Hiring a person comes with a built-in vetting process — resumes, interviews, reference checks — that gives you real confidence they'll behave predictably once they're on the team. Hiring an "agent" skips all of that. You get something capable, sometimes brilliant, with none of the track record.

In this episode, Redpanda CTO Tyler Akidau sits down with Head of AI Marc Millstone to unpack why the enterprises actually getting production value out of agents aren't the fastest movers — they're the ones who can explain, after the fact, exactly what an agent did and why. Drawing on patterns Tyler's seen across dozens of enterprise conversations, they get into what a real evaluation process for an agent looks like, why it has to go well beyond a traditional audit log, and what to get right from day one if you're just getting started.

Learn from these experts

Bald man with a full beard wearing a light suit jacket and black shirt on a white background.
Tyler Akidau

CTO • Redpanda

Marc Millstone

Head of Ai • Redpanda

Watch now
Customers that trust Redpanda
Activision
Akamai Technologies
Chess.com
Cisco Systems
Dun & Bradstreet
Fortis Games
LiveRamp
Moody's
Nanit
NYSE
Optimizely
Poolside
Sovrn
StoneX
Teads
Texas Instruments
The Hotels Network
Truecaller
Vodafone
Wunderkind
Zafin
Rivian and Volkswagen Group Technologies
Activision
Akamai Technologies
Aquatic
Chess.com
Cisco Systems
Dun & Bradstreet
Fortis Games
LiveRamp
Moody's
Nanit
NYSE
Optimizely
Poolside
Sovrn
StoneX
Teads
Texas Instruments
The Hotels Network
Truecaller
Vodafone
Wunderkind
Zafin

Prior Data & AI Weekly Sessions

Inside the Streamhouse: A Shared Architecture for the Age of AI

What is a streamhouse, and why do AI agents need one? Founding members of the Streamhouse Working Group on real-time, production-native data architecture.

Watch now

AI Agents in Production: Lessons from Engineering and Operations

What does it actually take to build and operationalize AI agents in production? Practical insights from leaders who've built the systems that keep AI reliable at scale.

Watch now

From Pilot to Production: What It Actually Takes to Scale Agentic AI

What does it actually take to move agentic AI from pilot to production? Practical lessons on scaling, reliability, and what breaks along the way.

Watch now

High-Availability Deployment: The Trade-offs That Matter

The real trade-offs behind high-availability deployment — replication, failover, latency, and cost — from engineers who've made these calls in production.

Watch now

Navigating the EU AI Act: A Conversation Between Legal and Engineering

What the EU AI Act actually requires in practice, and how engineering and legal teams can work together to build AI systems that stand up to regulatory scrutiny.

Watch now

FDEs vs. In-House Teams: Who Owns What?

Where forward deployed engineers add the most leverage, where in-house teams need to own it, and how to structure the handoff.

Watch now
Data & AI Weekly promo with speakers Melissa Czapiga and Tyler Hannan, watch now on-demand button.

How the Best Data Teams Build for Real-Time: Lessons from ClickHouse and Redpanda

The architectural decisions and hard-won lessons behind data teams that get real-time right, with ClickHouse and Redpanda.

Watch now
Data & AI Weekly podcast banner with hosts Tyler Akidau, Rahul Raina, and Jasmine Mirshahabi.

What Nobody Tells You About Becoming a Senior Data Engineer

What it actually takes to become a senior data engineer, from the people who've hired, promoted, and held the role.

Watch now