Why 99% of the web wasn't built for agents

Show notes:

Most software was built long before agents needed to use it, and that mismatch, not model capability, is quickly becoming the real bottleneck for agentic AI. We watch this play out across every data and infrastructure conversation we have: the models keep getting better faster than the rails they run on.

Catherine Jue, Co-Founder and CEO at Kernel, joins us to unpack what it actually takes to give agents real access to the internet, and why identity, not capability, is now the hardest unsolved problem in agentic browsing. Catherine also shares how Kernel is closing the gap between what agents can technically do and what they're safely authorized to do.

‍

Key takeaways:

(00:00) Catherine Jue, Co-Founder and CEO at Kernel, joins us to talk about giving AI agents real access to the internet.
(00:44) Kernel spins up cloud browsers for agents, because the world's APIs cover only a fraction of the work people do on a computer.
(02:44) The real unlock came when LLMs learned to understand a screenshot the same way a person does.
(03:53) Model capability, not interest, held browser agents back, and it is now improving month over month.
(05:21) Agent behavior is shifting from deterministic scripts to non-deterministic, prompted tasks.
(06:52) The classic screenshot-analyze-click loop is a clunky way for an LLM to understand an interface.
(08:10) A live browser telemetry stream lets agents skip the screenshot-and-guess loop, making them 10 to 50% more token efficient.
(12:11) Every browser agent inherits a human's full permissions, because the web has no concept of a scoped-down agent identity.
(13:30) A managed auth layer lets an agent log into a site on a user's behalf without ever touching their credentials.
(15:15) Agent-native protocols like OAuth, scoped API keys, and MCP point toward user-defined scopes that infrastructure can enforce in the browser.
(17:21) Sitting between the agent and the website creates a chance to block risky actions with a low-latency classifier.
(19:05) Seeing every agent's traffic lets infrastructure spot repeatable tasks in regulated industries like healthcare, insurance, and logistics.
(21:15) Cloudflare's Web Bot Auth protocol lets a legitimate agent identify itself instead of getting stuck behind bot detection.
(24:11) Whether a website welcomes agent traffic often comes down to resources, not resistance.
(25:53) Verified identity and consent let websites tell a user's agent apart from a competitor scraping their site.
(27:44) LLMs flip the old software model: instead of users adapting to an interface, agents adapt the interface to the user.
(29:08) Access to the live internet gives agents fresh context beyond what they were trained on.

Thanks for listening to "Hello, Agent!: The podcast at the intersection of data & agents." If you loved this episode, let us know with a 5-star review! Remember to subscribe so you don't miss an episode. To learn more about Redpanda, visit redpanda.com

Resources mentioned:

Kernel, the browser infrastructure platform Catherine co-founded that gives AI agents API-based access to the internet:
https://www.kernel.sh/

Anthropic, the AI research lab whose early computer-use models made it possible for agents to understand and navigate a live browser screen:
https://www.anthropic.com/

Cloudflare, the web infrastructure company behind the Web Bot Auth protocol and Turnstile bot detection referenced in the conversation:
https://www.cloudflare.com/