The lakehouse won the storage war. Streaming is winning the freshness war.

Two years ago the interesting argument in data infrastructure was about format. Would the lakehouse standardize on Iceberg, Delta or Hudi? Would warehouses open up or wall off? That argument is over. Iceberg won, every major engine reads it, and object storage is where enterprise data lives.

So the question moved. It is no longer where the data sits. It is how old it is allowed to be.

Freshness used to be a nice-to-have

We went back through the data-platform RFPs Redpanda has been invited into since 2023 and pulled the freshness requirement out of each. In 2023 the typical ask was "tables updated within the hour." In 2024 it was minutes. In 2025, seconds. In the 2026 cohort, the most common phrasing is some version of "sub-second from event to queryable."

That is a thousand-fold tightening in three years. Nothing else in the stack moved that fast.

What's pulling freshness forward

It isn't analysts. Analysts were always fine with yesterday's data. The pull is coming from three places:

• Agents. An agent acting on an hour-old inventory table will sell what you don't have. Agents are the first consumers who are punished immediately for stale data.
• Regulators. Real-time payments, instant settlement and consumer-duty rules in financial services mean "we'll reconcile overnight" is no longer a defensible answer.
• Customers. The app shows the order shipped; the support agent's screen says it's pending. Every one of those gaps is a ticket.

Why batch can't just run faster

The instinct is to shorten the batch window: run the job every five minutes instead of every hour. This works until it doesn't. Below a certain interval, the overhead of spinning up a job, scanning for changes and rewriting files exceeds the interval itself. Costs climb, small files pile up, and the "near real-time" pipeline becomes the most expensive and least reliable thing in the platform.

At some point the honest architecture is to stop pretending and write the tables from the stream.

The convergence

This is the quiet shift happening in 2026: the streaming layer and the lakehouse are no longer two systems with an ETL job between them. The stream is the write path. Events land once, become Iceberg tables continuously, and are queryable by any engine within seconds, with the same schema and the same governance from both sides.

"The lakehouse answered 'where.' Streaming answers 'when.' Enterprises are finding out they need both answers from the same system."

The vendors who win the next phase won't be the ones with the best table format. Everyone has the same one now. They'll be the ones who can promise a freshness number and hit it under load.

What to ask your platform team

Not "do we have a lakehouse." You do. Ask: "What is the age of the newest row in our most important table, right now?" If the answer is a shrug or a number in hours, you have a storage architecture, not a data architecture. The gap between the two is where the next two years of platform spend is going.