Designing reliable training infrastructure for edge and foundation models with Tianshu Yu at Liquid AI

Show Notes:

Most ML training bugs don't announce themselves. They hide in floating-point rounding, batch distribution drift, and concurrency issues that only emerge across thousands of GPUs. The hard part isn't writing the algorithm — it's building infrastructure disciplined enough to tell the difference between a numerically equivalent optimization and a quietly corrupted model.

We speak with Tianshu Yu, member of the technical staff at Liquid AI, about the real work of building reliable ML systems at scale. Tianshu previously led reinforcement learning infrastructure at ByteDance, where he was a core contributor to VERL — one of the most widely used open-source RL training frameworks. He now works on pre-training and architecture design for edge-deployed vision language models.

The conversation covers the three-phase structure of reinforcement learning, why checkpoints serve debugging, fault recovery, and model selection simultaneously, and how capability collapse emerges when data distributions concentrate during fine-tuning. We also get into what changes when you design models for edge hardware: the shift from attention to convolution-dominant ops, latency budgets as a first-class architectural constraint, and why a 350M parameter model has meaningfully different engineering requirements than a frontier model.

‍

Key Takeaways:

Here you go:

(00:36) Tianshu's work on VERL at ByteDance centered on a core RL infrastructure problem: mismatch between rollout distributions and training distributions is one of the major causes of training instability in reinforcement learning.

(02:36) Reinforcement learning lets a model learn from samples it generates itself — training proceeds until reward plateaus, but reward alone is insufficient; eval benchmarks must be monitored holistically alongside it.

(03:39) Capability collapse is a real risk during RL fine-tuning: overfitting on a concentrated, English-heavy dataset with a large learning rate can degrade multilingual and other capabilities — especially in small models with constrained capacity.

(06:55) Checkpoints serve three distinct purposes: monitoring quality across training steps, debugging anomalies, and fault recovery — and a perfectly smooth reward curve can still hide jagged eval results that reveal infra or reward-logging bugs.

(08:52) Fixing the airplane while it's flying: ByteDance's 54-day training run saw roughly 170 code updates per week, with checkpoints serving as resume points after every bug fix, code change, or hardware failure.

(11:16) Training bugs decompose into three sources: bad dataset mix, infrastructure errors (like sharding that produces incorrect loss), and algorithmic flaws such as an architecture lacking expressivity.

(12:22) Debugging is iterative hypothesis testing, not a single diagnostic pass: observe loss oscillation, hypothesize a data distribution shift, log per-batch distributions to validate, fix, and repeat — a training run is never truly bug-free.

(14:41) Open-source frameworks like VERL provide compounding benefits: the right abstraction layer separating algorithm from infrastructure, community validation that raises confidence in kernels and sharding, and direct reproducibility of published RL algorithms.

(20:33) Edge models operate under fundamentally different constraints: Liquid AI's models run at ~350M parameters on phones and vehicles, where tight latency budgets (say "open the window" in a car) and four-year hardware refresh cycles shape everything.

(22:58) Architecture must be co-designed with hardware topology: CPUs lack large tensor cores, so convolutions outperform attention on those devices — and Liquid iterates between architecture design and edge inference team profiling until latency budgets are met.

(28:22) Many published claims about RL algorithm properties are actually artifacts of bugs in the researchers' infrastructure or data pipelines — making ML research closer to experimental science than disciplined software engineering.