Designing reliable training infrastructure for edge & foundation models with Tianshu Yu at Liquid AI

Show Notes:

Most ML training bugs don't announce themselves. They hide in floating-point rounding, batch distribution drift, and concurrency issues that only emerge across thousands of GPUs. The hard part isn't writing the algorithm — it's building infrastructure disciplined enough to tell the difference between a numerically equivalent optimization and a quietly corrupted model.

We speak with Tianshu Yu, member of the technical staff at Liquid AI, about the real work of building reliable ML systems at scale. Tianshu previously led reinforcement learning infrastructure at ByteDance, where he was a core contributor to VERL — one of the most widely used open-source RL training frameworks. He now works on pre-training and architecture design for edge-deployed vision language models.

The conversation covers the three-phase structure of reinforcement learning, why checkpoints serve debugging, fault recovery, and model selection simultaneously, and how capability collapse emerges when data distributions concentrate during fine-tuning. We also get into what changes when you design models for edge hardware: the shift from attention to convolution-dominant ops, latency budgets as a first-class architectural constraint, and why a 350M parameter model has meaningfully different engineering requirements than a frontier model.

‍

Key Takeaways:

(00:00) Introduction.

(00:52) Verl, ByteDance's open-source RL framework, addressed a major cause of training instability: mismatches between rollout distributions and training distributions in reinforcement learning.

(02:31) Reinforcement learning lets a model learn from samples it generates itself, monitored by training reward until plateau while holistically watching eval benchmarks.

(03:00) Capability collapse is a real risk during RL fine-tuning: optimizing on a concentrated data distribution (e.g., English-only tasks) can degrade unrelated capabilities like multilingual performance, especially in smaller models.

(05:30) Later training stages demand higher-quality data — signal-to-noise ratio matters more as training progresses, making data selection increasingly critical.

(06:53) Checkpoints serve three distinct purposes in large-scale training: monitoring quality curves to spot anomalies, enabling fault recovery after hardware or communication failures, and supporting resumption across mid-training code updates.

(09:02) Long training runs are "fixing the airplane while it's flying" — ByteDance's 54-day run saw ~170 code updates per week, with checkpoints enabling resume-after-fix rather than costly restarts.

(10:59) Training bugs come from three sources — dataset mix, infrastructure (e.g., sharding producing incorrect loss), and algorithm design — and disentangling them is an iterative, hypothesis-driven debugging process; no training run is ever truly bug-free.

(15:02) Open-source training frameworks contribute a battle-tested baseline, the right abstraction layer separating infra from algorithm concerns, and directly reproducible implementations of published algorithms.

(18:30) At Liquid AI, Tianshu's work spans inference stack engineering, MoE training stabilization, vision RL frameworks, and now pre-training and architecture design for next-generation vision language models.

(20:30) Edge models (~350M parameters) face constraints beyond size: tight real-time latency budgets (e.g., in-car voice commands), memory limits, and hardware that iterates on four-year cycles rather than GPU-style exponential curves.

(22:58) Architecture must match the chip: CPUs lack large tensor cores optimized for matrix multiplication, so attention runs inefficiently there while convolutions perform much better — Liquid designs different architectures for different hardware targets.

(26:31) Architecture selection isn't done at runtime; it's an iterative loop where pre-training validates accuracy, the edge inference team profiles latency on target hardware, and the architecture is refined until it fits the latency budget.

(28:15) A surprising takeaway: many published claims about algorithm properties are artifacts of bugs in infrastructure or data pipelines — ML research increasingly resembles experimental science, where an unclean "instrument" can invalidate years of results.

‍

Resources Mentioned

VERL - github.com/verl-project/verl

Robust LLM Training Infrastructure at ByteDance - arxiv.org/abs/2509.16293

‍

Thanks for listening to "Hello Agent!: The podcast at the intersection of data & agents. Remember to subscribe so you don't miss an episode.