Zhoutong’s Research Log

Deep dives into LLM architecture, post-training, and the bridge between research and production.

Rollout–Training Mismatch in LLM RL

What PPO and rollout correction already handle, where forward alignment helps, and how score centering and REINFORCE Zero address the remaining disagreement.

Nov 8, 2026
25 min

Entropy in LLM RL: Dynamics and Control

Why entropy collapse and uncontrolled growth can stall RL—and how update rules, regularization, and sampling shape effective learning.

Oct 21, 2026
22 min

Weight Synchronization in LLM RL: From Optimizer Step to Rollout

How distributed trainers deliver updated weights to rollout workers, reduce transfer and installation costs, and safely switch generation to a new policy.

Oct 5, 2026
40 min

Low-Precision RL: From Rollout to Update

How precision choices across rollout, training, and synchronization shape the policy optimized by LLM reinforcement learning.

Sep 1, 2026
30 min

From Linear Attention to KDA

A technical note on the recurrent-state lineage from linear attention to KDA, K3’s hybrid stack, and DSA as a sparse-retrieval contrast.

Aug 15, 2026
21 min

Every Token Gets a Weight

A token-weighting view of PPO, GRPO, DAPO, CISPO, SDPO, SRPO, RLSD, and the newer conditional-trust direction in on-policy distillation.

Jun 14, 2026
28 min

On-Policy Self-Distillation for RL

Why on-policy self-distillation is a natural complement to standard RL, and how SDPO, OPSD, and GLM-5 implement it.

Mar 8, 2026
15 min

Rethinking Supervised Fine-Tuning: A First-Principles Derivation

Why standard SFT causes catastrophic forgetting, and how contextual on-policy self-distillation fixes it from first principles.

Feb 17, 2026
15 min

Schematics of (a) SwiGLU FFN, (b) MoE FFN, and (c) STEM with a single prefetched token embedding.

The Rise of Static Memory in LLMs

How PLE, STEM, Engram, and LongCat decouple knowledge storage from reasoning by moving static facts into embedding modules.

Feb 1, 2026
13 min
No matching items