Reinforcement learning for LLMs
Policy gradients, PPO and its successors, reward design and evaluation: how to train language models with feedback and keep the reward honest.
Research
The work behind the consulting and the courses: what I study, what I write down, and what I build to understand it. The writing is in English.
Areas
Diffusion language models
Non-autoregressive text generation, parallel decoding and what it changes for latency and quality.
Self-supervised and graph learning
Representation learning, contrastive objectives and learning on graphs.
Sequence models and ML systems
Transformers, state-space models, and the infrastructure that takes experiments to production.
Selected writing
Short, reusable notes and worked derivations on my blog.
- RL Bite: Monte Carlo Tree Search Apr 2025
- RL Bite: Monotonic Policy Improvement and Deriving PPO Apr 2025
- RL Bite: Policy Gradient and REINFORCE Mar 2025
- RL Bite: Learning the Q Function Mar 2025
- TLDR: Graph Contrastive Learning and Representation Scattering Feb 2025
Projects
-
Awesome T5 and Awesome SSM
Curated reading lists for encoder-decoder models and state-space models, with study notes.
-
Hathor
A text-to-audio tool with voice cloning, voice design and an HTTP API.
More on GitHub.
Reading list
New papers I break down into short modules are on the papers page. If a topic here fits a problem you have, see how to work with me.