Interactive ML for large language models

Our recent research focus is to study IML problems that arise from large language models, including alignment, post-training, reasoning, and test-time scaling — viewing a model's generated outputs as actions and bringing IML tools to bear on them. We are currently developing efficient algorithms inspired by learning theory and evaluating them on real-world tasks. Some of these efforts are in collaboration with Krafton AI.

• A unified theoretical framework for LLM Alignment (arXiv'25)
• Theoretical guarantees of the popular heuristic called 'online DPO' and its generalization (ICML'26)

Reinforcement learning and bandits

We develop efficient algorithms for reinforcement learning and bandit problems, with an emphasis on rigorous theoretical guarantees. The multi-armed/contextual bandit problem can be viewed as a special case of reinforcement learning with no states. No, this does not mean existing RL algorithms are efficient in the bandit setting -- algorithms designed for bandits typically work much better. Bandit algorithms have been deployed in real-world systems, including online news recommendation at Microsoft (paper link) and large-scale experimentation and A/B testing at Meta Platforms. Our research on bandits is funded in part by Meta Platforms, with whom we maintain an ongoing research collaboration.

• Minimum empirical divergence-style algorithms (AISTATS'22, NeurIPS'23, AISTATS'25, TMLR'26)
• Accelerated rates when there are many good arms (ICML'23, AISTATS'25)
• Towards parameter-free algorithms, i.e., perform as well as knowing the unknown such as noise level, norm of the parameter to be learned, etc. (NeurIPS'22, ICML'23, ICML'24, AISTATS'25)
• Fixed budget algorithms and their guarantees (as opposed to fixed confidence guarantees) (ICLR'26, ICML'26)
• Logistic/generalized linear/sparse/low-rank models with sample/computational efficiency (NeurIPS'17, NeurIPS'20, ICML'21a, ICML'21b, AAAI'22, AISTATS'22, NeurIPS'22, AISTATS'24, ICML'24, NeurIPS'24, AISTATS'26)

Learning theory

We develop learning-theoretic tools for designing statistically efficient machine learning algorithms. Our work spans both general learning theory, including methods that adapt to favorable properties of the data such as low noise, and practical confidence bounds for quantifying statistical uncertainty. These ideas often serve as key building blocks for efficient algorithms in bandits, reinforcement learning, and other interactive learning problems. Some examples:

• Variance-adaptive regression algorithms with guarantees that improve in low feature-conditional noise regime (COLT'26)
• Improved PAC-Bayes bounds with a sharper divergence (COLT'25) and variance-adaptive guarantees (COLT'24)
• Confidence Sequences via Online Learning” — invited talk at UW–Madison SILO (Fall '24) [video]
• Variance-adaptive confidence sets for linear bandits, enabling practical adaptation to unknown noise levels (ICML '24)
• Improved unified confidence bounds for generalized linear models (NeurIPS '24)