arXiv, 2026
Hi, I am a 4th year PhD candidate at Machine Learning Department, Carnegie Mellon University. I am advised by Aarti Singh and Drew Bagnell. I am also closely working with Wen Sun. I am partially supported by Two Sigma PhD fellowship.
In the past, I have interned at FAIR Paris (with Remi Munos), Amazon NYC (with Udaya Ghai and Dean Foster), and Microsoft Research NYC (with Akshay Krishnamurthy and Dylan Foster). I finished my master degree in MLD, advised by Kris Kitani. I completed my undergraduate at UC San Diego with CS and Math majors and I was advised by Sicun Gao.
Research
I am interested in the theory, science and application of interactive decision-making. My current study focuses on when and how we can achieve efficient and robust learning, from thinking about the two foundations of interactive decision-making: the environment (structure, data) and the policy (nowadays with foundation models). I am also interested in the application of principled decision-making algorithms in large-scale real-world applications, such as generative models and robotics.
/
arXiv, 2026
NeurIPS, 2025
We study the algorithmic trade-off between expert distillation and end-to-end RL in POMDPs, where the empirical results on visual based robotics locomotion tasks corrobate our theoretical findings.
NeurIPS ARLET workshop, 2025
ICML, 2025
ICLR, 2025
Through a large scale scientific study, we find 1) a scaling law of LLM self-improvement, 2) a loss of coverage through LLM post-training, 3) test-time scaling through self-improvement and many other interesting phenomena.
NeurIPS, 2024
We prove that offline contrastive-based method (e.g., DPO)
requires a stronger coverage property than online RL-based method (e.g., RLHF). We propose
Hybrid Preference Optimization to combine the benefits of both offline and online methods.
ICML, 2024
We consider a practical setting of hybrid RL where the agent only has access to offline observation data without action labels (e.g., videos of human demonstrations), and we show that it is possible to achieve efficient learning in this setting with a practical algorithm.
ICML, 2024
We introduce a new theoretical framework, RichCLD, in which the agent performs control based on high-dimensional observations, but the environment is governed by low-dimensional latent states and Lipschitz continuous dynamics.
COLT, 2023
[code]
We prove the benefit of representation learning on diverse source environments which allows efficient learning on the
source environment with the learned representation under the low-rank MDPs setting.
ICML, 2023
[code]
ICLR, 2023
Combining online data and offline data can solve RL with both statistical and computation efficiency.
Representation Learning for General-sum Low-rank Markov Games
Efficient Reinforcement Learning in Block MDPs: A Model-free Representation Learning Approach
No-regret Model-Based Meta RL for Personalized Navigation
Transform2Act: Learning a Transform-and-Control Policy for Efficient Agent Design
PC-MLP: Model-based Reinforcement Learning with Policy Cover Guided Exploration
Provably Efficient Model-based Policy Adaptation
ICLR, 2023
ICML, 2022
An efficient rich-observation RL algorithm that learns to decode from rich observations to latent states
(via adversarial training), while balancing exploration and exploitation.
L4DC, 2022
ICLR, 2022
ICML, 2021
[code]
A simple provably efficient model-based algorithm that achieves competitive performance in both dense reward
continuous control tasks and sparse reward control tasks that require efficient exploration.
ICML, 2020
[Project Page]
[code]
We study Sim-to-Real/policy transfer/policy adaptation under a model-based framework
resulting an algorithm that enjoys strong theoretical guarantees
and excellent empirical performance.
Talks
Rethinking the Foundation of LLM Post-Training: Signal and Objective
- Frontiers in Online Reinforcement Learning Workshop, March 2026.
- Stanford, March 2026.
- Harvard ML Foundations Group, February 2026.
Harnessing Additional Feedback in LLM Post-Training
- CMU Goomba Lab, October 2025. [Slides]
- FAIR Paris, July 2025.
Teaching & Service
Lecturer
- CMU 10734: Foundations of Autonomous Decision Making under Uncertainty (Fall 2024, Fall 2025)
Guest Lecturer
- Cornell CS6789: Foundations of Reinforcement Learning (Fall 2024)
- CMU 17740: Algorithmic Foundations of Interactive Learning (Fall 2024)
Teaching Assistant
- UCSD CSE291: Topics in Search and Optimization (Winter 2020)
- UCSD CSE154: Deep Learning (Fall 2019)
- UCSD CSE150: Introduction to AI: Search and Reasoning (Winter 2019, Spring 2020)
- UCSD CSE30: Computer Organization and Systems Programming (Spring 2019, Winter 2018)
- UCSD CSE11: Introduction to CS & OOP (Fall 2018)
Reviewer
- Conference: COLT (2025-), ICML (2021-), NeurIPS (2021-), ALT (2024-), ICLR (2022-), AAAI (2021-2022)
- Journal: TMLR, JMLR, IEEE Transactions on Signal Processing
- Outstanding Reviewer: NeurIPS 2022