Generative Policies
RL for expressive policy classes such as flow-based policies, diffusion/flow-matching models, and LLMs.
Qinwei Ma
Hi! I'm Qinwei Ma (马钦伟). You can call me Martin, or by my nicknames Martini/Aquapony. I am a first-year PhD student in the College of AI at Tsinghua University, advised by Prof. Alex Lamb. I received my undergraduate training from IIIS / Yao Class at Tsinghua University from 2021 to 2026.
My research is organized around a unified theme: reinforcement learning for generative models and world-model-grounded decision making. I am interested in general RL methods for expressive generative policies, including flow-based policies and LLMs, and in using world models not only to train policies, but also to constrain policy behavior through dynamics, feasibility, and long-horizon consistency.
From February to August 2026, I interned with the Tencent Hunyuan Video World Model Team, working on self-forcing, distillation, and RL post-training for video world models. I also previously collaborated with Prof. James Zou and colleagues at Stanford.
Previously, I had the honor to be mentored by many wonderful researchers, including Prof. Hang Zhao, Prof. Chuang Gan, Prof. Tong Zhang, and Dr. Lei Li. My long-term collaborator Jingzhe Shi has also been my high school and undergraduate classmate.
Before college, I won a gold medal in the 36th National High School Physics Olympiad, which directly granted me admission to Yao Class. I also took part in the Mathematics and Informatics Olympiads. I studied at Shanghai Foreign Language School, where German was my first foreign language.
Most importantly, I wish to thank my girlfriend Wanfei Li, who keeps supporting me in both research and daily life.
Feel free to contact me for potential collaboration!
News
- 2026-09: I started my PhD in the College of AI at Tsinghua University, advised by Prof. Alex Lamb.
- 2026: Intrinsic Entropy of Context Length Scaling in LLMs was selected as an ICLR 2026 Oral.
- 2026: PRISM-Physics was accepted to ICLR 2026.
- 2025-09: RapVerse was accepted to ICCV 2025.
- 2024-09: Scaling Law for Time Series Forecasting was accepted to NeurIPS 2024.
Research Interest
I want to build AI systems that are more capable, helpful, and controllable. My current focus is reinforcement learning for generative models and world-model-grounded decision making: how expressive models learn from feedback, how they explore and improve, and how world models can impose structure on action through physics, feasibility, and temporal consistency.
World Models
Learning models of dynamics that support planning, policy evaluation, and constraints over long horizons.
Learning Theory
Understanding the principles behind scaling behavior, feedback learning, and the limits of current model families.
What I'm working on now. I am thinking about ELBO-based RL and flow-matching policies, robotic world models, and long-horizon world models. At a high level, I am especially interested in methods that make generative policies both stronger and more grounded in the dynamics of the environments where they act.
Selected Publications
* denotes equal contribution; † denotes second-author contribution.
Timestep Weighting: A Hidden Key to Effective ELBO-Based Flow-Matching RL
Preprint, under review at ICLR 2027
Intrinsic Entropy of Context Length Scaling in LLMs
ICLR 2026 Oral
PRISM-Physics: Causal DAG-Based Process Evaluation for Physics Reasoning
ICLR 2026
A process-level physics reasoning benchmark based on directed acyclic graphs that encode causal dependencies among solution steps, with rule-based symbolic formula matching for robust validation.
ERA: Transforming VLMs into Embodied Agents via Embodied Prior Learning and Online Reinforcement Learning
Preprint
A two-stage framework that integrates embodied prior learning and online reinforcement learning for embodied vision-language agents.
Scaling Law for Time Series Forecasting
NeurIPS 2024
A theoretical and empirical study of scaling behavior in time series forecasting, including the role of look-back horizon.
Other Publications
- RapVerse: Coherent Vocals and Whole-Body Motions Generations from Text. Jiaben Chen, Xin Yan, Yihang Chen, Siyuan Cen, Zixin Wang, Qinwei Ma, Haoyu Zhen, Kaizhi Qian, Lie Lu, Chuang Gan. ICCV 2025. Project
- CHOPS: CHat with custOmer Profile Systems for Customer Service with LLMs. Jingzhe Shi, Jialuo Li, Qinwei Ma, Zaiwen Yang, Huan Ma, Lei Li. COLM 2024. Code arXiv OpenReview
- Explore-Execute Chain: Towards an Efficient Structured Reasoning Paradigm. Kaisen Yang*, Lixuan He*, Rushi Shah, Kaicheng Yang, Qinwei Ma, Dianbo Liu, Alex Lamb. Preprint, 2025. arXiv Code
- Gradient Imbalance in Direct Preference Optimization. Qinwei Ma, Jingzhe Shi, Can Jin, Jenq-Neng Hwang, Serge Belongie, Lei Li. Preprint. arXiv
Experience
Tsinghua University, College of AI
PhD student, 2026-present
Advisor: Prof. Alex Lamb.
Tsinghua University, IIIS / Yao Class
Undergraduate student, 2021-2026
Computer Science and Technology.
Tencent Hunyuan Video World Model Team
Research intern, February 2026-August 2026
Worked on self-forcing, distillation, and RL post-training for video world models.
Stanford University
Former research intern/collaborator
Past collaboration with Prof. James Zou and colleagues.
University of Washington
Research collaborator, March 2024-October 2025
Collaborated with Dr. Lei Li on NLP and time series research.
University of Illinois Urbana-Champaign
Research intern, January 2025-June 2025
Research advisor: Prof. Tong Zhang.
Northwestern University
Research collaborator, June 2024-October 2024
Research advisor: Prof. Zhaoran Wang.
MIT
Remote researcher, November 2023-April 2024
Research advisor: Prof. Chuang Gan.
Representative Honors and Awards
A complete list can be found in my CV.
- 2022-2024: Three school-level scholarships in scientific and technological achievements, artistic and cultural performance, and social work.
- 2024: First prize in the Tsinghua Challenge Cup.
- 2019: Gold medalist in the 36th National High School Physics Olympiad (CPhO), ranking 26th in the final round.
Personal Interests
Besides research, I make a serious effort to keep my life rich. A few things matter especially much to me.
Musical Theater
I am a huge fan of musical theater and have had the chance to perform in several productions:
- Aaron Burr in Hamilton (English)
- Schikaneder in Mozart!
- Favell in Rebecca
- Marius in Les Miserables
I have also played major roles in musical excerpts, including Gabe in Next to Normal and Raoul in The Phantom of the Opera. I directed a ten-minute mixed excerpt of Next to Normal for the tenth anniversary of the Tsinghua Musical Club.
Sports
Although I suffered a major injury in my sophomore year of college, I still try to stay active through soccer, badminton, and other sports.
Bridge
Bridge has been an important hobby since junior high school. In college, I was a member of the Tsinghua Bridge Team. I competed in the National College Students' Bridge Open Tournament Mixed Division and won second place. In separate team events, I also ranked eighth and sixth among the finalists with the Tsinghua team.
Werewolf
I am also a big fan of werewolf-style games. I was once invited to the popular Chinese variety show 京城大师赛, but could not attend because of a schedule conflict.
Skills and Service
Language and Skills
- Languages: Chinese (native), English (very proficient), German (B2).
- Programming: Python, C/C++, and related research tooling.
- Mathematics: Calculus, linear algebra, probability theory, statistics, abstract algebra, game theory, and related areas.
Service
- Teaching Assistant, Deep Self-Supervised Learning, Tsinghua IIIS, Spring 2026.
- Teaching Assistant, NLP, Shanghai Qizhi Institute and Xiongan Institute of AI, Autumn 2025; gave a guest lecture.
- ICLR 2025 reviewer.
- Part-time teacher in AI and physics.
Fun Facts
- My favorite musical is Hamilton. My favorite movie is Pulp Fiction.
- I hope to help improve legal systems, in China and globally, around technological safety and privacy, as well as gender equality and women's rights.