尚靖桓 Jinghuan Shang

Research Scientist at Robotics and AI Institute (RAI)

---- aka The AI Institute aka Boston Dynamics AI Institute

I obtained my CS Ph.D. from Stony Brook University under the supervision of Prof. Michael Ryoo. Prior to this, I received my BS in CS from IEEE Honored Class at Shanghai Jiao Tong University (SJTU) in 2018, with honor from Shanghai. I was an Research Intern at RAI. I also interned at Motional.

Research interest: Foundation models and action policies for embodied agents with visual, sequential, and vision-language-action representations.

I'm always looking for self-motivated robots to collaborate with 😝

[CV] [GitHub] [Notes] [Scholar]

News

2026/01 - VER, MoE multi-teacher visual representation distillation for policy learning -- ICLR 2026 acceptance

2025/12 - Sceniris, a super fast procedural scene generation tool. >200x speed up!

2025/12 - Anytask, automatic generative simulation and synthetic robot data collection system

2025 - Serving as the Web Chair of CoRL 2025

2025/01 - LLaRA is accepted to ICLR 2025!

2024/09 - Continual Learning with Global Alignment is accepted to NeurIPS 2024!

2024/09 - Theia, a Vision Foundation Model for robotics. It's smaller but having much stronger robot learning performance. CoRL 2024 acceptance!

2024/01 - Crossway Diffusion is accepted to ICRA 2024.

2023/06 - Introducing Active Vision Reinforcment Learning under Limited Visual Observability and the library Active-Gym.

Publications and Preprints

VER: Vision Expert Transformer for Robot Learning via Foundation Distillation and Dynamic Routing
Yixiao Wang, Mingxiao Huo, Zhixuan Liang, Yushi Du, Lingfeng Sun, Haotian Lin, Jinghuan Shang, Chensheng Peng, Mohit Bansal, Mingyu Ding, Masayoshi Tomizuka
ICLR 2026
[Paper] [arXiv]
Sceniris: a Fast Procedural Scene Generation Framework
Jinghuan Shang, Harsh Patel, Ran Gong, Karl Schmeckpeper
Pre-print 2025
[arXiv] GitHub Repo stars
AnyTask: an Automated Task and Data Generation Framework for Advancing Sim-to-Real Policy Learning
Ran Gong*, Xiaohan Zhang*, Jinghuan Shang*, Maria Vittoria Minniti*, Jigarkumar Patel, Valerio Pepe, Riedana Yan, Ahmet Gundogdu, Ivan Kapelyukh, Ali Abbas, Xiaoqiang Yan, Harsh Patel, Laura Herlant, Karl Schmeckpeper
*The first four authors contributed equally
Pre-print 2025
[Project Page] [arXiv]
Theia: Distilling Diverse Vision Foundation Models for Robot Learning
Jinghuan Shang, Karl Schmeckpeper, Brandon B. May, Maria Vittoria Minniti, Tarik Kelestemur, David Watkins, Laura Herlant
CoRL 2024
[Project Page] [arXiv] [Demo] GitHub Repo stars
Continual Learning with Global Alignment
Xueying Bai, Jinghuan Shang, Yifan Sun, Niranjan Balasubramanian
NeurIPS 2024
[Paper] [Code]
LLaRA: Supercharging Robot Learning Data for Vision-Language Policy
Xiang Li, Cristina Mata, Jongwoo Park, Kumara Kahatapitiya, Yoo Sung Jang, Jinghuan Shang, Kanchana Ranasinghe, Ryan Burgert, Mu Cai, Yong Jae Lee, and Michael S. Ryoo
ICLR 2025; Spotlight at LangRob@CoRL, 2024
[arXiv] GitHub Repo stars
Crossway Diffusion: Improving Diffusion-based Visuomotor Policy via Self-supervised Learning
Xiang Li, Varun Belagali, Jinghuan Shang and Michael S. Ryoo
ICRA 2024
[arXiv] GitHub Repo stars
Active Vision Reinforcement Learning under Limited Visual Observability
Jinghuan Shang and Michael S. Ryoo
NeurIPS 2023
[Project Page] [arXiv] GitHub Repo stars GitHub Repo stars
Learning Viewpoint-Agnostic Visual Representations by Recovering Tokens in 3D Space
Jinghuan Shang, Srijan Das, and Michael S. Ryoo
NeurIPS 2022
[Project Page] [arXiv] GitHub Repo stars
Does Self-supervised Learning Really Improve Reinforcement Learning from Pixels?
Xiang Li, Jinghuan Shang, Srijan Das, and Michael S. Ryoo
NeurIPS 2022
[arXiv]
Neural Neural Textures Make Sim2Real Consistent
Ryan Burgert, Jinghuan Shang, Xiang Li, and Michael S. Ryoo
CoRL 2022
[Project Page] [Code] [arXiv]
StARformer: Transformer with State-Action-Reward Representations for Robot Learning
Jinghuan Shang, Xiang Li, Kumara Kahatapitiya, Yu-Cheol Lee, Michael S. Ryoo
IEEE TPAMI, Special Issue on Transformer Models in Vision, 2022
[PDF] [IEEE Xplore] GitHub Repo stars
StARformer: Transformer with State-Action-Reward Representations for Visual Reinforcement Learning
Jinghuan Shang, Kumara Kahatapitiya, Xiang Li, and Michael S. Ryoo
ECCV 2022
[arXiv] [Video] [Poster] GitHub Repo stars
Self-Supervised Disentangled Representation Learning for Third-Person Imitation Learning
Jinghuan Shang and Michael S. Ryoo
IROS 2021
[arXiv] [Talk] [Digest Slide]

Honors and Awards

2022/10- NeurIPS 2022 Scholar Award
2018/08- Merit Scholarship from the Department of Computer Science at Stony Brook University
2018/06- Outstanding graduate among all graduates from universities in Shanghai
2016- 1st Prize in Shanghai Division of China Undergraduate Mathematical Contest in Modeling (CUMCM)
2015-2017- 3 times of Academic Excellence Scholarship at Shanghai Jiao Tong University

Fun

I love cooking.

I solve some algorithm problems in my spare time. Luckily got Top 10 in 2020 SBU ICPC Slection Contest. Here is my [Leetcode].

Last modified 2024/02. Style Credit: latex.css