尚靖桓 Jinghuan Shang

Research Scientist at The AI Institute

I obtained my CS Ph.D. from Stony Brook University under the supervision of Prof. Michael Ryoo. Prior to this, I received my BS in CS from IEEE Honored Class at Shanghai Jiao Tong University (SJTU) in 2018, with honor from Shanghai. I was an Research Intern at The AI Insitute. I also interned at Motional.

Research interest: Foundation models and action policies for embodied agents with visual, sequential, and vision-language-action representations.

I'm always looking for self-motivated robots to collaborate with 😝

[CV] [GitHub] [Notes] [Scholar]

News

2024/09 - Continual Learning with Global Alignment is accepted to NeurIPS 2024!

2024/09 - Theia, a Vision Foundation Model for robotics. It's smaller but having much stronger robot learning performance. CoRL 2024 acceptance!

2024/07 - LLaRA, an instruction-tuned VLM for robot policies.

2024/01 - Crossway Diffusion is accepted to ICRA 2024.

2023/06 - Introducing Active Vision Reinforcment Learning under Limited Visual Observability and the library Active-Gym.

Publications and Preprints

Theia: Distilling Diverse Vision Foundation Models for Robot Learning
Jinghuan Shang, Karl Schmeckpeper, Brandon B. May, Maria Vittoria Minniti, Tarik Kelestemur, David Watkins, Laura Herlant
CoRL 2024
[Project Page] [arXiv] [Demo] GitHub Repo stars
Continual Learning with Global Alignment
Xueying Bai, Jinghuan Shang, Yifan Sun, Niranjan Balasubramanian
NeurIPS 2024
Paper coming soon.
LLaRA: Supercharging Robot Learning Data for Vision-Language Policy
Xiang Li, Cristina Mata, Jongwoo Park, Kumara Kahatapitiya, Yoo Sung Jang, Jinghuan Shang, Kanchana Ranasinghe, Ryan Burgert, Mu Cai, Yong Jae Lee, and Michael S. Ryoo
arXiv
[arXiv] GitHub Repo stars
Crossway Diffusion: Improving Diffusion-based Visuomotor Policy via Self-supervised Learning
Xiang Li, Varun Belagali, Jinghuan Shang and Michael S. Ryoo
ICRA 2024
[arXiv] GitHub Repo stars
Active Vision Reinforcement Learning under Limited Visual Observability
Jinghuan Shang and Michael S. Ryoo
NeurIPS 2023
[Project Page] [arXiv] GitHub Repo stars GitHub Repo stars
Learning Viewpoint-Agnostic Visual Representations by Recovering Tokens in 3D Space
Jinghuan Shang, Srijan Das, and Michael S. Ryoo
NeurIPS 2022
[Project Page] [arXiv] GitHub Repo stars
Does Self-supervised Learning Really Improve Reinforcement Learning from Pixels?
Xiang Li, Jinghuan Shang, Srijan Das, and Michael S. Ryoo
NeurIPS 2022
[arXiv]
Neural Neural Textures Make Sim2Real Consistent
Ryan Burgert, Jinghuan Shang, Xiang Li, and Michael S. Ryoo
CoRL 2022
[Project Page] [Code] [arXiv]
StARformer: Transformer with State-Action-Reward Representations for Robot Learning
Jinghuan Shang, Xiang Li, Kumara Kahatapitiya, Yu-Cheol Lee, Michael S. Ryoo
IEEE TPAMI, Special Issue on Transformer Models in Vision, 2022
[PDF] [IEEE Xplore] GitHub Repo stars
StARformer: Transformer with State-Action-Reward Representations for Visual Reinforcement Learning
Jinghuan Shang, Kumara Kahatapitiya, Xiang Li, and Michael S. Ryoo
ECCV 2022
[arXiv] [Video] [Poster] GitHub Repo stars
Self-Supervised Disentangled Representation Learning for Third-Person Imitation Learning
Jinghuan Shang and Michael S. Ryoo
IROS 2021
[arXiv] [Talk] [Digest Slide]

Honors and Awards

2022/10- NeurIPS 2022 Scholar Award
2018/08- Merit Scholarship from the Department of Computer Science at Stony Brook University
2018/06- Outstanding graduate among all graduates from universities in Shanghai
2016- 1st Prize in Shanghai Division of China Undergraduate Mathematical Contest in Modeling (CUMCM)
2015-2017- 3 times of Academic Excellence Scholarship at Shanghai Jiao Tong University

Fun

I love cooking.

I solve some algorithm problems in my spare time. Luckily got Top 10 in 2020 SBU ICPC Slection Contest. Here is my [Leetcode].

Last modified 2024/02. Style Credit: latex.css