Biography

I am a staff research scientist at Horizon Robotics. Before that, I was a senior research engineer at DJI Auto, leading the 4D reconstruction team. We achieved pure-vision-based city-scale 3D reconstruction. I received my Ph.D. degree at the University of Adelaide, Australia, in 2022, advised by Prof. Chunhua Shen. Before coming to Adelaide,  I worked as a Partner at Humanplus Intelligent Robotics Co., Ltd, and led a team to develop an active-stereo camera. It is now sold worldwide and widely applied in many industrial scenarios.

I have a broad research interest in CV and ML, including 1) World model / Multi-modal model for E2E driving, 2) 3D reconstruction and generation. Our LeReS was shortlisted in the CVPR2021 Best Paper finalist.

I am a member of AnySyn3D, a research interest group that conducts various topics about the 3D.

Actively recruiting research interns! If you are interested in working with me and are self-motivated, feel free to drop me an email. Our team has multiple positions within the world model and real2sim2real rendering system.

News

  • Spe., 2024, DC-Gaussian is accepted to NIPS.
  • April, 2024, our SSIW is accepted to IJCV.
  • April, 2024, GeoWizard is accepted to ECCV. (Congrat. to Fu Xiao and thanks to all co-authors.)
  • Mar., 2024, GaussianPro is accepted to ICML. (Congrat. to Kai Cheng.)
  • Feb., 2024, Two CVPR papers are accepted.
  • Jan., 2024, Two ICLR papers are accepted (GIM got Spotlight!, Congrat. to Xuelun Shen.)
  • Jun., 2023, Three ICCV papers are accepted.
  • Mar., 2023, One CVPR is accepted.
  • Mar., 2023, We achieve the CHAMPION on CVPR 2023 monocular depth perception challenge.
  • Dec., 2022, One TPAMI paper is accepted.
  • Aug., 2022, One NIPS paper is accepted.
  • Jul., 2022, Two ECCV papers are accepted.
  • Apr., 2022, One paper is accepted to Transactions on Robotics.
  • Mar., 2022, One CVPR paper is accepted.
  • Jun., 2021, One TPAMI paper is accepted.
  • Jun., 2021, Our work ‘Learning to Recover 3D Scene Shape from a Single Image’ is in the CVPR’21 BEST PAPER CANDIDATE.

Conference

Depth Any Video with Scalable Synthetic Data

Honghui Yang*, Di Huang*Wei YinChunhua ShenHaifeng LiuXiaofei He, Binbin LinWanli OuyangTong He

2024

WebPage / Arxiv / Code / HuggingFace

HE-DriveHuman-Like End-to-End Driving with Vision Language Models

Junming Wang*Xingyu Zhang*Zebin XingSongen GuXiaoyang GuoYang HuZiying SongQian ZhangXiaoxiao LongWei Yin

2024

WebPage / Arxiv / Code

Adaptive Fusion of Single-View and Multi-View Depth for Autonomous Driving

Junda Cheng*, Wei Yin*Kaixuan WangXiaozhi Chen, Shijie Wang, Xin Yang

CVPR, 2024

Arxiv / Code

GeoWizard: Unleashing the Diffusion Priors for 3D Geometry Estimation from a Single Image

Xiao Fu*Wei Yin*Mu Hu*Kaixuan WangYuexin MaPing TanShaojie ShenDahua LinXiaoxiao Long

ECCV, 2024

WebPage / HuggingFace / Arxiv / Code

GIM: Learning Generalizable Image Matcher From Internet Videos

Xuelun Shen*, Zhipeng Cai*, Wei Yin*, Matthias MüllerZijun LiKaixuan WangXiaozhi ChenCheng Wang,

ICLR, 2024, (Spotlight, top-5%)

WebPage / HuggingFace / Arxiv / Code

GaussianPro: 3D Gaussian Splatting with Progressive Propagation

Kai Cheng, Xiaoxiao Long, Kaizhi Yang, Yao Yao, Wei Yin, Yuexin Ma, Wenping Wang, Xuejin Chen

Arxiv

WebPage / Arxiv / Code

UC-NeRF: Neural Radiance Field for Under-Calibrated Multi-view Cameras in Autonomous Driving

Kai ChengXiaoxiao LongWei YinJin WangZhiqiang WuYuexin MaKaixuan WangXiaozhi ChenXuejin Chen

ICLR, 2024

WebPage / Arxiv / Code

FrozenRecon: Pose-free 3D Scene Reconstruction with Frozen Depth Models

Guangkai Xu*, Wei Yin*Hao ChenChunhua Shen, Kai Cheng, Feng Zhao

ICCV, 2023

WebPage / Arxiv / Code

Metric3D: Towards Zero-shot Metric 3D Prediction from A Single Image

Wei YinChi ZhangHao ChenZhipeng CaiGang YuKaixuan WangXiaozhi ChenChunhua Shen

ICCV, 2023 (Champion in CVPR2023 Monocular Depth Challenge)

WebPage / HuggingFace / Arxiv / Code

Hierarchical Normalization for Robust Monocular Depth Estimation

Chi Zhang, Wei Yin, Zhibin Wang, Gang Yu, Bin Fu, Chunhua Shen

NIPS, 2022

Arxiv

Retrieval Augmented Classification for Long-Tail Visual Recognition

Alexander Long, Wei Yin, Thalaiyasingam Ajanthan, Vu Nguyen, Pulak Purkait, Ravi Garg, Alan Blair, Chunhua Shen, Anton van den Hengel

CVPR, 2022

Arxiv

Learning to Recover 3D Scene Shape from a Single Image

Wei Yin,  Jianming Zhang, Oliver Wang, Simon Niklaus, Simon Chen, Yifan Liu, Chunhua Shen

CVPR, 2021 (Best Paper Candidate)

Arxiv / Code

Enforcing geometric constraints of virtual normal for depth prediction

Wei Yin,  Yifan Liu, Chunhua Shen, Youliang Yan

ICCV, 2019

Arxiv / Code

Journal

Metric3D v2: A Versatile Monocular Geometric Foundation Model for Zero-shot Metric Depth and Surface Normal Estimation

Mu Hu*, Wei Yin*,  Chi ZhangZhipeng Cai, Kaixuan WangXiaoxiao Long, Hao Chen, Gang Yu, Chunhua Shen,  Shaojie Shen

TPAMI, 2024

WebPage / HuggingFace / Arxiv / Code

Scaling Up Multi-domain Semantic Segmentation with Sentence Embeddings

Wei Yin, Yifan Liu, Chunhua Shen, Baichuan Sun, Anton van den Hengel

IJCV, 2024

Arxiv / Code

SC-DepthV3: Robust Self-supervised Monocular Depth Estimation for Dynamic Scenes

Libo Sun, Jiawang Bian, Huangyin Zhan, Wei Yin, Ian Reid, Chunhua Shen

TPAMI, 2023

Arxiv / Code

Improving Monocular Visual Odometry Using Learned Depth

Libo Sun, Wei Yin, Enze Xie, Zhengrong Li, Changming Sun, Chunhua Shen

IEEE Transactions on Robotics (TRO), 2022

Arxiv

Towards Accurate Reconstruction of 3D Scene Shape from A Single Monocular Image

Wei Yin Jianming Zhang, Oliver Wang, Simon Niklaus, Simon Chen, Yifan Liu, Chunhua Shen

TPAMI, 2022

Arxiv / Code

Virtual Normal: Enforcing Geometric Constraints for Accurate and Robust Depth Prediction

Wei Yin,  Yifan Liu, Chunhua Shen

TPAMI, 2021

Arxiv / Code

Previous Interns & Projects

Ph.D. Thesis

3D Scene Reconstruction from A Monocular Image, The University of Adelaide, 2022

Professional Activities

  • Journal Reviewer: Transactions on Robotics, Transactions on Pattern Analysis and Machine Intelligence
  • Conference Reviewer: ICCV, CVPR, AAAI, NIPS, ECCV, ICLR, ICML

Employment

Prominent Awards

  • Achieve the Champion in CVPR2023 Depth Estimation Challenge (2023)
  • LeReS is shortlisted in CVPR 2021 Best Paper Candidates (2021)
  • Data61 Top-up Scholarship (2018~2021)
  • Adobe Gift Fund for Research (Twice, 2020)
  • China National Scholarship (2016)