Biography
I am a staff research scientist at Horizon Robotics. Before that, I was a senior research engineer at DJI Auto, leading the 4D reconstruction team. We achieved pure-vision-based city-scale 3D reconstruction. I received my Ph.D. degree at the University of Adelaide, Australia, in 2022, advised by Prof. Chunhua Shen. Before coming to Adelaide, I worked as a Partner at Humanplus Intelligent Robotics Co., Ltd, and led a team to develop an active-stereo camera. It is now sold worldwide and widely applied in many industrial scenarios.
I have a broad research interest in CV and ML, including 1) World model / Multi-modal model for E2E driving, 2) 3D reconstruction and generation. Our LeReS was shortlisted in the CVPR2021 Best Paper finalist.
I am a member of AnySyn3D, a research interest group that conducts various topics about the 3D.
Actively recruiting research interns! If you are interested in working with me and are self-motivated, feel free to drop me an email. Our team has multiple positions within the world model and real2sim2real rendering system.
News
- Spe., 2024, DC-Gaussian is accepted to NIPS.
- April, 2024, our SSIW is accepted to IJCV.
- April, 2024, GeoWizard is accepted to ECCV. (Congrat. to Fu Xiao and thanks to all co-authors.)
- Mar., 2024, GaussianPro is accepted to ICML. (Congrat. to Kai Cheng.)
- Feb., 2024, Two CVPR papers are accepted.
- Jan., 2024, Two ICLR papers are accepted (GIM got Spotlight!, Congrat. to Xuelun Shen.)
- Jun., 2023, Three ICCV papers are accepted.
- Mar., 2023, One CVPR is accepted.
- Mar., 2023, We achieve the CHAMPION on CVPR 2023 monocular depth perception challenge.
- Dec., 2022, One TPAMI paper is accepted.
- Aug., 2022, One NIPS paper is accepted.
- Jul., 2022, Two ECCV papers are accepted.
- Apr., 2022, One paper is accepted to Transactions on Robotics.
- Mar., 2022, One CVPR paper is accepted.
- Jun., 2021, One TPAMI paper is accepted.
- Jun., 2021, Our work ‘Learning to Recover 3D Scene Shape from a Single Image’ is in the CVPR’21 BEST PAPER CANDIDATE.
Conference
Depth Any Video with Scalable Synthetic Data
Honghui Yang*, Di Huang*, Wei Yin, Chunhua Shen, Haifeng Liu, Xiaofei He, Binbin Lin†, Wanli Ouyang, Tong He
2024
WebPage / Arxiv / Code / HuggingFace
HE-Drive: Human-Like End-to-End Driving with Vision Language Models
Junming Wang*, Xingyu Zhang*, Zebin Xing, Songen Gu, Xiaoyang Guo, Yang Hu, Ziying Song, Qian Zhang, Xiaoxiao Long, Wei Yin†
2024
Adaptive Fusion of Single-View and Multi-View Depth for Autonomous Driving
Junda Cheng*, Wei Yin*, Kaixuan Wang, Xiaozhi Chen, Shijie Wang, Xin Yang
CVPR, 2024
GeoWizard: Unleashing the Diffusion Priors for 3D Geometry Estimation from a Single Image
Xiao Fu*, Wei Yin*, Mu Hu*, Kaixuan Wang, Yuexin Ma, Ping Tan, Shaojie Shen, Dahua Lin, Xiaoxiao Long
ECCV, 2024
WebPage / HuggingFace / Arxiv / Code
GIM: Learning Generalizable Image Matcher From Internet Videos
Xuelun Shen*, Zhipeng Cai*, Wei Yin*, Matthias Müller, Zijun Li, Kaixuan Wang, Xiaozhi Chen, Cheng Wang,
ICLR, 2024, (Spotlight, top-5%)
WebPage / HuggingFace / Arxiv / Code
GaussianPro: 3D Gaussian Splatting with Progressive Propagation
Kai Cheng, Xiaoxiao Long, Kaizhi Yang, Yao Yao, Wei Yin, Yuexin Ma, Wenping Wang, Xuejin Chen
Arxiv
UC-NeRF: Neural Radiance Field for Under-Calibrated Multi-view Cameras in Autonomous Driving
Kai Cheng, Xiaoxiao Long, Wei Yin, Jin Wang, Zhiqiang Wu, Yuexin Ma, Kaixuan Wang, Xiaozhi Chen, Xuejin Chen
ICLR, 2024
FrozenRecon: Pose-free 3D Scene Reconstruction with Frozen Depth Models
Guangkai Xu*, Wei Yin*, Hao Chen, Chunhua Shen, Kai Cheng, Feng Zhao
ICCV, 2023
Metric3D: Towards Zero-shot Metric 3D Prediction from A Single Image
Wei Yin, Chi Zhang, Hao Chen, Zhipeng Cai, Gang Yu, Kaixuan Wang, Xiaozhi Chen, Chunhua Shen
ICCV, 2023 (Champion in CVPR2023 Monocular Depth Challenge)
WebPage / HuggingFace / Arxiv / Code
Hierarchical Normalization for Robust Monocular Depth Estimation
Chi Zhang, Wei Yin, Zhibin Wang, Gang Yu, Bin Fu, Chunhua Shen
NIPS, 2022
Retrieval Augmented Classification for Long-Tail Visual Recognition
Alexander Long, Wei Yin, Thalaiyasingam Ajanthan, Vu Nguyen, Pulak Purkait, Ravi Garg, Alan Blair, Chunhua Shen, Anton van den Hengel
CVPR, 2022
Learning to Recover 3D Scene Shape from a Single Image
Wei Yin, Jianming Zhang, Oliver Wang, Simon Niklaus, Simon Chen, Yifan Liu, Chunhua Shen
CVPR, 2021 (Best Paper Candidate)
Enforcing geometric constraints of virtual normal for depth prediction
Wei Yin, Yifan Liu, Chunhua Shen, Youliang Yan
ICCV, 2019
Journal
Metric3D v2: A Versatile Monocular Geometric Foundation Model for Zero-shot Metric Depth and Surface Normal Estimation
Mu Hu*, Wei Yin*, Chi Zhang, Zhipeng Cai, Kaixuan Wang, Xiaoxiao Long, Hao Chen, Gang Yu, Chunhua Shen, Shaojie Shen
TPAMI, 2024
WebPage / HuggingFace / Arxiv / Code
Scaling Up Multi-domain Semantic Segmentation with Sentence Embeddings
Wei Yin, Yifan Liu, Chunhua Shen, Baichuan Sun, Anton van den Hengel
IJCV, 2024
SC-DepthV3: Robust Self-supervised Monocular Depth Estimation for Dynamic Scenes
Libo Sun, Jiawang Bian, Huangyin Zhan, Wei Yin, Ian Reid, Chunhua Shen
TPAMI, 2023
Improving Monocular Visual Odometry Using Learned Depth
Libo Sun, Wei Yin, Enze Xie, Zhengrong Li, Changming Sun, Chunhua Shen
IEEE Transactions on Robotics (TRO), 2022
Towards Accurate Reconstruction of 3D Scene Shape from A Single Monocular Image
Wei Yin, Jianming Zhang, Oliver Wang, Simon Niklaus, Simon Chen, Yifan Liu, Chunhua Shen
TPAMI, 2022
Virtual Normal: Enforcing Geometric Constraints for Accurate and Robust Depth Prediction
Wei Yin, Yifan Liu, Chunhua Shen
TPAMI, 2021
Previous Interns & Projects
- Junda Chen (2023, Adaptive Fusion Depth, CVPR)
- Kai Cheng (2022-2023, UC-NerF CVPR, GaussianPro ICML)
- Xuelun Shen (2022-2023, GIM, ICML)
- Guangkai Xu (2022, FronzenRecon CVPR)
- Xiao Fu (2023, GeoWizard, ECCV)
- Mu Hu (2023, Metric3dV2, TPAMI)
- Rui Li (2022, Multi-frame Depth Estimation, CVPR)
Ph.D. Thesis
3D Scene Reconstruction from A Monocular Image, The University of Adelaide, 2022
Professional Activities
- Journal Reviewer: Transactions on Robotics, Transactions on Pattern Analysis and Machine Intelligence
- Conference Reviewer: ICCV, CVPR, AAAI, NIPS, ECCV, ICLR, ICML
Employment
- 2022.03-2024.02: Senior research engineer, DJI
- 2021.3~2021.11: Research intern at Amazon, advised by Dr. Anton Hengle
- 2020.5~2020.11: Research intern at Adobe Research, advised by Dr. Jianming Zhang and Dr. Oliver Wang
- 2016.12~2018.6: Partner at Humanplus Intelligent Robotics Co., Ltd
Prominent Awards
- Achieve the Champion in CVPR2023 Depth Estimation Challenge (2023)
- LeReS is shortlisted in CVPR 2021 Best Paper Candidates (2021)
- Data61 Top-up Scholarship (2018~2021)
- Adobe Gift Fund for Research (Twice, 2020)
- China National Scholarship (2016)