Xiuming Zhang

About

Xiuming Zhang 張修明

I'm currently a Principal Research Scientist in Sanja Fidler's Spatial Intelligence Lab at NVIDIA, working on closed-loop training of physical AI agents via generative world models. I'm based in the headquarter in Santa Clara, CA. Before NVIDIA, I worked on autonomous driving at Tesla Autopilot and on computational photography in Marc Levoy's Emerging Products Group at Adobe. I did my Ph.D. in computer vision and computer graphics with Bill Freeman at MIT CSAIL. While at MIT, I also received mentorship from Jon Barron and Yun-Ta Tsai via internships at Google Research. My dissertation Shape, Reflectance, and Illumination From Appearance studies inverse rendering and its applications, including 3D shape reconstruction and free-viewpoint relighting. Here is a more formal bio.

Google Scholar xiuming6zhang at gmail dot com

Recent Experiences

Senior Machine Learning Scientist, Tesla Autopilot, 2022–2025

Computer Scientist, Adobe Emerging Products Group, 2021–2022, w/ Marc Levoy

Student Researcher, Google Research, 2019–2021, w/ Jon Barron & Yun-Ta Tsai

Ph.D., MIT CSAIL , 2016–2021, w/ Bill Freeman

News

Jul. 2025	Joined Sanja Fidler's Spatial Intelligence Lab at NVIDIA.
Jun. 2025	Robotaxi launched in Austin, TX!
Sep. 2024	Helped ship Actually Smart Summon to FSD customers.
Mar. 2024	Shipped Autopark to FSD customers!
Dec. 2023	Shipped High-Fidelity Park Assist to all Tesla cars!
Apr. 2023	DiffusionRig (personalized face editing) released for CVPR 2023.
Oct. 2022	Joined Tesla Autopilot to work on 3D perception.
Oct. 2021	Joined Marc Levoy's Adobe Emerging Products Group to work on computational photography.
Aug. 2021	Graduated from MIT (doctoral dissertation).
Aug. 2021	NeRFactor conditionally accepted to SIGGRAPH Asia.
Dec. 2020	Neural Light Transport (NLT) accepted to TOG, to be presented at SIGGRAPH in August 2021.
Apr. 2019	MoSculp on exhibition in the MIT Museum (photos)!
Jan. 2019	Awarded Snap Research Fellowship.
Dec. 2018	Oral presentation of GenRe at NeurIPS.
Sep. 2018	MoSculp featured as 9/19 MIT Homepage and covered by Forbes, BBC, etc.
Oct. 2016	Paper on Bayesian modeling of Alzheimer's disease heterogeneity out in PNAS and covered by Psychology Today, MGH/HMS, etc.
May 2016	Work on Alzheimer's disease subtypes awarded the Magna Cum Laude Award and an oral presentation at ISMRM 2016.

Products

[credit]

[official] [reactions]
Full Self-Driving (FSD)
Worked on 3D perception and E2E safety for FSD releases to both customer and Robotaxi fleets, contributing to both online models shipped to cars and offline data pipelines scaling to 20K GPUs.
Tesla 2022–2025

Tesla Actually Smart Summon pickup visualization

[credit]

[official] [reactions]
Actually Smart Summon
Helped vehicles perceive static and dynamic 3D scenes, allowing them to autonomously pick up their owners (e.g., from a store exit in the rain) without collisions.
Tesla 2024

Tesla Autopark parking selection visualization

[credit]

[official] [reactions]
Autopark
Developed the offline part of parkable space prediction, enabling autonomous parking into parallel, perpendicular, and angled spots—marked or unmarked—in unseen environments.
Tesla 2023

Tesla High-Fidelity Park Assist visualization

[credit]

[official] [reactions] [patent]
High-Fidelity Park Assist
Tackled various aspects of this next-generation, real-time 3D reconstruction system, covering deformable human body, car geometry from partial views, and large-scale static scenes.
Tesla 2023

Recent Publications

* indicates equal contribution.

DriveJudge autonomous driving evaluation teaser

[paper] [bibtex]
DriveJudge: Rethinking Autonomous Driving Evaluation with Vision-Language Models
Xinglong Sun, Kevin Xie, Jenny Schmalfuss, Despoina Paschalidou, Xiuming Zhang, Sanja Fidler, Kashyap Chitta, Jose M. Alvarez
arXiv 2026

DiffusionRig facial expression generation teaser

[paper] [video] [project] [code] [bibtex]
DiffusionRig: Learning Personalized Priors for Facial Appearance Editing
Zheng Ding, Xuaner Zhang, Zhihao Xia, Lars Jebe, Zhuowen Tu, Xiuming Zhang
CVPR 2023

Portrait reconstruction and relighting teaser

[paper] [video] [project] [bibtex]
Portrait Reconstruction and Relighting Using the Sun as a Light Stage
Yifan Wang, Aleksander Holynski, Xiuming Zhang, Xuaner Zhang
CVPR 2023

[thesis] [video] [bibtex]
Shape, Reflectance, and Illumination From Appearance
Committee: William T. Freeman, Jonathan T. Barron, Antonio Torralba
MIT Doctoral Dissertation 2021

[paper] [video] [talk] [code] [project] [bibtex]
NeRFactor: Neural Factorization of Shape and Reflectance Under an Unknown Illumination
Xiuming Zhang, Pratul P. Srinivasan, Boyang Deng, Paul Debevec, William T. Freeman, Jonathan T. Barron
TOG 2021 (Proc. SIGGRAPH Asia)

[paper] [video] [code] [demo] [project] [bibtex]
Editing Conditional Radiance Fields
Steven Liu, Xiuming Zhang, Zhoutong Zhang, Richard Zhang, Jun-Yan Zhu, Bryan Russell
ICCV 2021

NeRV relighting and view synthesis animation

[paper] [video] [project] [bibtex]
NeRV: Neural Reflectance and Visibility Fields for Relighting and View Synthesis
Pratul P. Srinivasan, Boyang Deng, Xiuming Zhang, Matthew Tancik, Ben Mildenhall, Jonathan T. Barron
CVPR 2021

Neural Light Transport relighting animation

[paper] [video] [talk] [code] [project] [patent] [bibtex]
Neural Light Transport for Relighting and View Synthesis
Xiuming Zhang, Sean Fanello, Yun-Ta Tsai, Tiancheng Sun, Tianfan Xue, Rohit Pandey, Sergio Orts-Escolano, Philip Davidson, Christoph Rhemann, Paul Debevec, Jonathan T. Barron, Ravi Ramamoorthi, William T. Freeman
TOG 2021 (Presented at SIGGRAPH)

[paper] [video] [project] [bibtex]
Multi-Plane Program Induction With 3D Box Priors
Yikai Li *, Jiayuan Mao *, Xiuming Zhang, William T. Freeman, Joshua B. Tenenbaum, Noah Snavely, Jiajun Wu
NeurIPS 2020

Light Stage Super-Resolution relighting animation

[paper] [video] [talk] [project] [bibtex]
Light Stage Super-Resolution: Continuous High-Frequency Relighting
Tiancheng Sun, Zexiang Xu, Xiuming Zhang, Sean Fanello, Christoph Rhemann, Paul Debevec, Yun-Ta Tsai, Jonathan T. Barron, Ravi Ramamoorthi
TOG 2020 (Proc. SIGGRAPH Asia)

[paper] [video] [code] [supp] [project] [bibtex]
Portrait Shadow Manipulation
Xuaner Zhang, Jonathan T. Barron, Yun-Ta Tsai, Rohit Pandey, Xiuming Zhang, Ren Ng, David E. Jacobs
TOG 2020 (Proc. SIGGRAPH)

Perspective Plane Program Induction teaser

[paper] [code] [supp] [project] [bibtex]
Perspective Plane Program Induction From a Single Image
Yikai Li *, Jiayuan Mao *, Xiuming Zhang, William T. Freeman, Joshua B. Tenenbaum, Jiajun Wu
CVPR 2020

Posterior cortical atrophy factor visualization

[paper] [bibtex]
Latent Atrophy Factors Related to Phenotypical Variants of Posterior Cortical Atrophy
Colin Groot, B. T. Thomas Yeo, Jacob W. Vogel, Xiuming Zhang, Nanbo Sun, Elizabeth C. Mormino, Yolande A. L. Pijnenburg, Bruce L. Miller, Howard J. Rosen, Renaud La Joie, Frederik Barkhof, Philip Scheltens, Wiesje M. van der Flier, Gil D. Rabinovici, Rik Ossenkoppele
Neurology 2020

Program-Guided Image Manipulators teaser

[paper] [supp] [project] [bibtex]
Program-Guided Image Manipulators
Jiayuan Mao *, Xiuming Zhang *, Yikai Li, William T. Freeman, Joshua B. Tenenbaum, Jiajun Wu
ICCV 2019

Autism heterogeneity connectomics visualization

[paper] [bibtex]
Reconciling Dimensional and Categorical Models of Autism Heterogeneity: A Brain Connectomics and Behavioral Study
Siyi Tang *, Nanbo Sun *, Dorothea L. Floris, Xiuming Zhang, Adriana Di Martino, B. T. Thomas Yeo
Biological Psychiatry 2019

[paper] [talk] [code] [project] [supp] [project] [bibtex]
Learning to Reconstruct Shapes From Unseen Classes
Xiuming Zhang *, Zhoutong Zhang *, Chengkai Zhang, Joshua B. Tenenbaum, William T. Freeman, Jiajun Wu
NeurIPS 2018
Oral Presentation (Oral/Accepted/Submitted: 30/1011/4856)

[paper] [thesis] [video] [supp] [talk] [demo] [project] [bibtex]
MoSculp: Interactive Visualization of Shape and Time
Xiuming Zhang, Tali Dekel, Tianfan Xue, Andrew Owens, Qiurui He, Jiajun Wu, Stefanie Mueller, William T. Freeman
UIST 2018
Press Coverage: Forbes, BBC, MIT, 9/19 MIT Homepage, etc.
Outreach: MIT Museum

[paper] [code] [bibtex]
Learning Shape Priors for Single-View 3D Completion and Reconstruction
Jiajun Wu *, Chengkai Zhang *, Xiuming Zhang, Zhoutong Zhang, William T. Freeman, Joshua B. Tenenbaum
ECCV 2018

[paper] [code] [project] [bibtex]
Pix3D: Dataset and Methods for Single-Image 3D Shape Modeling
Xingyuan Sun *, Jiajun Wu *, Xiuming Zhang, Zhoutong Zhang, Chengkai Zhang, Tianfan Xue, Joshua B. Tenenbaum, William T. Freeman
CVPR 2018

Alzheimer's disease latent atrophy factor visualization

[paper] [supp] [code] [poster] [bibtex]
Bayesian Model Reveals Latent Atrophy Factors With Dissociable Cognitive Trajectories in Alzheimer's Disease
Xiuming Zhang, Elizabeth C. Mormino, Nanbo Sun, Reisa A. Sperling, Mert R. Sabuncu, B. T. Thomas Yeo
PNAS 2016
Magna Cum Laude Award & Oral Presentation at ISMRM 2016
Press Coverage: Psychology Today, MGH/HMS, etc.