Xiuming Zhang

I'm currently a Principal Research Scientist and Senior Manager in the Spatial Intelligence Lab at NVIDIA, working on world (action) models. I'm based at NVIDIA's headquarters in Santa Clara. Before NVIDIA, I worked on autonomous driving at Tesla Autopilot and on computational photography at Adobe NextCam. I completed my Ph.D. at MIT CSAIL, advised by Bill Freeman, with research in computer vision and computer graphics. While at MIT, I was also mentored by Jon Barron and Yun-Ta Tsai during internships at Google Research. My dissertation Shape, Reflectance, and Illumination From Appearance explores inverse rendering and its applications, including 3D shape reconstruction and free-viewpoint relighting. I completed my undergraduate studies at the National University of Singapore (NUS), where I conducted neuroscience research with Thomas Yeo.

• X
• Email
• Google Scholar
• LinkedIn
• Formal Bio

Recent Experience

* Equal contribution; † Corresponding authors.

[paper] [project] [bibtex]
BirdDreams: Joint Perspective and BEV Video Generation for a Plausible, Reactive Driving World Model
Junyoung Seo, Jaewoo Seo, Seungryong Kim, Laura Leal-Taixé, Xiuming Zhang^†, Seung Wook Kim^†
arXiv 2026

[paper] [bibtex]
DriveJudge: Rethinking Autonomous Driving Evaluation with Vision-Language Models
Xinglong Sun, Kevin Xie, Jenny Schmalfuss, Despoina Paschalidou, Xiuming Zhang, Sanja Fidler, Kashyap Chitta, Jose M. Alvarez
arXiv 2026

[paper] [video] [project] [code] [bibtex]
DiffusionRig: Learning Personalized Priors for Facial Appearance Editing
Zheng Ding, Xuaner Zhang, Zhihao Xia, Lars Jebe, Zhuowen Tu, Xiuming Zhang
CVPR 2023

[paper] [video] [project] [bibtex]
Portrait Reconstruction and Relighting Using the Sun as a Light Stage
Yifan Wang, Aleksander Holynski, Xiuming Zhang, Xuaner Zhang
CVPR 2023

[thesis] [video] [bibtex]
Shape, Reflectance, and Illumination From Appearance
Committee: William T. Freeman, Jonathan T. Barron, Antonio Torralba
MIT Doctoral Dissertation 2021

[paper] [video] [talk] [code] [project] [bibtex]
NeRFactor: Neural Factorization of Shape and Reflectance Under an Unknown Illumination
Xiuming Zhang, Pratul P. Srinivasan, Boyang Deng, Paul Debevec, William T. Freeman, Jonathan T. Barron
TOG 2021 (Proc. SIGGRAPH Asia)

[paper] [video] [code] [demo] [project] [bibtex]
Editing Conditional Radiance Fields
Steven Liu, Xiuming Zhang, Zhoutong Zhang, Richard Zhang, Jun-Yan Zhu, Bryan Russell
ICCV 2021

[paper] [video] [project] [bibtex]
NeRV: Neural Reflectance and Visibility Fields for Relighting and View Synthesis
Pratul P. Srinivasan, Boyang Deng, Xiuming Zhang, Matthew Tancik, Ben Mildenhall, Jonathan T. Barron
CVPR 2021

[paper] [video] [talk] [code] [project] [patent] [bibtex]
Neural Light Transport for Relighting and View Synthesis
Xiuming Zhang, Sean Fanello, Yun-Ta Tsai, Tiancheng Sun, Tianfan Xue, Rohit Pandey, Sergio Orts-Escolano, Philip Davidson, Christoph Rhemann, Paul Debevec, Jonathan T. Barron, Ravi Ramamoorthi, William T. Freeman
TOG 2021 (Presented at SIGGRAPH)

[paper] [video] [project] [bibtex]
Multi-Plane Program Induction With 3D Box Priors
Yikai Li *, Jiayuan Mao *, Xiuming Zhang, William T. Freeman, Joshua B. Tenenbaum, Noah Snavely, Jiajun Wu
NeurIPS 2020

[paper] [video] [talk] [project] [bibtex]
Light Stage Super-Resolution: Continuous High-Frequency Relighting
Tiancheng Sun, Zexiang Xu, Xiuming Zhang, Sean Fanello, Christoph Rhemann, Paul Debevec, Yun-Ta Tsai, Jonathan T. Barron, Ravi Ramamoorthi
TOG 2020 (Proc. SIGGRAPH Asia)

[paper] [video] [code] [supp] [project] [bibtex]
Portrait Shadow Manipulation
Xuaner Zhang, Jonathan T. Barron, Yun-Ta Tsai, Rohit Pandey, Xiuming Zhang, Ren Ng, David E. Jacobs
TOG 2020 (Proc. SIGGRAPH)

[paper] [code] [supp] [project] [bibtex]
Perspective Plane Program Induction From a Single Image
Yikai Li *, Jiayuan Mao *, Xiuming Zhang, William T. Freeman, Joshua B. Tenenbaum, Jiajun Wu
CVPR 2020

[paper] [bibtex]
Latent Atrophy Factors Related to Phenotypical Variants of Posterior Cortical Atrophy
Colin Groot, B. T. Thomas Yeo, Jacob W. Vogel, Xiuming Zhang, Nanbo Sun, Elizabeth C. Mormino, Yolande A. L. Pijnenburg, Bruce L. Miller, Howard J. Rosen, Renaud La Joie, Frederik Barkhof, Philip Scheltens, Wiesje M. van der Flier, Gil D. Rabinovici, Rik Ossenkoppele
Neurology 2020

[paper] [supp] [project] [bibtex]
Program-Guided Image Manipulators
Jiayuan Mao *, Xiuming Zhang *, Yikai Li, William T. Freeman, Joshua B. Tenenbaum, Jiajun Wu
ICCV 2019

[paper] [bibtex]
Reconciling Dimensional and Categorical Models of Autism Heterogeneity: A Brain Connectomics and Behavioral Study
Siyi Tang *, Nanbo Sun *, Dorothea L. Floris, Xiuming Zhang, Adriana Di Martino, B. T. Thomas Yeo
Biological Psychiatry 2019

[paper] [talk] [code] [project] [supp] [project] [bibtex]
Learning to Reconstruct Shapes From Unseen Classes
Xiuming Zhang *, Zhoutong Zhang *, Chengkai Zhang, Joshua B. Tenenbaum, William T. Freeman, Jiajun Wu
NeurIPS 2018
Oral Presentation (Oral/Accepted/Submitted: 30/1011/4856)

[paper] [thesis] [video] [supp] [talk] [demo] [project] [bibtex]
MoSculp: Interactive Visualization of Shape and Time
Xiuming Zhang, Tali Dekel, Tianfan Xue, Andrew Owens, Qiurui He, Jiajun Wu, Stefanie Mueller, William T. Freeman
UIST 2018
Press Coverage: Forbes, BBC, MIT, MIT homepage (9/19), etc.
Outreach: MIT Museum

[paper] [code] [bibtex]
Learning Shape Priors for Single-View 3D Completion and Reconstruction
Jiajun Wu *, Chengkai Zhang *, Xiuming Zhang, Zhoutong Zhang, William T. Freeman, Joshua B. Tenenbaum
ECCV 2018

[paper] [code] [project] [bibtex]
Pix3D: Dataset and Methods for Single-Image 3D Shape Modeling
Xingyuan Sun *, Jiajun Wu *, Xiuming Zhang, Zhoutong Zhang, Chengkai Zhang, Tianfan Xue, Joshua B. Tenenbaum, William T. Freeman
CVPR 2018

[paper] [supp] [code] [poster] [bibtex]
Bayesian Model Reveals Latent Atrophy Factors With Dissociable Cognitive Trajectories in Alzheimer's Disease
Xiuming Zhang, Elizabeth C. Mormino, Nanbo Sun, Reisa A. Sperling, Mert R. Sabuncu, B. T. Thomas Yeo
PNAS 2016
Magna Cum Laude Award & Oral Presentation at ISMRM 2016
Press Coverage: Psychology Today, MGH/HMS, etc.