publications

list of publications ordered by year

2026

  1. BIND: Binding 3D Robot Actions to 2D Image Features
    Cameron Smith, Arsh Tangri, Vitor Guizilini, Yue Wang, Muhammad Zubair Irshad, and Sergey Zakharov
    arXiv 2026Robotics
    PaperPDF WebsiteWEB CodeCODE 7 CiteCITE
  2. MilliVid: Hierarchical Latents for Long-Range Consistency in Video Generation
    Ishaan Preetam Chandratreya*, David Charatan*, Basile Van Hoorick, Sergey Zakharov, Vitor Guizilini, Phillip Isola, and Vincent Sitzmann
    NeurIPS 2026Generative ModelingWorld ModelsVideo Generation
    PaperPDF WebsiteWEB CodeCODE 115 CiteCITE 2
  3. H2RBench: A Real-to-Sim Benchmark for Evaluating Human-to-Robot Transfer
    Chuyang Xiao*, Haotian Zhan*, Sriram Krishna, Peilin Meng, Muhammad Zubair Irshad, Sergey Zakharov, and David Held
    CoRL 2026RoboticsSim2RealSimulation
    PaperPDF WebsiteWEB CodeCODE 3 CiteCITE
  4. UniFlow: A Unified Multi-Modal Policy for Robot Manipulation
    Ge Yan, Shun Iwase, Yuzhi Fan, Dieter Fox, Sergey Zakharov, and Katherine Liu
    CoRL 2026RoboticsSim2Real
    CiteCITE
  5. RecGen: 3D Multi-Object Scene Reconstruction from Sparse Observations
    Andrii Zadaianchuk*, Leonardo Barcellona*, Lennard Schuenemann, Christian Gumbsch, Zehao Wang, Muhammad Zubair Irshad, Fabien Despinoy, Rahaf Aljundi, Stratis Gavves, and Sergey Zakharov*
    ECCV 20263D ReconstructionGenerative ModelingPose Estimation
    PaperPDF WebsiteWEB CodeCODE 219 CiteCITE 9
  6. AnyView: Synthesizing Any Novel View in Dynamic Scenes
    Basile Van Hoorick, Dian Chen, Shun Iwase, Pavel Tokmakov, Muhammad Zubair Irshad, Igor Vasiljevic, Swati Gupta, Fangzhou Cheng, Sergey Zakharov, and Vitor Campagnolo Guizilini
    ECCV 2026Generative ModelingNeural RepresentationsVideo Generation
    PaperPDF WebsiteWEB CiteCITE 9
  7. SceneSmith: Agentic Generation of Simulation-Ready Indoor Scenes
    Nicholas Pfaff, Thomas Cohn, Sergey Zakharov, Rick Cory, and Russ Tedrake
    ICML 2026SpotlightWorld ModelsGenerative ModelingRobotics
    PaperPDF WebsiteWEB CodeCODE 615 CiteCITE 36 BlogBLOG
  8. From Demonstrations to Rewards: Test-Time Prompt Optimization for VLM Reward Models
    Christian Gumbsch, Leonardo Barcellona, Lennard Schünemann, Platon Karageorgis, Andrii Zadaianchuk, Zehao Wang, Sergey Zakharov, Fabien Despinoy, Rahaf Aljundi, and Efstratios Gavves
    RLC 2026Robotics
    PaperPDF WebsiteWEB CodeCODE 0 CiteCITE 1
  9. PolaRiS: Scalable Real-to-Sim Evaluations for Generalist Robot Policies
    Arhan Jain*, Mingtong Zhang*, Kanav Arora, William Chen, Marcel Torne, Muhammad Zubair Irshad, Sergey Zakharov, 2 more authors, Chelsea Finn, Wei-Chiu Ma, Dhruv Shah, Abhishek Gupta, and Karl Pertsch
    RSS 20263D ReconstructionRoboticsWorld Models
    PaperPDF WebsiteWEB CodeCODE 238 CiteCITE 26
  10. Humanoid Everyday: A Comprehensive Robotic Dataset for Open-World Humanoid Manipulation
    Zhenyu Zhao*, Hongyi Jing*, Xiawei Liu, Jiageng Mao, Abha Jha, Hanwen Yang, Rong Xue, Sergey Zakharov, Vitor Guizilini, and Yue Wang
    ICRA 2026Workshop Best PaperRoboticsDataset
    PaperPDF WebsiteWEB CodeCODE 170 CiteCITE 35
  11. A Careful Examination of Large Behavior Models for Multitask Dexterous Manipulation
    Jose Barreiros, Andrew Beaulieu, Aditya Bhat, Rick Cory, Eric Cousineau, 67 more authors, Sergey Zakharov, Mark Zolotas, Rares Ambrus, Kerri Fetzer-Borelli, Benjamin Burchfiel, Hadas Kress-Gazit, Siyuan Feng, Stacie Ford, and Russ Tedrake
    Science Robotics 2026RoboticsGenerative ModelingEvaluation
    PaperPDF WebsiteWEB CiteCITE 138
  12. RAIDEN: A Toolkit for Policy Learning with YAM Bimanual Robot Arms
    Shun Iwase, Patrick Miller, Jonathan Yao, Krishna Murthy Jatavallabhula, and Sergey Zakharov
    Robotics
    WebsiteWEB CodeCODE 89 CiteCITE

2025

  1. rola.png
    Robot Learning from Any Images
    Siheng Zhao*, Jiageng Mao*, Wei Chow, Zeyu Shangguan, Tianheng Shi, 4 more authors, Daniel Seita, Leonidas Guibas, Sergey Zakharov, Vitor Campagnolo Guizilini, and Yue Wang
    CoRL 2025RoboticsSim2RealWorld Models
    PaperPDF WebsiteWEB CodeCODE 241 CiteCITE 15
  2. Steerable Scene Generation with Post Training and Inference-Time Search
    Nicholas Pfaff, Hongkai Dai, Sergey Zakharov, Shun Iwase, and Russ Tedrake
    CoRL 2025Workshop Best PosterGenerative ModelingWorld ModelsRobotics
    PaperPDF WebsiteWEB CodeCODE 88 CiteCITE 11 BlogBLOG
  3. ZeroGrasp: Zero-Shot Shape Reconstruction Enabled Robotic Grasping
    Shun Iwase, Muhammad Zubair Irshad, Katherine Liu, Vitor Guizilini, Robert Lee, 1 more author, Amma Ayako, Koichi Nishiwaki, Kris Kitani, Rares Ambrus, and Sergey Zakharov
    CVPR 20253D ReconstructionRoboticsPose Estimation
    PaperPDF WebsiteWEB CodeCODE 98 CiteCITE 38
  4. OmniShape: Zero-Shot Multi-Hypothesis Object Pose and Shape Estimation in the Real World
    Katherine Liu, Sergey Zakharov, Dian Chen, Takuya Ikeda, Greg Shakhnarovich, Adrien Gaidon, and Rares Ambrus
    ICRA 20253D ReconstructionPose EstimationGenerative Modeling
    PaperPDF WebsiteWEB CiteCITE 2

2024

  1. equi.png
    SE3 Equivariant Ray Embeddings for Implicit Multi-View Depth Estimation
    Yinshuang Xu, Dian Chen, Katherine Liu, Sergey Zakharov, Rares Ambrus, Kostas Daniilidis, and Vitor Guizilini
    NeurIPS 20243D ReconstructionNeural RepresentationsDepth Estimation
    PaperPDF CiteCITE 12
  2. vista.png
    View-Invariant Policy Learning via Zero-Shot Novel View Synthesis
    Stephen Tian, Blake Wulfe, Kyle Sargent, Katherine Liu, Sergey Zakharov, Vitor Guizilini, and Jiajun Wu
    CoRL 2024RoboticsNovel View Synthesis
    PaperPDF WebsiteWEB CodeCODE 32 CiteCITE 61
  3. octmae.jpg
    Zero-Shot Multi-Object Shape Completion
    Shun Iwase, Katherine Liu, Vitor Guizilini, Adrien Gaidon, Kris Kitani, Rares Ambrus, and Sergey Zakharov
    ECCV 20243D Reconstruction
    PaperPDF WebsiteWEB CodeCODE 33 CiteCITE 16
  4. nerfmae.jpg
    NeRF-MAE: Masked AutoEncoders for Self-Supervised 3D Representation Learning for Neural Radiance Fields
    Muhammad Zubair Irshad, Sergey Zakharov, Vitor Guizilini, Adrien Gaidon, Zsolt Kira, and Rares Ambrus
    ECCV 2024Self-SupervisedNeural Representations3D Reconstruction
    PaperPDF WebsiteWEB CodeCODE 107 CiteCITE 45
  5. diffusionnocs.jpg
    DiffusionNOCS: Managing Symmetry and Uncertainty in Sim2Real Multi-Modal Category-level Pose Estimation
    Takuya Ikeda*, Sergey Zakharov*, Tianyi Ko, Muhammad Zubair Irshad, Robert Lee, Katherine Liu, Rares Ambrus, and Koichi Nishiwaki
    IROS 2024Pose EstimationSim2RealGenerative Modeling
    PaperPDF WebsiteWEB CodeCODE 41 CiteCITE 41
  6. ReFiNe: Recursive Field Networks for Cross-Modal Multi-Scene Representation
    Sergey Zakharov, Katherine Liu, Adrien Gaidon, and Rares Ambrus
    SIGGRAPH 2024Neural Representations3D Reconstruction
    PaperPDF WebsiteWEB CodeCODE 7 CiteCITE 3
  7. fsd.png
    FSD: Fast Self-Supervised Single RGB-D to Categorical 3D Objects
    Mayank Lunayach, Sergey Zakharov, Dian Chen, Rares Ambrus, Zsolt Kira, and Muhammad Zubair Irshad
    ICRA 2024Self-SupervisedPose Estimation3D Reconstruction
    PaperPDF WebsiteWEB CiteCITE 25

2023

  1. 0123.png
    Zero-1-to-3: Zero-shot One Image to 3D Object
    Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tokmakov, Sergey Zakharov, and Carl Vondrick
    ICCV 2023Generative Modeling3D ReconstructionNovel View Synthesis
    PaperPDF WebsiteWEB CodeCODE 3067 CiteCITE 1996
  2. delira.png
    DeLiRa: Self-Supervised Depth, Light, and Radiance Fields
    Vitor Guizilini, Igor Vasiljevic, Jiading Fang, Rares Ambrus, Sergey Zakharov, Vincent Sitzmann, and Adrien Gaidon
    ICCV 2023Self-SupervisedNeural Representations3D Reconstruction
    PaperPDF WebsiteWEB CodeCODE 637 CiteCITE 6
  3. NeO 360: Neural Fields for Sparse View Synthesis of Outdoor Scenes
    Muhammad Zubair Irshad, Sergey Zakharov, Katherine Liu, Vitor Guizilini, Thomas Kollar, Adrien Gaidon, Zsolt Kira, and Rares Ambrus
    ICCV 2023Neural Representations3D ReconstructionDriving
    PaperPDF WebsiteWEB CodeCODE 246 CiteCITE 72
  4. carto.png
    CARTO: Category and Joint Agnostic Reconstruction of ARTiculated Objects
    Nick Heppert, Muhammad Zubair Irshad, Sergey Zakharov, Katherine Liu, Rares Ambrus, Jeannette Bohg, Abhinav Valada, and Thomas Kollar
    CVPR 20233D ReconstructionNeural RepresentationsPose Estimation
    PaperPDF WebsiteWEB CodeCODE 25 CiteCITE 89
  5. Multi-Object Manipulation via Object-Centric Neural Scattering Functions
    Stephen Tian, Yancheng Cai, Hong-Xing Yu, Sergey Zakharov, Katherine Liu, Adrien Gaidon, Yunzhu Li, and Jiajun Wu
    CVPR 2023RoboticsNeural RepresentationsWorld Models
    PaperPDF WebsiteWEB CiteCITE 19
  6. Neural Groundplans: Persistent Neural Scene Representations from a Single Image
    Prafull Sharma, Ayush Tewari, Yilun Du, Sergey Zakharov, Rares Ambrus, Adrien Gaidon, William Freeman, Fredo Durand, Joshua Tenenbaum, and Vincent Sitzmann
    ICLR 2023Neural Representations3D ReconstructionGenerative Modeling
    PaperPDF WebsiteWEB CiteCITE 35

2022

  1. ROAD: Learning an Implicit Recursive Octree Auto-Decoder to Efficiently Encode 3D Shapes
    Sergey Zakharov, Rares Ambrus, Katherine Liu, and Adrien Gaidon
    CoRL 2022Neural Representations3D Reconstruction
    PaperPDF WebsiteWEB CodeCODE 15 CiteCITE 8
  2. Photo-realistic Neural Domain Randomization
    Sergey Zakharov, Rares Ambrus, Vitor Guizilini, Wadim Kehl, and Adrien Gaidon
    ECCV 2022Sim2RealNeural Representations
    PaperPDF CiteCITE 21
  3. spot.png
    SpOT: Spatiotemporal Modeling for 3D Object Tracking
    Colton Stearns, Davis Rempe, Jie Li, Rares Ambrus, Sergey Zakharov, Vitor Guizilini, Yanchao Yang, and Leonidas J Guibas
    ECCV 2022OralDriving
    PaperPDF WebsiteWEB CodeCODE 30 CiteCITE 18
  4. ShAPO: Implicit Representations for Multi Object Shape Appearance and Pose Optimization
    Muhammad Zubair Irshad*, Sergey Zakharov*, Rares Ambrus, Thomas Kollar, Zsolt Kira, and Adrien Gaidon
    ECCV 20223D ReconstructionNeural RepresentationsPose Estimation
    PaperPDF WebsiteWEB CodeCODE 197 CiteCITE 110
  5. Multi-Frame Self-Supervised Depth with Transformers
    Vitor Guizilini, Rares Ambrus, Dian Chen, Sergey Zakharov, and Adrien Gaidon
    CVPR 2022Self-Supervised3D ReconstructionDriving
    PaperPDF WebsiteWEB CodeCODE 637 CiteCITE 155

2021

  1. dpodv2.png
    DPODv2: Dense Correspondence-Based 6 DoF Pose Estimation
    Ivan Shugurov, Sergey Zakharov, and Slobodan Ilic
    TPAMI 2021Pose Estimation3D Reconstruction
    PaperPDF CiteCITE 85
  2. sssr.png
    Single-Shot Scene Reconstruction
    Sergey Zakharov, Rares Ambrus, Dennis Park, Vitor Guizilini, Wadim Kehl, Fredo Durand, Joshua Tenenbaum, Vincent Sitzmann, Jiajun Wu, and Adrien Gaidon
    CoRL 20213D ReconstructionNeural RepresentationsDriving
    PaperPDF CiteCITE 19
  3. refinement_dr.png
    Multi-View Object Pose Refinement With Differentiable Renderer
    Ivan Shugurov, Ivan Pavlov, Sergey Zakharov, and Slobodan Ilic
    RAL 2021Pose Estimation3D Reconstruction
    PaperPDF CiteCITE 39

2020

  1. Autolabeling 3D Objects with Differentiable Rendering of SDF Shape Priors
    Sergey Zakharov*, Wadim Kehl*, Arjun Bhargava, and Adrien Gaidon
    CVPR 2020Oral3D ReconstructionSelf-SupervisedDriving
    PaperPDF CodeCODE 163 CiteCITE 149 BlogBLOG
  2. tless6d.png
    6 dof pose estimation of textureless objects from multiple RGB frames
    Roman Kaskman, Ivan Shugurov, Sergey Zakharov, and Slobodan Ilic
    ECCVW 2020Pose Estimation
    PaperPDF CiteCITE 17

2019

  1. DPOD: 6D Pose Object Detector and Refiner
    Sergey Zakharov*, Ivan Shugurov*, and Slobodan Ilic
    ICCV 2019Workshop Award WinnerPose Estimation3D Reconstruction
    PaperPDF CodeCODE 58 CiteCITE 728
  2. deceptionnet.png
    DeceptionNet: Network-Driven Domain Randomization
    Sergey Zakharov, Wadim Kehl, and Slobodan Ilic
    ICCV 2019Sim2Real
    PaperPDF CiteCITE 118
  3. synda.png
    Seeing beyond appearance-mapping real images into geometrical domains for unsupervised cad-based recognition
    Benjamin Planche*, Sergey Zakharov*, Ziyan Wu, Andreas Hutter, Harald Kosch, and Slobodan Ilic
    IROS 2019Sim2RealSelf-Supervised
    PaperPDF CiteCITE 16
  4. hb.png
    HomebrewedDB: RGB-D Dataset for 6D Pose Estimation of 3D Objects
    Roman Kaskman, Sergey Zakharov, Ivan Shugurov, and Slobodan Ilic
    ICCVW 2019Pose EstimationDataset
    PaperPDF WebsiteWEB CodeCODE 29 CiteCITE 243

2018

  1. unrealda.png
    Keep it unreal: Bridging the realism gap for 2.5 d recognition with geometry priors only
    Sergey Zakharov*, Benjamin Planche*, Ziyan Wu, Andreas Hutter, Harald Kosch, and Slobodan Ilic
    3DV 2018OralSim2Real
    PaperPDF CiteCITE 33
  2. regression.png
    When regression meets manifold learning for object recognition and pose estimation
    Mai Bui, Sergey Zakharov, Shadi Albarqouni, Slobodan Ilic, and Nassir Navab
    ICRA 2018Pose Estimation
    PaperPDF CiteCITE 36

2017

  1. 3D Object Instance Recognition and Pose Estimation Using Triplet Loss with Dynamic Margin
    Sergey Zakharov, Wadim Kehl, Benjamin Planche, Andreas Hutter, and Slobodan Ilic
    IROS 2017Pose Estimation
    PaperPDF CiteCITE 53
  2. depthsynth.png
    Depthsynth: Real-time realistic synthetic data generation from cad models for 2.5 d recognition
    Benjamin Planche, Ziyan Wu, Kai Ma, Shanhui Sun, Stefan Kluckner, 1 more author, Terrence Chen, Andreas Hutter, Sergey Zakharov, Harald Kosch, and Jan Ernst
    3DV 2017OralSim2Real
    PaperPDF CiteCITE 114