Sergey Zakharov

prof_pic.jpg

I am a Senior Research Scientist and Technical Lead at Toyota Research Institute. My research goal is to build intelligent systems with a reliable 3D understanding of the physical world — so they can reconstruct it, generate it, and act in it.

My research spans 3D reconstruction, generative modeling, neural representations, world models, and sim-to-real, with a strong focus on generalization and real-world transfer across robotics and autonomous driving.

I earned my PhD from the Technical University of Munich under Slobodan Ilic.

news

Jun 2026 RecGen and AnyView are accepted to ECCV’26!
Jun 2026 Humanoid Everyday receives Best Paper at EAI Workshop, CVPR’26! Also Oral Presentation at “From Data to Decisions” Workshop, ICRA’26.
Jun 2026 Gave a talk on “Synthetic Worlds for Scalable Robot Learning” at the Synthetic Data for Robot Learning workshop at ICRA’26. Also co-organizing the Beyond Teleoperation workshop (June 5)!
Apr 2026 SceneSmith accepted to ICML’26 as a Spotlight (top 2.2%)!

selected publications

  1. RecGen: 3D Multi-Object Scene Reconstruction from Sparse Observations
    Andrii Zadaianchuk*, Leonardo Barcellona*, Lennard Schuenemann, Christian Gumbsch, Zehao Wang, Muhammad Zubair Irshad, Fabien Despinoy, Rahaf Aljundi, Stratis Gavves, and Sergey Zakharov*
    ECCV 20263D ReconstructionGenerative ModelingPose Estimation
    Paper Website Code 191 Cite
  2. SceneSmith: Agentic Generation of Simulation-Ready Indoor Scenes
    Nicholas Pfaff, Thomas Cohn, Sergey Zakharov, Rick Cory, and Russ Tedrake
    ICML 2026SpotlightWorld ModelsGenerative ModelingRobotics
    Paper Website Code 465 Cite 12
  3. 0123.png
    Zero-1-to-3: Zero-shot One Image to 3D Object
    Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tokmakov, Sergey Zakharov, and Carl Vondrick
    ICCV 2023Generative Modeling3D ReconstructionNovel View Synthesis
    Paper Website Code 3053 Cite 1785
  4. Autolabeling 3D Objects with Differentiable Rendering of SDF Shape Priors
    Sergey Zakharov*, Wadim Kehl*, Arjun Bhargava, and Adrien Gaidon
    CVPR 2020Oral3D ReconstructionSelf-SupervisedDriving
    Paper Code 163 Cite 143 Blog
  5. DPOD: 6D Pose Object Detector and Refiner
    Sergey Zakharov, Ivan Shugurov, and Slobodan Ilic
    ICCV 2019Pose Estimation3D Reconstruction
    Paper Code 58 Cite 708
  6. hb.png
    HomebrewedDB: RGB-D Dataset for 6D Pose Estimation of 3D Objects
    Roman Kaskman, Sergey Zakharov, Ivan Shugurov, and Slobodan Ilic
    ICCVW 2019Pose EstimationDataset
    Paper Website Code 29 Cite 213