ObjectSDF++

Improved Object-Compositional
Neural Implicit Surfaces

ICCV 2023

Qianyi Wu1     Kaisiyuan Wang2     Kejie Li3     Jianmin Zheng4     Jianfei Cai1    
1Monash University   2University of Sydney  
3University of Oxford   4Nanyang Technological University  
Paper Video Code



ObjectSDF++ produces accurate scene geometry and objects geometry in an unified framework, under the supervision of multiview images with instance mask.

Video


Abstract

TL;DR: A scene is composited by various objects. Therefore, we present ObjectSDF++, an improved neural implicit surfaces framework that reconstruct each object surface simultaneously from multiview images with instance mask guidance.

ObjectSDF++ is a new framework for neural implicit surface reconstruction that focuses on object-compositional scene, enhancing both object and scene reconstruction quality. The method introduces an occlusion-aware object opacity rendering formulation, which directly volume-renders object opacity supervised by instance masks. Additionally, ObjectSDF++ incorporates a novel regularization term for object distinction, effectively addressing the issue of unexpected reconstructions in invisible regions. These improvements make ObjectSDF++ producing high-quality 3D reconstruction results.

Results Gallery


Replica Dataset

Object reconstruction results of scene

Reference image

vMAP (RGB-D)

ObjectSDF (RGB)

ObjectSDF++ (RGB)


ScanNet Dataset

Scene reconstruction result of by structure


Related Links

For those intrigued by the utilization of neural implicit representation for object-compositional modeling and understanding, we kindly invite you to explore our curated paper collection repository. We diligently update it with relevant content, and we would genuinely appreciate any pull requests or contributions on this subject.

Check these concurrent works we found, which provide thought-provoking ideas towards this direction:
  • vMAP. An object-level RGB-D SLAM system with vectorised training to achieve efficient performance.
  • RICO. Regulate the surface reconstruction on unobserved region in object-compositional scenario.
  • PCFF. To overcome the view-inconsistent mask issue in real application.
  • Panoptic Lifting. A framework to learn panoptic 3D volumetrirepresentations from 2D multiview panoptic segmentation masks.

    • BibTeX

    @inproceedings{Wu2023objectsdfplus,
        author    = {Wu, Qianyi and Wang, Kaisiyuan and Li, Kejie and Zheng, Jianmin and Cai, Jianfei},
        title     = {ObjectSDF++: Improved Object-Compositional Neural Implicit Surfaces},
        booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision},
        year      = {2023}
    }