Get 3D scene and segmentation from one image without 3D truth using segmentation model with S4C
Get 3D scene and segmentation from one image without 3D truth using segmentation model with S4C
S4C: Self-Supervised Semantic Scene Completion with Neural Fields
arXiv paper abstract https://arxiv.org/abs/2310.07522
arXiv PDF paper https://arxiv.org/pdf/2310.07522.pdf
3D semantic scene understanding is a fundamental ... in computer vision ... SSC ... jointly estimating dense geometry and semantic information from sparse observations of a scene.
Current methods ... trained on 3D ground truth based on ... LiDAR ... relies on special sensors and annotation by hand
... work presents the first self-supervised approach to SSC called S4C that does not rely on 3D ground truth data.
... proposed method can reconstruct a scene from a single image and only relies on videos and pseudo segmentation ground truth generated from off-the-shelf image segmentation network during training.
Unlike existing methods, which use discrete voxel grids, ... represent scenes as implicit semantic fields ... allows querying any point within the camera frustum for occupancy and semantic class.
... achieves performance close to fully supervised state-of-the-art methods ... demonstrates strong generalization capabilities and can synthesize accurate segmentation maps for far away viewpoints.
Please like and share this post if you enjoyed it using the buttons at the bottom!
Stay up to date. Subscribe to my posts https://morrislee1234.wixsite.com/website/contact
Web site with my other posts by category https://morrislee1234.wixsite.com/website
Comments