Get 3D object from scene by lifting Segment Anything Model masks into 3D field with NOC
Get 3D object from scene by lifting Segment Anything Model masks into 3D field with NOC
NOC: High-Quality Neural Object Cloning with 3D Lifting of Segment Anything
arXiv paper abstract https://arxiv.org/abs/2309.12790
arXiv PDF paper https://arxiv.org/pdf/2309.12790.pdf
With the development of the neural field, reconstructing the 3D model of a target object from multi-view inputs has recently attracted increasing attention from the community ... under-explored how to reconstruct a certain object indicated by users on-the-fly.
Considering the Segment Anything Model (SAM) has shown effectiveness in segmenting any 2D images, in this paper, ... propose Neural Object Cloning (NOC), a novel high-quality 3D object reconstruction method, which leverages the benefits of both neural field and SAM from two aspects.
... to separate the target ... propose a novel strategy to lift the multi-view 2D segmentation masks of SAM into a unified 3D variation field.
The 3D variation field is then projected into 2D space and generates the new prompts for SAM. This process is iterative until convergence to separate the target object from the scene.
... further lift the 2D features of the SAM encoder into a 3D SAM field in order to improve the reconstruction quality of the target object.
NOC lifts the 2D masks and features of SAM into the 3D neural field for high-quality target object reconstruction ... demonstrate the advantages of ... method ...
Please like and share this post if you enjoyed it using the buttons at the bottom!
Stay up to date. Subscribe to my posts https://morrislee1234.wixsite.com/website/contact
Web site with my other posts by category https://morrislee1234.wixsite.com/website
Comments