Get 3D shape of object from one image using similar objects from CLIP with ShapeClipper
Get 3D shape of object from one image using similar objects from CLIP with ShapeClipper
ShapeClipper: Scalable 3D Shape Learning from Single-View Images via Geometric and CLIP-based Consistency
arXiv paper abstract https://arxiv.org/abs/2304.06247
arXiv PDF paper https://arxiv.org/pdf/2304.06247.pdf
... present ShapeClipper, a novel method that reconstructs 3D object shapes from real-world single-view RGB images.
Instead of relying on laborious 3D, multi-view or camera pose annotation, ShapeClipper learns shape reconstruction from a set of single-view segmented images.
... facilitate shape learning via CLIP-based shape consistency, where ... encourage objects with similar CLIP encodings to share similar shapes.
... also leverage off-the-shelf normals as an additional geometric constraint so the model can learn better bottom-up reasoning of detailed surface geometry.
These two ... constraints ... improve its ability to learn both global shape structure and local geometric details.
... achieve superior performance over state-of-the-art methods.
Please like and share this post if you enjoyed it using the buttons at the bottom!
Stay up to date. Subscribe to my posts https://morrislee1234.wixsite.com/website/contact
Web site with my other posts by category https://morrislee1234.wixsite.com/website
Comments