Get 3D shape of object from one image using similar objects from CLIP with ShapeClipper
- morrislee
- Apr 14, 2023
- 1 min read
Get 3D shape of object from one image using similar objects from CLIP with ShapeClipper
ShapeClipper: Scalable 3D Shape Learning from Single-View Images via Geometric and CLIP-based Consistency
arXiv paper abstract https://arxiv.org/abs/2304.06247
arXiv PDF paper https://arxiv.org/pdf/2304.06247.pdf
... present ShapeClipper, a novel method that reconstructs 3D object shapes from real-world single-view RGB images.
Instead of relying on laborious 3D, multi-view or camera pose annotation, ShapeClipper learns shape reconstruction from a set of single-view segmented images.
... facilitate shape learning via CLIP-based shape consistency, where ... encourage objects with similar CLIP encodings to share similar shapes.
... also leverage off-the-shelf normals as an additional geometric constraint so the model can learn better bottom-up reasoning of detailed surface geometry.
These two ... constraints ... improve its ability to learn both global shape structure and local geometric details.
... achieve superior performance over state-of-the-art methods.
Please like and share this post if you enjoyed it using the buttons at the bottom!
Stay up to date. Subscribe to my posts https://morrislee1234.wixsite.com/website/contact
Web site with my other posts by category https://morrislee1234.wixsite.com/website
Comments