Get hand interactions in 3D in real-time from monocular video
Get hand interactions in 3D in real-time from monocular video
RGB2Hands: Real-Time Tracking of 3D Hand Interactions from Monocular RGB Video
arXiv paper abstract https://arxiv.org/abs/2106.11725v1
arXiv PDF paper https://arxiv.org/pdf/2106.11725v1.pdf
Tracking and reconstructing the 3D pose and geometry of two hands in interaction is a challenging problem that has a high relevance for several human-computer interaction applications, including AR/VR, robotics, or sign language recognition.
... present the first real-time method for motion capture of skeletal pose and 3D surface geometry of hands from a single RGB camera that explicitly considers close interactions.
... propose a novel multi-task CNN that regresses multiple complementary pieces of information, including segmentation, dense matchings to a 3D hand model, and 2D keypoint positions, together with newly proposed intra-hand relative depth and inter-hand distance maps.
... approach offers previously unseen two-hand tracking performance from RGB, and quantitatively and qualitatively outperforms existing RGB-based methods that were not explicitly designed for two-hand interactions.
Moreover, our method even performs on-par with depth-based real-time methods.
Please like and share this post if you enjoyed it using the buttons at the bottom!
Stay up to date. Subscribe to my posts https://morrislee1234.wixsite.com/website/contact
Web site with my other posts by category https://morrislee1234.wixsite.com/website
Comments