Improve object segmentation in video using instance information with ISVOS
Improve object segmentation in video using instance information with ISVOS
Look Before You Match: Instance Understanding Matters in Video Object Segmentation
arXiv paper abstract https://arxiv.org/abs/2212.06826
arXiv PDF paper https://arxiv.org/pdf/2212.06826.pdf
... memory-based methods have demonstrated impressive results in video object segmentation (VOS) recently.
Nevertheless, due to the lack of instance understanding ability, the above approaches are oftentimes brittle to large appearance variations or viewpoint changes resulted from the movement of objects and cameras.
... argue that instance understanding matters in VOS, and integrating it with memory-based matching can enjoy the synergy, which is intuitively sensible from the definition of VOS task, i.e., identifying and segmenting object instances within the video.
... present a two-branch network for VOS, where the query-based instance segmentation (IS) branch delves into the instance details of the current frame and the VOS branch performs spatial-temporal matching with the memory bank.
... employ the well-learned object queries from IS branch to inject instance-specific information into the query key, with which the instance-augmented matching is further performed.
... achieves state-of-the-art performance ... outperforming alternative methods by clear margins.
Please like and share this post if you enjoyed it using the buttons at the bottom!
Stay up to date. Subscribe to my posts https://morrislee1234.wixsite.com/website/contact
Web site with my other posts by category https://morrislee1234.wixsite.com/website
Comments