Segment object in video from text description by global information from object queries with TempCD
Segment object in video from text description by global information from object queries with TempCD
Temporal Collection and Distribution for Referring Video Object Segmentation
arXiv paper abstract https://arxiv.org/abs/2309.03473
arXiv PDF paper https://arxiv.org/pdf/2309.03473.pdf
Project page https://toneyaya.github.io/tempcd
Referring video object segmentation aims to segment a referent throughout a video sequence according to a natural language expression ... aligning the ... language ... with the objects' motions and their ... associations at the global ... level but segmenting objects at the frame level.
... propose to simultaneously maintain a global referent token and a sequence of object queries, where the former is responsible for capturing video-level referent according to the language expression, while the latter serves to ... segment objects with each frame.
Furthermore, to explicitly capture object motions and spatial-temporal cross-modal reasoning over objects, ... propose a novel temporal collection-distribution mechanism for interacting between the global referent token and object queries.
Specifically, the temporal collection mechanism collects global information for the referent token from object queries to the temporal motions to the language expression.
In turn, the temporal distribution first distributes the referent token to the referent sequence across all frames and then performs efficient cross-frame reasoning between the referent sequence and object queries in every frame.
... method outperforms state-of-the-art methods on all benchmarks consistently and significantly.
Please like and share this post if you enjoyed it using the buttons at the bottom!
Stay up to date. Subscribe to my posts https://morrislee1234.wixsite.com/website/contact
Web site with my other posts by category https://morrislee1234.wixsite.com/website
Comments