top of page

News to help your R&D in artificial intelligence, machine learning, robotics, computer vision, smart hardware

As an Amazon Associate I earn

from qualifying purchases

Writer's picturemorrislee

Segment objects in a video that are mentioned in a text query

Segment objects in a video that are mentioned in a text query


End-to-End Referring Video Object Segmentation with Multimodal Transformers

arXiv paper abstract https://arxiv.org/abs/2111.14821



The referring video object segmentation task (RVOS) involves segmentation of a text-referred object instance in the frames of a given video.


... complex ... multimodal task, which combines text reasoning, video understanding, instance segmentation and tracking, existing approaches typically rely on sophisticated pipelines in order to tackle it.


... propose ... simple ... Multimodal Tracking Transformer (MTTR), models the RVOS task as a sequence prediction problem.


... MTTR is based on the realization that video and text can both be processed together effectively and elegantly by a single multimodal Transformer model.


MTTR is end-to-end trainable, free of text-related inductive bias components and requires no additional mask-refinement post-processing steps.


... MTTR significantly outperforms previous art ... impressive +5.7 and +5.0 mAP gains on the A2D-Sentences and JHMDB-Sentences datasets respectively, while processing 76 frames per second. ...



Please like and share this post if you enjoyed it using the buttons at the bottom!


Stay up to date. Subscribe to my posts https://morrislee1234.wixsite.com/website/contact

Web site with my other posts by category https://morrislee1234.wixsite.com/website


31 views0 comments

Comments


ClickBank paid link

bottom of page