Real-time video object segmentation by encoding frame smoothness and low memory with MAVOS
Real-time video object segmentation by encoding frame smoothness and low memory with MAVOS
Efficient Video Object Segmentation via Modulated Cross-Attention Memory
arXiv paper abstract https://arxiv.org/abs/2403.17937
arXiv PDF paper https://arxiv.org/pdf/2403.17937.pdf
Project page https://github.com/Amshaker/MAVOS
Recently, transformer-based approaches have shown promising results for semi-supervised video object segmentation.
... approaches typically struggle on long videos due to increased GPU memory demands, as they frequently expand the memory bank every few frames.
... propose a transformer-based approach, named MAVOS, that introduces an optimized and dynamic long-term modulated cross-attention (MCA) memory to model temporal smoothness without requiring frequent memory expansion.
... MCA effectively encodes both local and global features at various levels of granularity while efficiently maintaining consistent speed regardless of the video length.
... contributions leading to real-time inference and markedly reduced memory demands without any degradation in segmentation accuracy on long videos.
Compared to ... transformer-based ... MAVOS increases the speed by 7.6x ... reducing the GPU memory by 87% with comparable segmentation performance on short and long video datasets ...
Please like and share this post if you enjoyed it using the buttons at the bottom!
Stay up to date. Subscribe to my posts https://morrislee1234.wixsite.com/website/contact
Web site with my other posts by category https://morrislee1234.wixsite.com/website
