top of page

News to help your R&D in artificial intelligence, machine learning, robotics, computer vision, smart hardware

As an Amazon Associate I earn

from qualifying purchases

Writer's picturemorrislee

Segment unknown object match text description by generative and discriminative models with Ref-Diff

Segment unknown object match text description by generative and discriminative models with Ref-Diff


Ref-Diff: Zero-shot Referring Image Segmentation with Generative Models

arXiv paper abstract https://arxiv.org/abs/2308.16777



Zero-shot referring image segmentation ... aims to find an instance segmentation mask based on the given referring descriptions, without training on this type of paired data.


Current ... methods mainly ... using pre-trained discriminative models (e.g., CLIP). However... generative models (e.g., Stable Diffusion) have potentially understood the relationships between various visual elements and text descriptions


... introduce a novel Referring Diffusional segmentor (Ref-Diff) for this task, which leverages the fine-grained multi-modal information from generative models.


... demonstrate that without a proposal generator, a generative model alone can achieve comparable performance to existing SOTA weakly-supervised models.


When ... combine both generative and discriminative models, ... Ref-Diff outperforms these competing methods by a significant margin.


This indicates that generative models are also beneficial for this task and can complement discriminative models for better referring segmentation ...



Please like and share this post if you enjoyed it using the buttons at the bottom!


Stay up to date. Subscribe to my posts https://morrislee1234.wixsite.com/website/contact

Web site with my other posts by category https://morrislee1234.wixsite.com/website



41 views0 comments

Comments


ClickBank paid link

bottom of page