Detect unknown objects using DINO and SAM to generate object proposals with NIDS-Net
Detect unknown objects using DINO and SAM to generate object proposals with NIDS-Net
Adapting Pre-Trained Vision Models for Novel Instance Detection and Segmentation
arXiv paper abstract https://arxiv.org/abs/2405.17859
arXiv PDF paper https://arxiv.org/pdf/2405.17859
Novel Instance Detection and Segmentation (NIDS) aims at detecting and segmenting novel object instances given a few examples of each instance.
... propose a unified framework (NIDS-Net) comprising object proposal generation, embedding creation for both instance templates and proposal regions, and embedding matching for instance label assignment.
... utilize the Grounding DINO and Segment Anything Model (SAM) to obtain object proposals with accurate bounding boxes and masks.
... generation of ... instance embeddings ... utilize foreground feature averages of patch embeddings from the DINOv2 ViT backbone, followed by refinement through a weight adapter
... weight adapter can adjust the embeddings locally within their feature space and effectively limit overfitting ... enables a straightforward matching strategy
... framework surpasses ... state-of-the-art ... across four detection datasets ... instance segmentation ... outperforms the top RGB methods ... competitive with the best RGB-D method ...
Please like and share this post if you enjoyed it using the buttons at the bottom!
Stay up to date. Subscribe to my posts https://morrislee1234.wixsite.com/website/contact
Web site with my other posts by category https://morrislee1234.wixsite.com/website
Comments