Serve Segment Anything Model

Overview • How to Run • Model application examples • Controls • Acknowledgment

Overview

Application key points:

• Manually selected ROI
• Deploy on GPU(faster) or CPU(slower)
• Accurate predictions in most cases
• Correct prediction interactively with red and green clicks
• Select one of 3 pretrained models
• Models are class agnostic, you can segment any object from any domain

The Segment Anything Model (SAM) can generate masks for objects on images using different types of prompts such as points and bounding boxes, and it can also be applied to raw image to generate masks for all objects.

Besides segmenting new objects, proposed method allows to correct external masks, e.g. produced by other instance or semantic segmentation models. A user can fix false negative and false positive regions with positive (green) and negative (red) clicks, respectively.

🔥🔥🔥 Check out our youtube tutorial and the complete guide in our blog:

How to Run

Pretrained models

1. Start the application from Ecosystem.

2. Select the pretrained model and deploy it on your device by clicking Serve button.

3. You'll see Model has been successfully loaded message indicating that the application has been successfully started and you can work with it from now on.

Custom models

Copy model file path from Team Files and select model architecture:

Screencast2023-04-17.20.11.26.mp4

Model application examples

Single-click segmentation of a complicated object using Smart Tool:

Screencast2023-04-17.19.35.45.mp4

Mask correction with Positive and Negative points in Smart Tool:

vid2.mp4

Applying model in raw mode via NN image labeling app:

vid3.mp4

Applying model to object in bounding box:

vid4.mp4

Applying model to the points:

Screencast2023-04-20.16.20.32.mp4

Applying model in the combined mode (using both points and bounding box):

vid6.mp4

If you want predicted masks to be automatically overdrawn be replacing old masks with new masks, you can set "replace_masks" parameter to true:

Screencast2023-04-20.16.14.09.mp4

You can also use this app for segmenting objects on videos using video labeling tool (here is full guide on working with smart tool in video labeling tool):

Screencast2023-04-20.18.25.56.mp4

If you want to label batch of images fast, you can use this app in combination with batched smart tool:

sam_batched_smart_tool.mp4

Controls

Key	Description
Left Mouse Button	Place a positive click
Shift + Left Mouse Button	Place a negative click
Scroll Wheel	Zoom an image in and out
Right Mouse Button + Move Mouse	Move an image
Space	Finish the current object mask
Shift + H	Higlight instances with random colors
Ctrl + H	Hide all labels

— Auto add positivie point to rectangle button (ON by default for SmartTool apps)

— SmartTool selector button, switch between SmartTool apps and models

Acknowledgment

This app is based on the great work Segment Anything: github.