--- library_name: pytorch license: other tags: - foundation - android pipeline_tag: image-segmentation --- ![](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/mobilesam/web-assets/model_demo.png) # MobileSam: Optimized for Qualcomm Devices Transformer based encoder-decoder where prompts specify what to segment in an image thereby allowing segmentation without the need for additional training. The image encoder generates embeddings and the lightweight decoder operates on the embeddings for point and mask based image segmentation. This is based on the implementation of MobileSam found [here](https://github.com/facebookresearch/segment-anything). This repository contains pre-exported model files optimized for Qualcomm® devices. You can use the [Qualcomm® AI Hub Models](https://github.com/quic/ai-hub-models/blob/main/qai_hub_models/models/mobilesam) library to export with custom configurations. More details on model performance across various devices, can be found [here](#performance-summary). Qualcomm AI Hub Models uses [Qualcomm AI Hub Workbench](https://workbench.aihub.qualcomm.com) to compile, profile, and evaluate this model. [Sign up](https://myaccount.qualcomm.com/signup) to run these models on a hosted Qualcomm® device. ## Getting Started There are two ways to deploy this model on your device: ### Option 1: Download Pre-Exported Models Below are pre-exported model assets ready for deployment. | Runtime | Precision | Chipset | SDK Versions | Download | |---|---|---|---|---| | ONNX | float | Universal | QAIRT 2.37, ONNX Runtime 1.23.0 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/mobilesam/releases/v0.46.0/mobilesam-onnx-float.zip) | QNN_DLC | float | Universal | QAIRT 2.42 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/mobilesam/releases/v0.46.0/mobilesam-qnn_dlc-float.zip) | TFLITE | float | Universal | QAIRT 2.42, TFLite 2.17.0 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/mobilesam/releases/v0.46.0/mobilesam-tflite-float.zip) For more device-specific assets and performance metrics, visit **[MobileSam on Qualcomm® AI Hub](https://aihub.qualcomm.com/models/mobilesam)**. ### Option 2: Export with Custom Configurations Use the [Qualcomm® AI Hub Models](https://github.com/quic/ai-hub-models/blob/main/qai_hub_models/models/mobilesam) Python library to compile and export the model with your own: - Custom weights (e.g., fine-tuned checkpoints) - Custom input shapes - Target device and runtime configurations This option is ideal if you need to customize the model beyond the default configuration provided here. See our repository for [MobileSam on GitHub](https://github.com/quic/ai-hub-models/blob/main/qai_hub_models/models/mobilesam) for usage instructions. ## Model Details **Model Type:** Model_use_case.semantic_segmentation **Model Stats:** - Model checkpoint: vit_t - Input resolution: 720p (720x1280) - Number of parameters (SAMEncoder): 6.95M - Model size (SAMEncoder) (float): 26.6 MB - Number of parameters (SAMDecoder): 6.16M - Model size (SAMDecoder) (float): 23.7 MB ## Performance Summary | Model | Runtime | Precision | Chipset | Inference Time (ms) | Peak Memory Range (MB) | Primary Compute Unit |---|---|---|---|---|---|--- | MobileSAMDecoder | ONNX | float | Snapdragon® 8 Gen 3 Mobile | 5.734 ms | 5 - 218 MB | NPU | MobileSAMDecoder | ONNX | float | Qualcomm® QCS8550 (Proxy) | 8.389 ms | 4 - 230 MB | NPU | MobileSAMDecoder | ONNX | float | Qualcomm® QCS9075 | 10.04 ms | 4 - 7 MB | NPU | MobileSAMDecoder | ONNX | float | Snapdragon® 8 Elite For Galaxy Mobile | 4.263 ms | 2 - 180 MB | NPU | MobileSAMDecoder | ONNX | float | Snapdragon® 8 Elite Gen 5 Mobile | 3.481 ms | 0 - 186 MB | NPU | MobileSAMDecoder | QNN_DLC | float | Snapdragon® X Elite | 5.137 ms | 4 - 4 MB | NPU | MobileSAMDecoder | QNN_DLC | float | Snapdragon® 8 Gen 3 Mobile | 3.319 ms | 4 - 230 MB | NPU | MobileSAMDecoder | QNN_DLC | float | Qualcomm® QCS8275 (Proxy) | 11.595 ms | 2 - 199 MB | NPU | MobileSAMDecoder | QNN_DLC | float | Qualcomm® QCS8550 (Proxy) | 4.692 ms | 4 - 6 MB | NPU | MobileSAMDecoder | QNN_DLC | float | Qualcomm® SA8775P | 5.737 ms | 1 - 200 MB | NPU | MobileSAMDecoder | QNN_DLC | float | Qualcomm® QCS8450 (Proxy) | 8.499 ms | 0 - 222 MB | NPU | MobileSAMDecoder | QNN_DLC | float | Qualcomm® SA7255P | 11.595 ms | 2 - 199 MB | NPU | MobileSAMDecoder | QNN_DLC | float | Qualcomm® SA8295P | 7.157 ms | 0 - 193 MB | NPU | MobileSAMDecoder | QNN_DLC | float | Snapdragon® 8 Elite For Galaxy Mobile | 2.538 ms | 0 - 198 MB | NPU | MobileSAMDecoder | QNN_DLC | float | Snapdragon® 8 Elite Gen 5 Mobile | 2.182 ms | 4 - 203 MB | NPU | MobileSAMDecoder | TFLITE | float | Snapdragon® 8 Gen 3 Mobile | 3.854 ms | 0 - 237 MB | NPU | MobileSAMDecoder | TFLITE | float | Qualcomm® QCS8275 (Proxy) | 12.771 ms | 0 - 208 MB | NPU | MobileSAMDecoder | TFLITE | float | Qualcomm® QCS8550 (Proxy) | 5.561 ms | 0 - 18 MB | NPU | MobileSAMDecoder | TFLITE | float | Qualcomm® SA8775P | 6.547 ms | 0 - 209 MB | NPU | MobileSAMDecoder | TFLITE | float | Qualcomm® QCS8450 (Proxy) | 9.544 ms | 0 - 225 MB | NPU | MobileSAMDecoder | TFLITE | float | Qualcomm® SA7255P | 12.771 ms | 0 - 208 MB | NPU | MobileSAMDecoder | TFLITE | float | Qualcomm® SA8295P | 8.152 ms | 0 - 205 MB | NPU | MobileSAMDecoder | TFLITE | float | Snapdragon® 8 Elite For Galaxy Mobile | 2.935 ms | 0 - 209 MB | NPU | MobileSAMEncoder | ONNX | float | Snapdragon® 8 Gen 3 Mobile | 254.846 ms | 128 - 866 MB | NPU | MobileSAMEncoder | ONNX | float | Qualcomm® QCS8550 (Proxy) | 341.905 ms | 114 - 116 MB | NPU | MobileSAMEncoder | ONNX | float | Qualcomm® QCS9075 | 414.464 ms | 68 - 71 MB | NPU | MobileSAMEncoder | ONNX | float | Snapdragon® 8 Elite For Galaxy Mobile | 185.472 ms | 117 - 762 MB | NPU | MobileSAMEncoder | ONNX | float | Snapdragon® 8 Elite Gen 5 Mobile | 152.087 ms | 130 - 692 MB | NPU | MobileSAMEncoder | QNN_DLC | float | Snapdragon® X Elite | 99.194 ms | 12 - 12 MB | NPU | MobileSAMEncoder | QNN_DLC | float | Snapdragon® 8 Gen 3 Mobile | 65.564 ms | 11 - 1629 MB | NPU | MobileSAMEncoder | QNN_DLC | float | Qualcomm® QCS8275 (Proxy) | 209.778 ms | 0 - 1209 MB | NPU | MobileSAMEncoder | QNN_DLC | float | Qualcomm® QCS8550 (Proxy) | 96.271 ms | 12 - 16 MB | NPU | MobileSAMEncoder | QNN_DLC | float | Qualcomm® SA8775P | 102.588 ms | 0 - 1614 MB | NPU | MobileSAMEncoder | QNN_DLC | float | Qualcomm® QCS8450 (Proxy) | 467.917 ms | 12 - 1732 MB | NPU | MobileSAMEncoder | QNN_DLC | float | Qualcomm® SA7255P | 209.778 ms | 0 - 1209 MB | NPU | MobileSAMEncoder | QNN_DLC | float | Qualcomm® SA8295P | 474.029 ms | 0 - 1248 MB | NPU | MobileSAMEncoder | QNN_DLC | float | Snapdragon® 8 Elite For Galaxy Mobile | 49.974 ms | 12 - 1214 MB | NPU | MobileSAMEncoder | QNN_DLC | float | Snapdragon® 8 Elite Gen 5 Mobile | 44.853 ms | 11 - 1322 MB | NPU | MobileSAMEncoder | TFLITE | float | Snapdragon® 8 Gen 3 Mobile | 127.122 ms | 4 - 1521 MB | NPU | MobileSAMEncoder | TFLITE | float | Qualcomm® QCS8275 (Proxy) | 292.444 ms | 4 - 1275 MB | NPU | MobileSAMEncoder | TFLITE | float | Qualcomm® QCS8550 (Proxy) | 176.108 ms | 0 - 3 MB | NPU | MobileSAMEncoder | TFLITE | float | Qualcomm® SA8775P | 180.445 ms | 4 - 1275 MB | NPU | MobileSAMEncoder | TFLITE | float | Qualcomm® QCS8450 (Proxy) | 588.02 ms | 0 - 3179 MB | NPU | MobileSAMEncoder | TFLITE | float | Qualcomm® SA7255P | 292.444 ms | 4 - 1275 MB | NPU | MobileSAMEncoder | TFLITE | float | Qualcomm® SA8295P | 504.031 ms | 4 - 1899 MB | NPU | MobileSAMEncoder | TFLITE | float | Snapdragon® 8 Elite For Galaxy Mobile | 96.546 ms | 3 - 1271 MB | NPU ## License * The license for the original implementation of MobileSam can be found [here](https://github.com/facebookresearch/segment-anything/blob/main/LICENSE). ## References * [Segment Anything](https://arxiv.org/abs/2306.14289) * [Source Model Implementation](https://github.com/facebookresearch/segment-anything) ## Community * Join [our AI Hub Slack community](https://aihub.qualcomm.com/community/slack) to collaborate, post questions and learn more about on-device AI. * For questions or feedback please [reach out to us](mailto:ai-hub-support@qti.qualcomm.com).