New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add fusion script for segment anything v2 #22167

Merged

tianleiwu merged 2 commits into main from tlwu/sam2_fusion

Sep 21, 2024

Contributor

tianleiwu commented Sep 20, 2024 •

edited

Loading

Description

Add MultiHeadAttention fusion for SAM2.
Add LayerNormalization fusion for NCHW format by inserting Transpose from NCHW to NHWC before layer normalization, and add another Transpose after layer norm to convert NHWC back to NCHW. Hopefully, those extra Transpose nodes will be removed when prefer_nhwc is enabled later.
Add a condition that the input shall be 3D when fuse SkipLayerNorm.
Update convert_to_onnx.py to add --optimize and --use_gpu options to output optimized onnx model for CPU/CUDA eps.
Add an option --dtype fp16|fp32 in convert_to_onnx.py to support converting optimized model to float16.
Update the demo to use the optimized onnx models.

Motivation and Context

To support optimization of SAM2 for CPU/CUDA eps that is exported in #22119


          fusion script for SAM2

b081188

tianleiwu requested review from kunal-vaishnavi, wangyems and apsonawane and removed request for kunal-vaishnavi and wangyems

September 20, 2024 18:50

kunal-vaishnavi reviewed

View reviewed changes

onnxruntime/python/tools/transformers/fusion_attention_sam2.py Outdated Show resolved Hide resolved

kunal-vaishnavi reviewed

View reviewed changes

onnxruntime/python/tools/transformers/fusion_attention_sam2.py Show resolved Hide resolved

kunal-vaishnavi reviewed

View reviewed changes

onnxruntime/python/tools/transformers/fusion_attention_sam2.py Show resolved Hide resolved

kunal-vaishnavi reviewed

View reviewed changes

onnxruntime/python/tools/transformers/fusion_attention_sam2.py Outdated Show resolved Hide resolved

kunal-vaishnavi reviewed

View reviewed changes

onnxruntime/python/tools/transformers/fusion_layernorm.py Outdated Show resolved Hide resolved

kunal-vaishnavi reviewed

View reviewed changes

onnxruntime/python/tools/transformers/models/sam2/convert_to_onnx.py Show resolved Hide resolved

kunal-vaishnavi reviewed

View reviewed changes

onnxruntime/python/tools/transformers/models/sam2/convert_to_onnx.py Show resolved Hide resolved


          review feedback

9da0a78

tianleiwu mentioned this pull request

Add benchmark script for segment anything v2 #22169

Merged

tianleiwu requested a review from kunal-vaishnavi

September 20, 2024 23:11

kunal-vaishnavi approved these changes

View reviewed changes

tianleiwu merged commit 1431215 into main

87 checks passed

tianleiwu deleted the tlwu/sam2_fusion branch

September 21, 2024 04:32

tianleiwu added a commit that referenced this pull request


          Add benchmark script for segment anything v2 (#22169)

171b901

### Description
Add benchmark script segment anything v2. 
It depends on #22119 for
onnx export, and #22167 for
sam2 graph fusion.

### Motivation and Context

Benchmark SAM2 model performance.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet