Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

whisper : add Metal support in the Decoder #1047

Closed
ggerganov opened this issue Jun 25, 2023 · 2 comments
Closed

whisper : add Metal support in the Decoder #1047

ggerganov opened this issue Jun 25, 2023 · 2 comments
Assignees
@ggerganov
Labels
decoding Decoding related issues performance CPU and memory usage - results and comparisons

Comments

@ggerganov
Copy link
Owner

GPU inference on Apple Silicon via Metal backend was recently added to llama.cpp: ggerganov/llama.cpp#1642

We should port the changes to whisper.cpp and allow the Decoder to run on the GPU in a similar way
@ggerganov ggerganov added performance CPU and memory usage - results and comparisons decoding Decoding related issues labels Jun 25, 2023
ggerganov added a commit that referenced this issue Jun 25, 2023
@ggerganov
metal : sync ggml-metal (ref #1047)
f1c9df5
@ggerganov ggerganov self-assigned this Jun 25, 2023
@ggerganov ggerganov moved this to In Progress in ggml : roadmap Jun 25, 2023
@RRUK01 RRUK01 mentioned this issue Aug 6, 2023
Closed
@ggerganov
Copy link
Owner Author

Done via #whisper #1270

@ggerganov ggerganov moved this from In Progress to Done in ggml : roadmap Sep 15, 2023
@aehlke
Copy link

aehlke commented Sep 16, 2023

What are the instructions for running this (Metal, or Metal + non-ANE OpenML)? Sorry, couldn't figure out from the changes whether the most performant config is now the default or how to configure it to run with this new addition. Thanks.

jacobwu-b pushed a commit to jacobwu-b/Transcriptify-by-whisper.cpp that referenced this issue Oct 24, 2023
@ggerganov
metal : sync ggml-metal (ref ggerganov#1047)
4f0124e
jacobwu-b pushed a commit to jacobwu-b/Transcriptify-by-whisper.cpp that referenced this issue Oct 24, 2023
@ggerganov
metal : sync ggml-metal (ref ggerganov#1047)
5e24007
landtanin pushed a commit to landtanin/whisper.cpp that referenced this issue Dec 16, 2023
@ggerganov
metal : sync ggml-metal (ref ggerganov#1047)
7856391
iThalay pushed a commit to iThalay/whisper.cpp that referenced this issue Sep 23, 2024
@ggerganov @iThalay
metal : sync ggml-metal (ref ggerganov#1047)
3b9e496
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ggerganov ggerganov

Labels
decoding Decoding related issues performance CPU and memory usage - results and comparisons
Projects
ggml : roadmap
Status: Done
Milestone
No milestone
Development

No branches or pull requests
2 participants
@aehlke @ggerganov