Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix: optimize model selection logic to avoid cuda out of memory error. #26

Merged
merged 1 commit into from
Mar 21, 2023

Conversation

remiliacn
Copy link
Contributor

概述

小小的优化一下,我刚刚用的时候爆显存了hhh。

使用torch库自动检测显存大小。
应该可以有效解决小显存用户会运行的时候默认使用fp16然后报RuntimeError: CUDA out of memory的问题

关于precision参数是怎么决定的参考的这里

测试结果

Explicitly passing a `revision` is encouraged when loading a model with custom code to ensure no malicious code has been contributed in a newer revision.
Explicitly passing a `revision` is encouraged when loading a configuration with custom code to ensure no malicious code has been contributed in a newer revision.
Explicitly passing a `revision` is encouraged when loading a model with custom code to ensure no malicious code has been contributed in a newer revision.
Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 8/8 [00:13<00:00,  1.65s/it]
GPU memory: 10.74 GB
Choosing precision int8 according to your VRAM. If you want to decide precision yourself, please add argument --precision when launching the application.
Running on local URL:  http://127.0.0.1:17860

To create a public link, set `share=True` in `launch()`.

@Akegarasu Akegarasu merged commit e4d219d into Akegarasu:main Mar 21, 2023
@remiliacn remiliacn deleted the precision_optimization branch March 21, 2023 06:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants