Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Dev refactor onediff comfy nodes #796

Merged
merged 14 commits into from
Apr 11, 2024
Merged

Dev refactor onediff comfy nodes #796

merged 14 commits into from
Apr 11, 2024

Conversation

ccssu
Copy link
Contributor

@ccssu ccssu commented Apr 10, 2024 •
edited
Loading

Online quantization for comfyui

Install

  1. OneDiff Installation Guide
  2. OneDiffx Installation Guide

Usage:

Option Range Default Description
quantized_conv_percentage [0, 100] 100 Example value representing 100% quantization for linear layers｜
quantized_linear_percentage [0, 100] 100 Example value representing 100% quantization for convolutional layers
conv_compute_density_threshold [0, ∞) 100 Computational density threshold for quantizing convolutional modules to 100.
linear_compute_density_threshold [0, ∞) 300 Computational density threshold for quantizing linear modules to 300.

Notes:

  1. Specify the directory for saving graphsusing export COMFYUI_ONEDIFF_SAVE_GRAPH_DIR="/path/to/save/graphs".
  2. The log *.pt file is cached. Quantization result information can be found in cache_dir/quantization_stats.json.

Performance Comparison

SDXL

Updated on Mon 08 Apr 2024

quant_sdxl

Accelerator Baseline (non-optimized) OneDiff(optimized) OneDiff Quant(optimized)
NVIDIA GeForce RTX 3090 8.03 s 4.44 s ( ~44.7%) 3.34 s ( ~58.4%)
  • torch python -c "import torch; print(torch.__version__)": {version: 2.2.1+cu121}
  • oneflow python -m oneflow --doctor: {git_commit: 710818c, version: 0.9.1.dev20240406+cu121, enterprise: True}
  • Start comfyui command: python main.py --gpu-only

SD1.5

sd 1.5

SVD

svd_quant

@ccssu ccssu requested review from doombeaker and lixiang007666 and removed request for doombeaker April 10, 2024 03:00
@ccssu
Copy link
Contributor Author

ccssu commented Apr 10, 2024 •
edited
Loading

SVD

Accelerator Baseline (non-optimized) OneDiff(optimized) OneDiff Quant(optimized)
NVIDIA A800-SXM4-80GB 35.54 s 25.59 s (27.99 %) 22.30 s (37.25 %)

The following table shows the workflows used separately：

Note that you can download all images in this page and then drag or load them on ComfyUI to get the workflow embedded in the image.

Baseline (non-optimized) OneDiff(optimized) OneDiff Quant(optimized)
svd_baseline OneDiff(optimized) OneDiff Quant(optimized)
Download the required model files

For NA/EU users

cd ComfyUI

wget -O models/checkpoints/sd_xl_base_1.0.safetensors https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0/resolve/main/sd_xl_base_1.0.safetensors

wget -O models/checkpoints/svd_xt_1_1.safetensors https://huggingface.co/vdo/stable-video-diffusion-img2vid-xt-1-1/resolve/main/svd_xt_1_1.safetensors

For CN users

cd ComfyUI

wget -O models/checkpoints/sd_xl_base_1.0.safetensors https://hf-mirror.com/stabilityai/stable-diffusion-xl-base-1.0/resolve/main/sd_xl_base_1.0.safetensors

wget -O models/checkpoints/svd_xt_1_1.safetensors https://hf-mirror.com/vdo/stable-video-diffusion-img2vid-xt-1-1/resolve/main/svd_xt_1_1.safetensors

Environment

  • torch : {version: 2.2.1+cu121}
  • oneflow : {git_commit: 8e20ea9, version: 0.9.1.dev20240410+cu122, enterprise: True}
  • ComfyUI Tue Apr 9 commit: 4201181b35402e0a992b861f8d2f0e0b267f52fa
  • Start comfyui command: python main.py --gpu-only
  • Python 3.10.13

@ccssu ccssu requested a review from doombeaker April 11, 2024 06:47
@ccssu ccssu enabled auto-merge (squash) April 11, 2024 08:08
@ccssu ccssu merged commit e4d56a0 into main Apr 11, 2024
5 checks passed
@ccssu ccssu deleted the dev_refactor_onediff_comfy_nodes branch April 11, 2024 08:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@lixiang007666 lixiang007666 lixiang007666 left review comments

@hjchen2 hjchen2 hjchen2 left review comments

@doombeaker doombeaker doombeaker approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@ccssu @doombeaker @hjchen2 @lixiang007666