Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dev refactor onediff comfy nodes #796

Merged
merged 14 commits into from
Apr 11, 2024
Merged

Dev refactor onediff comfy nodes #796

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
e2aa54c
refactor_onediff_comfy_nodes
ccssu Apr 9, 2024
b6b81f9
add OneDiffOnlineQuantizationOptimizer
ccssu Apr 10, 2024
389fdb8
Delete redundant code
ccssu Apr 10, 2024
5c45d9f
add OnlineQuantization.md
ccssu Apr 10, 2024
2596ef1
refine
ccssu Apr 10, 2024
9655038
Merge branch 'main' into dev_refactor_onediff_comfy_nodes
ccssu Apr 10, 2024
e5a116f
controlnet Compatibility compilation options changed
ccssu Apr 10, 2024
b6552af
fix No newline at end of file warning
ccssu Apr 10, 2024
ba6cc46
fix No newline at end of file warning
ccssu Apr 10, 2024
fa3361b
fix set comfyui root
ccssu Apr 10, 2024
89f4569
renamed Optimizer through Booster
ccssu Apr 11, 2024
9cadca1
fix No module named onediff_quant in onediff_comfy_nodes/modules/onef…
ccssu Apr 11, 2024
a8a7c5b
Merge branch 'main' into dev_refactor_onediff_comfy_nodes
ccssu Apr 11, 2024
89ba5cd
fix check in nodes_oneflow_booster.py
ccssu Apr 11, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
refine
  • Loading branch information
@ccssu
ccssu committed Apr 10, 2024
commit 2596ef15280b94545e6fb59d7b49c7d9e13c5742
18 changes: 15 additions & 3 deletions onediff_comfy_nodes/extras_nodes/nodes_oneflow_optimizer.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
import torch
from comfy import model_management
from comfy.cli_args import args

from onediff.infer_compiler.utils import is_community_version

from ..modules.oneflow.config import ONEDIFF_QUANTIZED_OPTIMIZED_MODELS
Expand All @@ -17,12 +18,14 @@
from ..modules.oneflow.optimizer_basic import BasicOneFlowOptimizerExecutor
from ..modules.oneflow.optimizer_deepcache import DeepcacheOptimizerExecutor
from ..modules.oneflow.optimizer_patch import PatchOptimizerExecutor
from ..modules.oneflow.optimizer_quantization import \
OnelineQuantizationOptimizerExecutor
from ..modules.oneflow.utils import OUTPUT_FOLDER, load_graph, save_graph
from ..modules.optimizer_scheduler import OptimizerScheduler
from ..utils.import_utils import is_onediff_quant_available

if is_onediff_quant_available() and not is_community_version():
from ..modules.oneflow.optimizer_quantization import \
OnelineQuantizationOptimizerExecutor

model_management_hijacker.hijack() # add flow.cuda.empty_cache()
nodes_hijacker.hijack()
samplers_hijack.hijack()
Expand Down Expand Up @@ -219,6 +222,13 @@ def INPUT_TYPES(s):

@torch.no_grad()
def apply(self, quantized_conv_percentage=0, quantized_linear_percentage=0, conv_compute_density_threshold=0, linear_compute_density_threshold=0):
if not is_onediff_quant_available() and is_community_version():
raise RuntimeError(f'OneDiff quantization and community version are not available. '
f'Please refer to the documentation for reinstalling OneDiff Enterprise: '
f'https://github.com/siliconflow/onediff/blob/main/README_ENTERPRISE.md#install-onediff-enterprise\n'
f'is_community_version={is_community_version()}\n'
f'is_onediff_quant_available={is_onediff_quant_available()}')

return (
OnelineQuantizationOptimizerExecutor(
conv_percentage=quantized_conv_percentage,
Expand Down Expand Up @@ -463,6 +473,7 @@ def save_graph(self, samples, model, filename_prefix):
"BatchSizePatcher": BatchSizePatcher,
"OneDiffOnlineQuantizationOptimizer": OneDiffOnlineQuantizationOptimizer,
"SVDSpeedup": SVDSpeedup,
"OneFlowDeepcacheOptimizer": OneFlowDeepcacheOptimizer,
}

NODE_DISPLAY_NAME_MAPPINGS = {
Expand All @@ -475,7 +486,8 @@ def save_graph(self, samples, model, filename_prefix):
"OneDiffControlNetLoader": "Load ControlNet Model - OneDiff",
"OneDiffDeepCacheCheckpointLoaderSimple": "Load Checkpoint - OneDiff DeepCache",
"BatchSizePatcher": "Batch Size Patcher",
"OneDiffOnlineQuantizationOptimizer": "Online OneFlow Quantizer - OneDiff"
"OneDiffOnlineQuantizationOptimizer": "Online OneFlow Quantizer - OneDiff",
"OneFlowDeepcacheOptimizer": "OneFlow Deepcache Optimizer - OneDiff"
}


Expand Down
12 changes: 11 additions & 1 deletion onediff_comfy_nodes/workflows/OnlineQuantization.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@

Notes:

1. Specify the directory for saving graphsusing export COMFYUI_ONEDIFF_SAVE_GRAPH_DIR="/path/to/save/graphs".
1. Specify the directory for saving graphs using export COMFYUI_ONEDIFF_SAVE_GRAPH_DIR="/path/to/save/graphs".
2. The log *.pt file is cached. Quantization result information can be found in `cache_dir`/quantization_stats.json.

## Performance Comparison
Expand All @@ -39,3 +39,13 @@ Updated on Mon 08 Apr 2024

### SVD
![svd_quant](https://github.com/siliconflow/onediff/assets/109639975/93ebe3d5-8413-4a7e-8b93-fd016f61abe9)

| Accelerator | Baseline (non-optimized) | OneDiff(optimized) | OneDiff Quant(optimized) |
| ----------------------- | ------------------------ | ------------------ | ------------------------ |
| NVIDIA A800-SXM4-80GB | 63.64 s | 47.57 s ( ~25.25%) | 49.83 s ( ~21.7%) |

- torch `python -c "import torch; print(torch.__version__)"`: {version: 2.4.0.dev20240405+cu121}
- oneflow `python -m oneflow --doctor`: {git_commit: 4ed3138, version: 0.9.1.dev20240402+cu122, enterprise: True}
- ComfyUI Tue Apr 9 commit: 4201181b35402e0a992b861f8d2f0e0b267f52fa
- Start comfyui command: `python main.py --gpu-only`
- Python 3.10.13
Loading