Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Generation preview #36

Open
ambocclusion opened this issue Apr 21, 2024 · 12 comments
Open

Generation preview #36

ambocclusion opened this issue Apr 21, 2024 · 12 comments
Labels
enhancement New feature or request runtime ui User interface

Comments

@ambocclusion
Copy link

Is it possible to get previews of an in-progress generation, similar to what is shown in the web ui, using comfyscript? Ideally I'd like some form of stream I can process per-step.
@Chaoses-Ib Chaoses-Ib added enhancement New feature or request runtime labels Apr 21, 2024
@Chaoses-Ib
Copy link
Owner

TaskQueue._watch() currently doesn't handle previews. I'll add callbacks for them later.

Chaoses-Ib added a commit that referenced this issue Apr 21, 2024
@Chaoses-Ib
feat(runtime): add node preview display and callbacks (#36)
a131eb4
Chaoses-Ib added a commit that referenced this issue Apr 21, 2024
@Chaoses-Ib
feat(runtime): add node preview display and callbacks (#36)
6e7cd33
@Chaoses-Ib
Copy link
Owner

Chaoses-Ib commented Apr 21, 2024 •
edited
Loading

v0.5.0a1 is released. You can get previews like this:

from PIL import Image

# Prevent displaying previews
# <0.5.0
# queue.watch_display(False, False, False)
queue.watch_display(False)

with Workflow() as wf:
    model, clip, vae = CheckpointLoaderSimple('v1-5-pruned-emaonly.ckpt')
    conditioning = CLIPTextEncode('beautiful scenery nature glass bottle landscape, , purple galaxy bottle,', clip)
    conditioning2 = CLIPTextEncode('text, watermark', clip)
    latent = EmptyLatentImage(512, 512, 1)
    latent = KSampler(model, 12, 20, 8, 'euler', 'normal', conditioning, conditioning2, latent, 1)
    image = VAEDecode(latent, vae)
    SaveImage(image, 'ComfyUI')

def preview(task: Task, node_id: str, image: Image.Image):
    # 'KSampler.0'
    print(node_id)
wf.task.add_preview_callback(preview)

A stream form (sync/async iter) is also possible. But I'm not sure if it‘s worth the effort, as one can get similar results with a callback and a wait.

@ambocclusion
Copy link
Author

Holy crap, you rock! Works perfectly!

@Chaoses-Ib Chaoses-Ib reopened this Apr 25, 2024
@Chaoses-Ib
Copy link
Owner

There are still some things left to do, like reporting progress and preview together, and fixing Jupyter Notebook display issues.

@ambocclusion
Copy link
Author

Ah sorry

@ambocclusion
Copy link
Author

screen-20240425-115831.3.mp4

Here are the results of your preview work! Again, I'm really grateful for your work on Comfyscript! Thank you

@Chaoses-Ib

This comment was marked as resolved.

Sign in to view
@ghostsquad
Copy link

v0.5.0a1 is released. You can get previews like this:

from PIL import Image

# Prevent displaying previews
queue.watch_display(False, False, False)

with Workflow() as wf:
    model, clip, vae = CheckpointLoaderSimple('v1-5-pruned-emaonly.ckpt')
    conditioning = CLIPTextEncode('beautiful scenery nature glass bottle landscape, , purple galaxy bottle,', clip)
    conditioning2 = CLIPTextEncode('text, watermark', clip)
    latent = EmptyLatentImage(512, 512, 1)
    latent = KSampler(model, 12, 20, 8, 'euler', 'normal', conditioning, conditioning2, latent, 1)
    image = VAEDecode(latent, vae)
    SaveImage(image, 'ComfyUI')

def preview(task: Task, node_id: str, image: Image.Image):
    # 'KSampler.0'
    print(node_id)
wf.task.add_preview_callback(preview)

A stream form (sync/async iter) is also possible. But I'm not sure if it‘s worth the effort, as one can get similar results with a callback and a wait.

this code block doesn't make entire sense. Maybe I'm just a bit rusty with python. wf would only be valid within the context of the with block, right?

@Chaoses-Ib
Copy link
Owner

with block is justing calling wf.__enter__() and wf.__exit__(). And __exit__() for Workflow is just queuing the workflow to the server. So using it after __exit__() is fine.

But you are right, this is not a very intuitive API. Maybe I should move it as an arg of Workflow(), of allow set the callback in the with block.

@Chaoses-Ib Chaoses-Ib added the ui User interface label Jun 20, 2024
Chaoses-Ib added a commit that referenced this issue Aug 1, 2024
@Chaoses-Ib
docs(Images): add retrieving images examples (#64, #36)
aa8d728
@angelmankel
Copy link

angelmankel commented Aug 20, 2024 •
edited
Loading

Hi @Chaoses-Ib, I understand this is for runtime mode but I was just wondering if it was possible to get previews in real mode? Thanks in advance for any advice.

@Chaoses-Ib
Copy link
Owner

Chaoses-Ib commented Aug 20, 2024 •
edited
Loading

For now, you can directly set the hook in real mode:

from comfy_script.runtime.real import *
load(args=ComfyUIArgs('--preview-method', 'auto'))
from comfy_script.runtime.real.nodes import *

import comfy.utils

def hook(value, total, preview_image):
    print(value, total, preview_image)
comfy.utils.set_progress_bar_global_hook(hook)

with Workflow() as wf:
    model, clip, vae = CheckpointLoaderSimple('v1-5-pruned-emaonly.ckpt')
    conditioning = CLIPTextEncode('beautiful scenery nature glass bottle landscape, , purple galaxy bottle,', clip)
    conditioning2 = CLIPTextEncode('text, watermark', clip)
    latent = EmptyLatentImage(512, 512, 1)
    latent = KSampler(model, 4, 20, 8, 'euler', 'normal', conditioning, conditioning2, latent, 1)
    image = VAEDecode(latent, vae)
    image = SaveImage(image, 'ComfyUI')
'''
1 20 ('JPEG', <PIL.Image.Image image mode=RGB size=64x64 at 0x27228632CE0>, 512)
2 20 ('JPEG', <PIL.Image.Image image mode=RGB size=64x64 at 0x27228632BC0>, 512)
3 20 ('JPEG', <PIL.Image.Image image mode=RGB size=64x64 at 0x27228633AC0>, 512)
...
18 20 ('JPEG', <PIL.Image.Image image mode=RGB size=64x64 at 0x272286489D0>, 512)
19 20 ('JPEG', <PIL.Image.Image image mode=RGB size=64x64 at 0x27228648D90>, 512)
20 20 ('JPEG', <PIL.Image.Image image mode=RGB size=64x64 at 0x27228728400>, 512)
'''

Note node id and prompt id are not available in real mode. If you need to identify nodes, you need some other methods, like:

import comfy.utils

node_id = None
def hook(value, total, preview_image):
    print(value, total, preview_image)
    print(node_id)
comfy.utils.set_progress_bar_global_hook(hook)

with Workflow() as wf:
    model, clip, vae = CheckpointLoaderSimple('v1-5-pruned-emaonly.ckpt')
    conditioning = CLIPTextEncode('beautiful scenery nature glass bottle landscape, , purple galaxy bottle,', clip)
    conditioning2 = CLIPTextEncode('text, watermark', clip)
    latent = EmptyLatentImage(512, 512, 1)
    node_id = 'sampler'
    latent = KSampler(model, 3, 20, 8, 'euler', 'normal', conditioning, conditioning2, latent, 1)
    image = VAEDecode(latent, vae)
    image = SaveImage(image, 'ComfyUI')

@angelmankel
Copy link

Awesome, thanks for the quick response. I'll try this out.

Chaoses-Ib added a commit that referenced this issue Sep 6, 2024
@Chaoses-Ib
feat(runtime/virtual): change watch_display() args (#36, #52)
7994173
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request runtime ui User interface
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@ghostsquad @ambocclusion @angelmankel @Chaoses-Ib