Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: add metadata to images #1940

Merged
merged 71 commits into from
Feb 26, 2024
Merged

feat: add metadata to images #1940

merged 71 commits into from
Feb 26, 2024
+864 −178

Conversation

mashb1t
Copy link
Collaborator

@mashb1t mashb1t commented Jan 15, 2024 •
edited
Loading

Resolves #241 + #1996
Merge #1932 to also enable metadata in images in the browser, not only images in the outputs folder.

This feature is in contrast to #160 and might not be merged, but at least if offers the functionality to whoever wants to use it locally, as there was demand for it.

I've implemented both a Fooocus (json) and A1111 (plain text) meta data scheme, where the latter is 100% compatible with A1111 and Civitai, but can not be used to reproduce the image outside of Fooocus, as there are so many improvements and special things happening in Fooocus it's just not applicable anywhere else.

The A1111 meta data scheme is based on A1111 infotext handling and i've re-implemented the style resolution feature to extract styles and check the corresponding styles checkboxes.

Both schemes might be extended in the future.

This feature also introduces various code optimisations such as extraction of Performance and Steps enums as well as centralisation of values for ControlNet count and LoRA count.

Gradio (setting in Developer Debug Mode)

Default is Fooocus Scheme
image

Config options 

"default_save_metadata_to_images": true,
"default_metadata_scheme": "a1111",
"metadata_created_by": "mashb1t"

Arg --disable-metadata

--disable-metadata completely prevents metadata processing and output in Gradio

Metadata Reader

  1. open Image Input > Metadata tab
  2. drag & Drop image to image upload
  3. automatic preview of image metadata
  4. apply metadata to Gradio inputs on button click

Fooocus scheme
Screenshot 2024-01-29 at 15 13 17

A1111 scheme
Screenshot 2024-01-29 at 15 09 52

Metadata in files

Speed Fooocus scheme
image

LCM A1111 scheme (yes, with negative prompt, because it technically exists but doesn't have an influence)
image

Speed A1111 scheme
image

Civitai

Speed Fooocus scheme
image

LCM A1111 scheme
image

image

Speed A1111 scheme
image

image

@mashb1t
feat: add metadata logging for images
c5a15c7 
inspired by https://github.com/MoonRide303/Fooocus-MRE
@mashb1t
feat: add config and checkbox for save_metadata_to_images
8d56318
@mashb1t
feat: add argument disable_metadata
493e484
@mashb1t
feat: add support for A1111 metadata schema
191f814 
https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/cf2772fab0af5573da775e7437e6acdca424f26e/modules/processing.py#L672
@mashb1t
feat: add model hash support for a1111
f7489cc
@mashb1t
feat: use resolved prompts with included expansion and styles for a11…
1a52367 
…11 metadata
@mashb1t
fix: code cleanup and resolved prompt fixes
6662381
@mashb1t
feat: add config metadata_created_by
7b9deb1
@mashb1t
fix: use stting isntead of quote wrap for A1111 created_by
cd65f21
@mashb1t
fix: correctlyy hide/show metadata schema on app start
a2153db
@mashb1t
fix: do not generate hashes when arg --disable-metadata is used
80ad0d0
@mashb1t
refactor: rename metadata_schema to metadata_scheme
ba5d0b6
This was referenced Jan 15, 2024
Closed
Closed
@lllyasviel
Copy link
Owner

will take a look late Jan / early Feb

This was referenced Jan 17, 2024
Closed
Closed
@mashb1t mashb1t linked an issue Jan 21, 2024 that may be closed by this pull request
Please add additional feature for saving image #1996
Closed
@Eleksar387
Copy link

I really like this implementation. I haven't using Fooocus anymore just because I do want my images to have the generation data included. However it doesn't seem to work for me. I merged it and also checked the checkbox then but neither json or automatic1111 seems to work. Still no image data. No idea what I am doing wrong.

@mashb1t mashb1t mentioned this pull request Jan 24, 2024
Merged
@mashb1t
Copy link
Collaborator Author

mashb1t commented Jan 24, 2024 •
edited
Loading

@Eleksar387 Metadata will only be added to images in the output folder (not in browser) until #1932 is merged. Please check if images in the output folder have the metadata format you've selected.

@Eleksar387
Copy link

@mashb1t Understood. That's what I did. As I see the additonal checkbox in the UI I assumed I've merged it correctly. I use Diffusion Toolkit as an image viewer that also displays prompt data. If I check the add metadata button in Fooocus the images do not even appear in this image browser. They do appear if I uncheck it.
I am not saying it's not working. Maybe I am doing something wrong. It would be phantastic if this would be added to Fooocus. So it's the users choice.

@mashb1t
Copy link
Collaborator Author

mashb1t commented Jan 25, 2024 •
edited
Loading

@Eleksar387 the default fooocus metadata format i used is probably not compatible with anything else as it's just a json string in the pnginfo. You might have to use the A1111 format to establish compatibility to their scanner, see https://github.com/RupertAvery/DiffusionToolkit/blob/master/Diffusion.Scanner/MetadataScanner.cs.
I am not using and have never used Diffusion Toolkit, but the A1111 metadata scheme is compatible with all A1111 metadata readers as it uses the exact same format.
Please open the png in a text editor and check if metadata is present (in the first few lines of binary code) and let me know which scheme you used and if it works when selecting A1111.

@Eleksar387
Copy link

@mashb1t I just checked it again. If I tick "Save Metadata to Images > Automatic1111" the images don't even appear in Diffusion Toolkit. Same if I tick "Fooocus (json)". Although in both cases the images are created and in the folder. But no txt file. If I uncheck "Save Metadata to Images" the images do appear in Diffusion Tookit - but of course without metadata.

@mashb1t
Copy link
Collaborator Author

mashb1t commented Jan 25, 2024

Seems like an issue for their repository, at least they do not display then images with different metadata.

Please open the png in a text editor and check if metadata is present (in the first few lines of binary code)

Please confirm the metadata exists in the file.

@Eleksar387
Copy link

You are right! Metadata does exist. Sorry for bothering you.

@Eleksar387 Eleksar387 mentioned this pull request Jan 25, 2024
Open
@mashb1t
Copy link
Collaborator Author

mashb1t commented Feb 4, 2024

Would it make more sense to map to the closest available instead ?

I've re-evaluated the impact and it is minor if not not noticable (source of noise pattern generation cpu vs gpu), so the mapping has been adjusted. See also https://www.reddit.com/r/comfyui/comments/15hayzo/comment/juqcpep/

@mashb1t mashb1t mentioned this pull request Feb 4, 2024
Merged
@emden09g
Copy link

emden09g commented Feb 7, 2024

Love the #1940 Metadata enhancement. Though I would love it even more if the metadata was not only written as a comment (which is fairly userfull as I as I would like to emphasize) but as EXIF-Datafields (which are realy usefull for search and filtering in any photoediting/-organisation software like Lightroom etc.

@mashb1t mashb1t added the Size L large change, may depend on other changes, thoroughly test label Feb 9, 2024
@mashb1t mashb1t mentioned this pull request Feb 22, 2024
Closed
@eddyizm
Copy link
Contributor

eddyizm commented Feb 24, 2024

Is this ready to pull in to our local fork?

@mashb1t mashb1t added this to the 2.2.0 milestone Feb 24, 2024
@mashb1t
Copy link
Collaborator Author

mashb1t commented Feb 24, 2024 •
edited
Loading

@eddyizm i'm planning to include this feature in the next release 2.2.0 tomorrow/Monday, incl. jpg and webp support. Feel free to review.

@eddyizm
Copy link
Contributor

eddyizm commented Feb 24, 2024

Nice! I pulled it down to check it out and see where the merge conflicts are at. Not too much work.
Looking forward for all the new stuff!

@mashb1t mashb1t mentioned this pull request Feb 25, 2024
Closed
@mashb1t mashb1t linked an issue Feb 25, 2024 that may be closed by this pull request
Load parameters from the clipboard does not reset unused LoRAs #2354
Closed
@mashb1t
Merge branch 'develop' into feature/add-metadata-to-files
0b77db9 
# Conflicts:
#	modules/async_worker.py
#	modules/config.py
#	modules/meta_parser.py
#	modules/private_logger.py
#	modules/util.py
#	webui.py
@mashb1t
fix: handle empty strings in metadata
65dae45 
e.g. raw negative prompt when none is set
@mashb1t mashb1t changed the base branch from main to develop February 26, 2024 13:26
@mashb1t mashb1t merged commit ba9eadb into lllyasviel:develop Feb 26, 2024
@VictorZakharov
Copy link

@mashb1t Merge to main when ?

@mashb1t
Copy link
Collaborator Author

mashb1t commented Feb 26, 2024

@VictorZakharov when all ToDo's in https://github.com/lllyasviel/Fooocus/milestone/1 are done and the release candidate has been tested. Feel free to already test develop!

@Benm678
Copy link

Benm678 commented Mar 3, 2024

Hi, not sure if this is the right place to comment/report? When using the metadata feature, the metadata is indeed embedded and can be read using alternative readers - but the history log still reports "All images are clean, without any hidden data/meta, and safe to share with others."
image

image

Obviously as the user I know that I clicked the metadata option in the advanced tab, but the log shouldn't report that.

@mashb1t
Copy link
Collaborator Author

mashb1t commented Mar 3, 2024

@Benm678 thanks for the hint. I've already considered removing this line in the log header, but wanted to get some feedback first. As there was no feedback from lllyasviel in the last 2-3 weeks i've decided to keep the line and basically wait for feedback of the community.
The log is only appended to => no dynamic outoput of this line is possible => enabling metadata saving later after initial log creation makes modification of this line in header impossible.
We could integrate JS to find out if one of the images features metadata by a hidden parameter in the log per image, or simply remove the line altogether, which currently is the consideration.
=> planning to remove the line in the next release, but open for feedback!

@mashb1t mashb1t mentioned this pull request Mar 3, 2024
Merged
@mashb1t
Copy link
Collaborator Author

mashb1t commented Mar 3, 2024

@Benm678 => see #2425

@VictorZakharov
Copy link

@mashb1t I am checking 2.2.0, how do I enter loras? That whole section is gone.
image

@mashb1t
Copy link
Collaborator Author

mashb1t commented Mar 3, 2024

@VictorZakharov this is off-topic, please open a new discussion/issue if needed.
Check if your config has default_loras or default_max_lora_number is set.

@VictorZakharov VictorZakharov mentioned this pull request Mar 3, 2024
Closed
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@lllyasviel lllyasviel Awaiting requested review from lllyasviel lllyasviel is a code owner

Assignees
No one assigned
Labels
Size L large change, may depend on other changes, thoroughly test
Projects
None yet
Milestone
2.2.0
7 participants
@mashb1t @lllyasviel @Eleksar387 @VictorZakharov @emden09g @eddyizm @Benm678