Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix overwriting existing values in config.json in auto #418

Merged
merged 8 commits into from
May 6, 2023
Merged

Fix overwriting existing values in config.json in auto #418

merged 8 commits into from
May 6, 2023

Conversation

wookiefriseur
Copy link
Contributor

Woops, sorry. Renamed the branch and it was auto closed. Here we go again.


jq merge direction in this case is right to left so if the user had set up custom paths it would replace them with the default ones.

This PR switches the direction to use the defaults as fallback instead of overwriting user settings.


Didn't want to create an issue for the tiny change.

Thanks for your work on the repo, it saved me a lot of time, 👍

`jq` merge direction in this case is right to left so if the user had set up custom paths it would replace them with the default ones. 

This PR switches the direction to use the defaults as fallback instead of overwriting user settings.
@AbdBarho
Copy link
Owner

My fear is that users will go into the settings, change the paths to something else which is not mounted, and then complain that they can't find the outputs outside of the container.

Do you have any ideas for alternatives?

@wookiefriseur
Copy link
Contributor Author

wookiefriseur commented Apr 23, 2023

Oh, so that was on purpose. I would argue that this makes it more tedious for regular use on a PC as the container is not up 24/7. So on every run or build the directory settings are wiped.

I think users that would not know how to fix a problem with the output directory would not change any settings in the first place. But ok, I can see how some users might omit the /output/ prefix and then wonder where the files are. Might happen if the user and the admin are different people (for example 1 tech savvy person setting it up for another one). If that happens, the container has to be restarted by the admin anyways though, entrypoint won't save them without a restart.


Ideas

OK, let's brainstorm:

  • IF there is an entry in the user config that has a corresponding entry in the default config,
    • AND the entry starts with "outdir_",
    • AND the value of this entry is not empty
    • AND it does not match the valid directory pattern,
  • THEN replace it with the default value.
# Step 1: Copy default config to user config if missing
cp -n /docker/config.json /data/config/auto/config.json

# Step 2: Put the default config and user config into vars
cfg_json_default=$(cat /docker/config.json)
cfg_json_user=$(cat /data/config/auto/config.json)

# Step 3: Fill existing user config with default entries, if missing
cfg_json_merged=$(echo $cfg_json_default $cfg_json_user | jq '. * input')

# Step 4: Define valid directory pattern 
# (must start with /output/, can contain words, numbers, underscores, dashes, slashes, single dots, no traversal)
pattern_valid_path='^\/output(\.)?(\/\.?[\w\-\_]+)+\/?'

# Step 5: replace entries in the user config if they have matching keys in the default config 
# and the directory pattern for the value does not match (not a valid directory)
echo "imagine some very smart code here"

# Step ???: save it
echo $cfg_json_merged  | sponge /data/config/auto/config.json

I'm sure some of those actions can be combined into 1, but since it's working on small strings instead of small files it shouldn't slow down the start.

I hate shell scripting, but I can try to write something if you are fine with a solution like that. I'd have to look into jq more but it is definitely possible.

@AbdBarho
Copy link
Owner

Thank you, it does not have to be bash. We can have a python script that runs on startup and checks if the path starts with /output/, if not, overwrite.

@wookiefriseur
Copy link
Contributor Author

wookiefriseur commented Apr 24, 2023

Oh yeah, Python would have been easier but at least that way I learned some jq basics. I tested the changes on my machine. Simulating with strings as well as using it in the live docker build. Works so far.

I added 2 commits to the PR. Let me know if you want any changes.

Stuff I am not checking for:

1.) Manually edited config.jsons with invalid JSON, because that wouldn't happen when using the webui form

2.) Empty values for all paths, because with no paths the user is likely to see by themselves where the problem is. And if they then enter a path like C:\mystuff it will be reset by entypoint.sh at the next run. So all fine.

Copy link
Owner

@AbdBarho AbdBarho left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, sorry for delayed response.

services/AUTOMATIC1111/config.json Outdated Show resolved Hide resolved
services/AUTOMATIC1111/entrypoint.sh Outdated Show resolved Hide resolved
@wookiefriseur
Copy link
Contributor Author

Ok, finished. Works in manual tests and with the container build process. Feel free to change the PR as you see fit. Let me know if you require bigger changes.


For clarification, just so that we don't misunderstand each other:

Input:

{
  "outdir_samples": "/output/samples_custom",
  "outdir_grids": "../break/stuff",
  "outdir_txt2img_samples": "/output/txt2img_custom",
  "outdir_img2img_samples": "C:\\customdir\\",
  "outdir_extras_samples": "/output/extras_custom",
  "outdir_txt2img_grids": "/output/txt2img-grid",
  "custom_key": "my_custom_value",
  "outdir_save": "C:/customdir"
}

Output:

{
  "outdir_samples": "/output/samples_custom",
  "outdir_grids": "",
  "outdir_txt2img_samples": "/output/txt2img_custom",
  "outdir_img2img_samples": "/output/img2img",
  "outdir_extras_samples": "/output/extras_custom",
  "outdir_txt2img_grids": "/output/txt2img-grid",
  "custom_key": "my_custom_value",
  "outdir_save": "/output/saved",
  "outdir_img2img_grids": "/output/img2img-grids"
  "outdir_init_images": "/output/init-images",
  "font": "DejaVuSans.ttf"
}

PS: I also sneaked in some gitignore changes, I hope you won't notice 😄. I get generated files in data, that are mistakenly being tracked, as well as VSCode workspace settings. This change fixes that.

Copy link
Owner

@AbdBarho AbdBarho left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks

@AbdBarho AbdBarho merged commit 9ac33db into AbdBarho:master May 6, 2023
@wookiefriseur wookiefriseur deleted the auto/fix/config.json-overwrite branch May 7, 2023 21:12
Jordan-Lambda pushed a commit to Jordan-Lambda/lambda-cloud-stable-diffusion-2.0-webui-easy that referenced this pull request Aug 2, 2023
`jq` merge direction in this case is right to left so if the user had
set up custom paths it would replace them with the default ones.

This PR switches the direction to use the defaults as fallback instead
of overwriting user settings.

----
Didn't want to create an issue for the tiny change.

Thanks for your work on the repo, it saved me a lot of time, 👍

---------

Co-authored-by: AbdBarho <ka70911@gmail.com>
cloudaxes pushed a commit to cloudaxes/stable-diffusion-webui-docker that referenced this pull request Sep 6, 2023
`jq` merge direction in this case is right to left so if the user had
set up custom paths it would replace them with the default ones.

This PR switches the direction to use the defaults as fallback instead
of overwriting user settings.

----
Didn't want to create an issue for the tiny change.

Thanks for your work on the repo, it saved me a lot of time, 👍

---------

Co-authored-by: AbdBarho <ka70911@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants