Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WebP Save: add argument for using "auto-filter"/"smart_deblock" encoding feature. #3852

Merged
merged 1 commit into from
Feb 19, 2024

Conversation

goodusername123
Copy link
Contributor

@goodusername123 goodusername123 commented Feb 14, 2024

Adds the ability to configure whether or not the "auto-filter" feature from libwebp is used when encoding lossy WebPs, This is put behind a new optional argument/toggle called/named "smart_deblock" in libvips.
It's described by Google as doing the following: "This algorithm will spend additional time optimizing the filtering strength to reach a well-balanced quality."
Note this process is quite slow for the quality gains it achieves.

Also a warning I must give that I apologize for is I am not in a proper position to actually compile libvips and test it with this new code, however this all seems straight forward enough and anyone who is able to can confirm if this works by checking the outputs when changing the smart_deblock boolean.

If there's anything I missed or concerns about details in this pull request then feel free to say.

Adds the ability to configure whether or not the "auto-filter" feature from libwebp is used when encoding lossy WebPs.
This is put behind a new optional argument/toggle called/named "smart_deblock" in libvips.
@goodusername123 goodusername123 changed the title WebP Save: add toggle for using "auto-filter"/"smart_deblock" encoding feature. WebP Save: add argument for using "auto-filter"/"smart_deblock" encoding feature. Feb 14, 2024
@jcupitt
Copy link
Member

jcupitt commented Feb 15, 2024

Hi @goodusername123, thanks for this!

How about calling this switch autofilter? It would match the libwebp name and make it easier for users to understand what it did.

We should have a test as well, but I can add that in another commit.

The OS X CI test above isn't working, it looks like homebrew have broken something again, I'll fix that.

@goodusername123
Copy link
Contributor Author

@jcupitt

How about calling this switch autofilter? It would match the libwebp name and make it easier for users to understand what it did.

My original intention behind naming it "smart_deblock" was to try and keep consistency with how "sharp_yuv" is exposed as "smart_subsample", although I guess after looking back it seems the original reason for this being the case in the first place is because "sharp_yuv" in the past didn't have a proper name/switch and had to be activated via setting "preprocessing" to a value of 4 in older versions of libwebp back when libvips originally added support.

Another reason for choosing the name "smart_deblock" was because while the auto part of "autofilter" is somewhat self explaining I found the "filter" part to be a bit ambiguous since the words "filter" or "filtering" seem be a bit too general purpose and could mean a number of different things (for example in PNG filtering seems to refer to prediction modes) while "deblock"/"deblocking" I find is a lot more straightforward and easy to understand and in RFC 6386 (VP8 is what the lossy side of WebP is) the filtering process is described as being for deblocking purposes, see pages 12, 83, and the very end of 122.
Also I don't think a name like "auto_deblock" would work well either since "auto" could be at first glance interpreted as auto disabling or enabling something while "smart" like used in "smart_subsample" I think would get across more clearly that it's simply using more effort towards something that's already being done as a part of the encoding process.

With all this in mind I do think there's some merit to the name I chose for those who don't know specifics about libwebp or video codecs and such, However maybe there's something I'm not taking into account or missing.

@goodusername123
Copy link
Contributor Author

goodusername123 commented Feb 16, 2024

After thinking about it a bit more and considering "smart_subsample" as a edge case and such I would say that It's fine whatever you decide the final name should be so feel free to change it as I'm too indecisive of a person myself (I have "Allow edits by maintainers" so you should be able to change anything If I understand GitHub correctly).

Also as an aside some of the documentation for the parameters/arguments/switches that interact with libwebp are a bit vague or missing important information such as how both quality (exposed as Q in libvips) and method (exposed as effort in libvips) affect effort for the lossless encoder and specifying the max for both in conjunction (100 and 6 respectively) will activate the brute force "lossless cruncher", This is out of scope for this pull request but I might make another pull request at some point to expend on the documentation if that's fine.

@jcupitt jcupitt merged commit 72f0fc4 into libvips:master Feb 19, 2024
5 of 6 checks passed
@jcupitt
Copy link
Member

jcupitt commented Feb 19, 2024

I've been reflecting too, let's just stick with the name you picked.

Thanks for this! I'll add a test and a line in the changelog in another commit.

jcupitt added a commit that referenced this pull request Feb 19, 2024
jcupitt added a commit that referenced this pull request Feb 19, 2024
@jcupitt
Copy link
Member

jcupitt commented Feb 19, 2024

Ahem, now I test this, I can't get it to do anything, I've reverted this merge.

I tried various combinations of RGB and RGBA, lossy and lossless, but webpsave always made identical files, whether or not smart-deblock was on. I tried with cwebp -af and also couldn't get it to do anything (except be very slow heh).

Do you have a sample file where you expect to see a change?

@goodusername123
Copy link
Contributor Author

@jcupitt Sorry for the late reply.

Ahem, now I test this, I can't get it to do anything, I've reverted this merge.

I tried various combinations of RGB and RGBA, lossy and lossless, but webpsave always made identical files, whether or not smart-deblock was on. I tried with cwebp -af and also couldn't get it to do anything (except be very slow heh).

Do you have a sample file where you expect to see a change?

Yes I do have this sample file here (also includes test encodes both with and without Autofilter along with the CMD output from cwebp):
WebP Autofilter encode test_rar.zip

also here is a quick comparison image showcasing a part of the image with very noticeable differences:
autofilter compare webp

I assume a possible reason why you haven't gotten different outputs when testing on your end is due to how the Autofilter function doesn't change or modify the actual encoded blocks but rather changes the values that determine how the image is decoded meaning only a very small part the the final WebP file is different since in total the Autofilter can only change the values that determine the amount of filtering/deblocking done to each of the 4 segments that the VP8 spec allows, For example compare how in the cwebp CMD output the "filter level" values change quite a lot between the Autofilter's decision making being disabled or enabled and also this does depend on the source image used as well since depending on the input the Autofilter might determine that the default values are good and not change them.

@Commenter25
Copy link

Any chance to revert the revert? While it absolutely depends on the source image, autofilter is able to make a subtle but essentially free quality gain, only costing additional encode time; and is well worth having available, in the same way an effort range is. It also seems to have the most impact on smaller images, which happen to be the ones where encoding time is the least concern!

Since this was reverted due to a lack of example, I found an image1 with noticeable improvements. Despite sharing the exact same 11,250 byte size2, the image encoded with autofilter is noticeably less blocky, with a lot of weird sharp color changes removed. I've highlighted areas where it's more noticeable, but it does somewhat appear throughout the image.

Comparison image containing the original art, the art in WebP quality 70, and WebP 70 again with the autofilter option enabled

Footnotes

  1. art source: https://www.newgrounds.com/art/view/aestheticden/pancakes

  2. actual files: niko.zip

jcupitt added a commit that referenced this pull request Oct 6, 2024
jcupitt added a commit that referenced this pull request Oct 6, 2024
@jcupitt
Copy link
Member

jcupitt commented Oct 6, 2024

Sorry, this dropped off my radar. I've reverted the revert and this will be in 8.16.

Thanks for the test images. I've credited you in the changelog @goodusername123, I hope that's OK.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants