Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Importing wand takes 700ms on Ubuntu (and can't be avoided via environment) #556

Open
craigds opened this issue Sep 8, 2021 · 4 comments

Comments

@craigds
Copy link

craigds commented Sep 8, 2021

In Ubuntu 18.04 LTS, the libmagickwand-6.q16-3 package installs the wand libraries at:

  • /usr/lib/x86_64-linux-gnu/libMagickCore-6.Q16.so.3
  • /usr/lib/x86_64-linux-gnu/libMagickWand-6.Q16.so.3

import wand finds these, but it takes ~700ms to do because it runs ctypes.find_library (apparently several times?)

Looking at the source code, it seemed like we could avoid this by setting MAGICK_HOME and WAND_MAGICK_LIBRARY_SUFFIX in our containers.

However, this doesn't actually help because wand hardcodes the .so extension, and .so.3 doesn't match.

Perhaps wand should handle a couple of environment variables containing the full paths to the libraries?

@emcconville
Copy link
Owner

I'm open to suggestions. Try NOT setting MAGICK_HOME, but just WAND_MAGICK_LIBRARY_SUFFIX=-6.Q16 value.

@craigds
Copy link
Author

craigds commented Sep 8, 2021

While checking that I found it interesting to note that the options doesn't seem to be required in our case. If I omit both environment variables and let wand run find_library over all the various combinations I get this:

>>> for suffix in suffixes:
...     libwand = find_library('MagickWand' + suffix)
...     print(f"{suffix:13s} : {libwand}")
...
...
...
              : None
HDRI          : None
HDRI-2        : None
-7            : None
-7HDRI        : None
-7HDRI-2      : None
-7.Q8         : None
-7.Q8HDRI     : None
-7.Q8HDRI-2   : None
-7.Q16        : None
-7.Q16HDRI    : None
-7.Q16HDRI-2  : None
-6            : libMagickWand-6.Q16.so.3
-6HDRI        : None
-6HDRI-2      : None
-Q16          : None
-Q16HDRI      : None
-Q16HDRI-2    : None
-Q8           : None
-Q8HDRI       : None
-Q8HDRI-2     : None
-6.Q16        : libMagickWand-6.Q16.so.3
-6.Q16HDRI    : None
-6.Q16HDRI-2  : None

It was surprising to me that the library is found while checking the -6 suffix even though it has the Q16 option tacked on to the library name. Perhaps the options are unnecessary? I wouldn't presume to know whether it is necessary on other systems though.

So the issue for us is that find_library is called 13 times before it finds a result. As you suggested, switching to setting the suffix but not MAGICK_HOME would solve this for us - find_library would run once and find the library immediately.

@emcconville
Copy link
Owner

emcconville commented Sep 9, 2021

It was surprising to me that the library is found while checking the -6 suffix even though it has the Q16 option tacked on to the library name.

That's ABI working at its best.

So the issue for us is that find_library is called 13 times before it finds a result.

I good time to consider migrating to ImageMagick-7. If performance is an important feature, I highly recommend ImageMagick-7's Q8 without HDRI. I know it's crummy to hear that, but Wand must exhaust ImageMagick-7 & support for High Dynamic Range images before attempting ImageMagick-6.

All that said, I still believe a "short-cut" environment variable is a good idea.

@craigds
Copy link
Author

craigds commented Sep 9, 2021

Imagemagick 7.x hasn't made it into an ubuntu LTS any ubuntu releases yet. Not sure why, seems like it's been around for years (?)

Looks like progress is being made though so maybe 22.04 will get it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants