Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Deskew method errors out when running in docker(linux) #563

Open
uday-felix opened this issue Feb 4, 2022 · 4 comments
Open

Deskew method errors out when running in docker(linux) #563

uday-felix opened this issue Feb 4, 2022 · 4 comments
Labels
seeking-help

Comments

@uday-felix
Copy link

uday-felix commented Feb 4, 2022 •
edited
Loading

I am ending up with an error when I use the below code when its run in docker.

try:
     with skew(blob=img_png) as img_png_1:           
         img_png_1.deskew(0.5 * img_png_1.quantum_range)
         pil_image = Image.open(io.BytesIO(img_png_1.make_blob("png")))            
         img_png = image_to_byte_array(pil_image)            
     log.info("Corrected the Skewness for %s", file_name)
 except Exception as e:
     log.exception(f'Exception:{e} occured while Corrected the Skewness for {file_name}')
     pass

Error Message: wand.exceptions.WandRuntimeError: MagickReadImage returns false, but did not raise ImageMagick exception. This can occur when a delegate is missing, or returns EXIT_SUCCESS without generating a raster.

When I run this in local windows, I don't see any error, but failed in docker(linux). It keeps working for few images and fails for few. Please advice.
@emcconville
Copy link
Owner

The error is generic, and occurs when the decoder delegate is unable to read the image. Usually occurs when there's not enough resources to continue operations. Looking at your code, you maybe able to reduce the memory usage. Try removing everything but deskew, or write blob data to disk to save resources.

@uday-felix
Copy link
Author

Thanks @emcconville . Also I have edited the policy.xml file which has improved.

@uday-felix
Copy link
Author

uday-felix commented Mar 9, 2022 •
edited
Loading

@emcconville Am still facing the issue after increasing the values in policy.xml.
As am using this is production, I have 1000 + files going through the above code in multi thread at the same time and this kicks in the error.
Can you suggest something ?

@edit:

I have changed my code, removed this part of code from the above pil_image = Image.open(io.BytesIO(img_png_1.make_blob("png")))

after the try block I have a code to save the image which was converted to bytes.

@emcconville
Copy link
Owner

Can you suggest something ?

Look into queue-based task solution. AWS's SQS/lambda or Celery, but there are hundreds of other distributed messaging systems.

I have 1000 + files going through the above code in multi thread at the same time and this kicks in the error.

That's probably the issue. With a queued system, you can just add more workers, or flag/reject tasks that fail.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
seeking-help
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@emcconville @uday-felix