Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OPENGL scheme is causing a memory leak #21

Closed
KingArmstring opened this issue Dec 24, 2022 · 15 comments
Closed

OPENGL scheme is causing a memory leak #21

KingArmstring opened this issue Dec 24, 2022 · 15 comments

Comments

@KingArmstring
Copy link

First, thanks for that great work. Second, once I set the scheme to OPENGL and turn the app with the profiler and start sending camera frames(30 fps) to the api, it works fine but I can see the memory consumption in the profiler is getting increased to 100mb by 100 mb until it is 2.5 GB then the app crashes. For one image, it is ok but since I am working with a stream it is not ok, I was able to solve that by setting mode to NATIVE, I will try to solve that in the next days if I managed to find enough time and make a PR. Thanks agains :)

@HokoFly
Copy link
Owner

HokoFly commented Dec 25, 2022

Thanks for your feedback. I willI check and fix the memory leak if you don't have enough time. Also welcome your PR.

First, thanks for that great work. Second, once I set the scheme to OPENGL and turn the app with the profiler and start sending camera frames(30 fps) to the api, it works fine but I can see the memory consumption in the profiler is getting increased to 100mb by 100 mb until it is 2.5 GB then the app crashes. For one image, it is ok but since I am working with a stream it is not ok, I was able to solve that by setting mode to NATIVE, I will try to solve that in the next days if I managed to find enough time and make a PR. Thanks agains :)

@KingArmstring
Copy link
Author

I have spent a decent amount of time on trying figuring out why could it be leaking memory and from the memory dump(got it from the profiler) it figured out that it is leaking memory because of the bitmap being passed to the builder .blur(Bitmap) function, tried the async one but did not help, still leaking a huge amount of memory/sec the native scheme works fine tho. I made sure to recycle the bitmap that I pass after each passing but also did not help. Also the profiler says that the mem leak is happening in the native memory so this means that it is happening due an opengl bug and not a misuse of the lib or the Java/Kotlin interfacing. Unfortunately my experience with OpenGL is not great so I think I won't be able to fix it.
Screen Shot 2023-01-13 at 7 45 02 PM

@HokoFly
Copy link
Owner

HokoFly commented Feb 22, 2023

I have spent a decent amount of time on trying figuring out why could it be leaking memory and from the memory dump(got it from the profiler) it figured out that it is leaking memory because of the bitmap being passed to the builder .blur(Bitmap) function, tried the async one but did not help, still leaking a huge amount of memory/sec the native scheme works fine tho. I made sure to recycle the bitmap that I pass after each passing but also did not help. Also the profiler says that the mem leak is happening in the native memory so this means that it is happening due an opengl bug and not a misuse of the lib or the Java/Kotlin interfacing. Unfortunately my experience with OpenGL is not great so I think I won't be able to fix it. Screen Shot 2023-01-13 at 7 45 02 PM

I have fixed the memory leak. Use the latest version please.

   implementation 'io.github.hokofly:hoko-blur:1.3.8'

@KingArmstring
Copy link
Author

thanksssssssssss so much :D
I will definitely use it.

I having a problem if u can help and I willing to pay, I having trouble doing multipass rendering using framebuffer with an external GL_TEXTURE_EXTERNAL_OES, is it impossible to use framebuffer with this kind of texture, and if yes should I be able to use PBuffer instead knowing that I am having like 30 fps would this be harmful for performance? Thanks for the fix I will make sure to read the commit before using the new version.

@HokoFly
Copy link
Owner

HokoFly commented Feb 22, 2023

thanksssssssssss so much :D I will definitely use it.

I having a problem if u can help and I willing to pay, I having trouble doing multipass rendering using framebuffer with an external GL_TEXTURE_EXTERNAL_OES, is it impossible to use framebuffer with this kind of texture, and if yes should I be able to use PBuffer instead knowing that I am having like 30 fps would this be harmful for performance? Thanks for the fix I will make sure to read the commit before using the new version.

Sorry buddy. I don't know much about your opengl problem either. I may not be able to provide more help. 🙂

@KingArmstring
Copy link
Author

I have tried the new fix in 1.3.8 and the mem leak is still there.

@HokoFly
Copy link
Owner

HokoFly commented Mar 9, 2023

I have tried the new fix in 1.3.8 and the mem leak is still there.

Could you provide the leak case or demo ? Version 1.3.8 works in demo MultiBlurActivity

@KingArmstring
Copy link
Author

KingArmstring commented Mar 9, 2023

well, let me tell you how I get the memory leak, I get the camera analyser usecase of camera x and for every frame I turn it to a bitmap and deliver it to the lib using the code u wrote in the docs, if I set the scheme to NATIVE, no issue happens, but if I select opengl scheme, then it happens. I made sure to close everything and there is no issue when using NATIVE scheme, so it is not my demo. For blurring an image or 2 it won't matter but blurring every single camera frame is an excessive work that will help showing the mem leak. If that desc is not enough I will make a demo, let me know.

@KingArmstring
Copy link
Author

KingArmstring commented Mar 15, 2023

I think I know what the issue is, I think that you don't delete the framebuffers and textures associated with them at all, even if u collect that in an obj that is called cacheSomething and get rid of them at the end, keep in mind that a user like me uses the lib in a way that creates a new framebuffer like 30 times per sec, so u have to either create one framebuffer and reuse it and then kill it at the end(recommended because it will save the resources of creating a new framebuffer 30 times every sec) OR you will have to delete the framebuffer after every cycle(not recommended)

when I say delete the fbo and the tex I mean using
GLES20.glDeleteTextures(1, texIds, 0)
GLES20.glDeleteFramebuffers(1, fboIds, 0)
I am sure u already know that, just wanted to document my search results.

@HokoFly
Copy link
Owner

HokoFly commented Mar 16, 2023

I think I know what the issue is, I think that you don't delete the framebuffers and textures associated with them at all, even if u collect that in an obj that is called cacheSomething and get rid of them at the end, keep in mind that a user like me uses the lib in a way that creates a new framebuffer like 30 times per sec, so u have to either create one framebuffer and reuse it and then kill it at the end(recommended because it will save the resources of creating a new framebuffer 30 times every sec) OR you will have to delete the framebuffer after every cycle(not recommended)

when I say delete the fbo and the tex I mean using GLES20.glDeleteTextures(1, texIds, 0) GLES20.glDeleteFramebuffers(1, fboIds, 0) I am sure u already know that, just wanted to document my search results.

Oh, I did not check the high-frequency case. I have made a demo to reproduce the problem. I'll figure it out.

@HokoFly
Copy link
Owner

HokoFly commented Mar 17, 2023

I just fix the memory leak. @KingArmstring

   implementation 'io.github.hokofly:hoko-blur:1.4.0'

@KingArmstring
Copy link
Author

Nice I will test that, I have already made my implementation using framebuffers and multipass rendering and it works find but I will take a look anyway.

@KingArmstring
Copy link
Author

KingArmstring commented Mar 17, 2023

Hi @HokoFly I have just checked the fix and it really works well(I found another issue which I will open another ticket for if u want but later since this will need sometime to take some screenshots for and that was the reason I had to do the blurring through gaussian mats myself)

I have a very big problem and I have been really stuck in it for more than 2 months now in that issue which is using the same technique(multipass blurring or using framebuffer to render for example the horizontal blurring then render the vertical blurring to the main buffer) I can do the same beautiful blurring u r doing if the input is coming from a normal image(i.e. input texture is of type GL_TEXTURE_2D) but if the input is coming from the camera which is my case(i.e. input texture is of type GL_TEXTURE_EXTERNAL_OES) I can't make the intermediate frame buffer work. I am willing to pay enough for the time I will get help in this in even if not you(because you already told me you are busy) but I am really stuck so bad in this. So please either help me or recommend someone who can and as I said I will pay per hour and I will consider this to be a huge favour.

@HokoFly
Copy link
Owner

HokoFly commented Mar 18, 2023

@KingArmstring To be honest, I haven't encountered a similar problem with yours. You could provide more detail about the problem or some demos and I will support as much as possible. If you want to keep your project private, contact me by email.

@HokoFly HokoFly closed this as completed Mar 20, 2023
@KingArmstring
Copy link
Author

I have sent you a message on the mail, as I told you up above I found another issue in the blurring and I will create a ticket for it later next weekend or so as I am not very well at the moment(a bit sick). The issue I am talking about is kinda a distortion in the blurred image, I will share a screenshot of the distortion and a screenshot of my implementation of the blurring to show you what I mean.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants