Memory Leaks in Termux #1804
Comments
In a bunch of error cases of bigint ops we weren't always cleaning up all the variables. These were noticed when investigating MoarVM#1804, but I don't think they are the actual cause.
|
I believe the leaks reported by asan aren't "real". MoarVM does not cleanup all it's memory when exiting unless passed the How much ram do you have? zef might just use quite a bit, especially if it's the first time it's run and it has to precompile a bunch of stuff, but @ugexe would be the person to ask. |
|
Zef installs things in parallel for the speedz, but that may not work well if you're quite memory restricted. However, the behavior is configurable:
I suggest re-trying with |
In a bunch of error cases of bigint ops we weren't always cleaning up all the variables. These were noticed when investigating #1804, but I don't think they are the actual cause.
The default test degree for zef is 1. The default for fetch degree is 5, although that is unlikely to be related to the issue here since
zef does search in parallel, although in a per-ecosystem way, i.e. zef searching by default in the worst case would search 2 ecosystems at the same time. However that doesn't seem connected to "MASSIVE forking / threading". It might be worth running zef with Another thing zef spawns threads for is to run each phase (e.g. Speaking of spawning processes, the primary way zef spawns threads is indirectly via spawning processes. For example the default for fetching basically does
Precompilation would occur during the staging phase, which happens immediately before testing. Precompilation does spawn processes, but by the time testing occurs that would already be finished i.e. the staging repository has all the precompiled files before (and regardless of if) we start to run any tests. That doesn't mean re-precompilation isn't occuring for some unknown reason, but theoretically it shouldn't be. To your point the first time zef is run to install itself it would have to precompile a bunch of stuff (itself) before it is able to stage (and thus re-precompile) itself to be installed. |
|
Thank you all for your answers! I'm running Termux on a Google Pixel 6A with 6GB of RAM. After installing zef sucessfully with --/test, I succeded to install with it locally two dozen Modules after downloading the source manually. Building and installing seems therefore no problem. However if I try to install some module directly, it only prints "Searching for module" (and at the most "Searching for missing dependency", with and without --debug). After that, it fails either with "Killed" or "Segmentation fault" after ca. 10 seconds or with Termux crashing after ca. 30 seconds. But there is an exception: Modules that come without any depenencies (packages with no missing dependencies keep failing) can be installed normally with no issues. I also ran After that, Termux crashed (again, memory usage was about 3GB) Thank you all for your detailed and quick answers, your help is much appreciated! |
|
@nitardus I would expect to see more newlines than what you've shown - for example |
|
Yes, it does, I made some simple mistakes in the formatting of my post yesterday when I tried to insert the copied output of the deceased termux session into the browser (these things are a bit cumbersome on the phone). I fixed this now. I had however added the last two subtest results manually, since I saw them on screen before termux blew up, but was not able to copy them to the clipboard in time (redirecting the test output to a file did fail due to buffering). I tried it again today and surprisingly this time zef died with a simple error. Here is the log of the run: I ran it immediately afterwards about 20 times: Nearly always it crashed at the same spot as yesterday (that is, after subtest 18 finished, it took around 30 seconds for subtest 19 and 20 to finish, and then after some other 10 seconds it crashes). |
|
Hmm, that specific test can be seen here. It is not particularly heavy weight, and even uses mocked repository objects that only have a few names available to search. The candidates function does the heavy lifting for that test. I'm not sure how to debug further beyond adding debug print statements at various points in that function to hopefully pinpoint the specific lines causing this. For example: add |
After installing Moar, npq and rakudo as shown here, termux crashed when I tried to install zef. It finally succeded installing zef with --/test, but it now crashes when I try to install anything: rakudo-gdb-m reveals that while zef searches for packages, there is MASSIVE forking / threading going on in the background, until it runs out of memory at ~ 3 GiB memory usage. I recompiled moar with all sort of compiler options, and after using -asan, compilation of rakudo failed as follows:
Any ideas what to do? As I develop mainly on Android, it would be really nice if I could get raku working here...
The text was updated successfully, but these errors were encountered: