globalThis is weird in Firefox...

Okay we're up to 666/735 core node tests passing and 7 I've xfailed because they use many browser-only APIs. Of the 68 remaining failures, 60 are cascading failures due to timeouts in test_core_python. After a timeout my node driver should probably be refreshed. Or maybe the communication protocol can somehow be fixed up to be able to clean up old garbage in stdout. Might require some work though. Maybe I can raise the default timeout in the short term.

Also, the node tests are REALLY slow. I'm not sure why they are so slow tbh, but it's actually about similar to what happened on my libffi port. There, node tests take 15x longer (sometimes even more), here they only take 6x longer than the selenuim tests.

due to a 50ms sleep by default after each send command and you are sending line by line

Hmm. I guess I can try sending all at once.

I think we would need to figure out how to make the test-core[node] faster.

I am working on embedding node into a Python extension so we can avoid this pexpect nonsense. It is working decently so far (I can run_js a Javascript code string and get back the return value). The biggest problem with this approach is that node doesn't seem to ship libnode.so or the other shared libraries from a build. It took around three hours to build node on my local machine. I am just building node in the most standard configuration though:

./configure --shared
make -j4

so I think it would be okay to stash libnode.so somewhere if we can't find someone who has already done this.

Hmm. I guess I can try sending all at once.

In the above SO link, there was also the option of setting that sleep value to 0 which should help.

so I think it would be okay to stash libnode.so somewhere if we can't find someone who has already done this.

Yeah, we can't reasonably ask people to build node from sources. I think shipping a .so also likely wouldn't work as that's not portable: it won't work for Linux/MacOS users who are trying to build outside of Docker.

Maybe profiling a typical pexpect session to understand what's going on would help?

we can't reasonably ask people to build node from sources

Well this isn't a key aspect of the project, I think we could just tell people that to run node tests they should either use docker or acquire a libnode.so that works on their system?

Apparently electron uses libnode.so so maybe we can find it in electron distros?

True, but it would overall make contributing more complex if say one of the node tests failed and one need to reproduce. Also even to include that .so into Docker doesn't sound straightforward. I would rather not compile node even for building Docker images.

My point is that it would be good to first try to understand why pexpect is slow. Serializing strings for input/output shouldn't be that much overhead.

Yeah "embed node into a python module" is a bit in the fun but logistically impractical direction. Also, it's pretty incredible how much better the docs for the Python C api are than for the Node C++ API. The Node / v8 API seems significantly more complicated but the docs are close to nonexistent.

Apparently electron uses libnode.so so maybe we can find it in electron distros?

I'm not too familiar with electron distros, but I agree that one of the challenges we have is really to keep the tools we use as simple as possible. It's already and a very complex build setup, so the more additional tools we use the more difficult it is for new contributors to approach it (and also for us to have solid enough expertise in those tools).

Here's the discussion where people said it would be good to ship an so (to avoid forcing embedders to build it):
nodejs/node#24028

There is also metacall but documentation is very thin.

It looks like electron links libnode.a so doesn't distribute it.

Yeah hopefully we can make pexpect work, turns out embedding node is a huge can of worms.

Looks like delaybeforesend=None did the trick, now node tests run the fastest of the three.

Looks like we already xfailed the pillow test on chrome and firefox. I think this is all ready now.

@rth Tests all pass. I think this is ready to merge, unless you have further comments.

Thanks for working on this @hoodmane !