Description
Intro
Hi guys! We have a huge microfront SPA with hundreds of scripts. There is a custom loading system and it uses fetch + eval. And now we want to migrate to SystemJS with AMD for backward compatibility.
I've tried following scenario:
- Loading each microservice with System.import().
- Using
define(['sharedDep1'], function(sharedDep1){})
inside js files - And using
define('sharedDep1', function(){})
for shared dependencies in a core script.
And then I got a very strange bug. Sometimes some of modules export a wrong value, as if the different modules are messed up.
Something like that:
System.import('moduleA').then(console.log) // And there is a moduleB in the console
After some investigation I figured out that it's because of a race condition. So it's very hard to reproduce and I've got it just because I have a huge SPA.
Here's what's happening:
System.import()
adds a script into the document- It sets onload event listener that will call
getRegister
after loading - Another script finished its loading and that caused my code with amd
define
to be called define
is working synchronous, it sets firstName and firstNamedDefine- It sets
setTimeout
that clear them in the end - Previous script is loaded and it fires onload event, then
getRegister
will be called andif (firstName && url) {
is true
After that for some reason systemjs will use one module instead of the other.
I think the problem in this PR #2352
I guess there is a race condition between eventloop macrotasks: setTimeout and onload event.
Demonstration
Because of the race condition it's hard to reproduce apart of our code. So I wrote a synthetic example with multiple fake requests.
I made a loop with random choice of import or define. Check it out:
Code Sandbox: https://codesandbox.io/s/modest-chandrasekhar-lb7z6?file=/index.html
Expected Behavior
define
call doesn't affect on System.import()
Every import returns its own module.
In my example it will be message We couldn't find the bug
in the console
Actual Behavior
If we call define
at the wrong moment, some modules will be mixed up.
In my example it will be message expect module b: {a: "a"} We've finally found the bug!
in the console
Activity
bmpolonsky commentedon Sep 29, 2021
My first clue was this.
I noticed that the modules have aliases of completely different services
bmpolonsky commentedon Sep 29, 2021
The same example but with the previous version of systemjs (6.10.2):
https://codesandbox.io/s/dawn-dawn-hevnx?file=/index.html
jolyndenning commentedon Oct 4, 2021
The named-register extra (used to process
define('name', ...)
has always had race conditions in every version. I've fixed more than five in the last two years, and added tests, but it's quite possible there are still race conditions. The change in 6.10.3 is was very related to it, reworking much of how named registers work. See #2352 for more detail.Thanks for providing the code sandbox, I'll look into it and see if I can identify the issue.
jolyndenning commentedon Oct 4, 2021
For some background info, the named-register extra has no way of differentiating a
define()
that occurs as a result from System.import vs one that does not. The job of the named-register extra is to implement theSystem.getRegister()
function so that it returns the first named register within a module loaded via System.import. It is supposed to ignore named registers that didn't originate from System.import(), and also is supposed to be concurrent safe for multiple System.imports that occur simultaneously.The way that named-register implements this is by storing the
firstNamedDefine
variable. When thefirstNamedDefine
variable exists, it is returned fromgetRegister
. The firstNamedDefine is set synchronously at the moment thatdefine
is called, and then is cleared in the next tick of the event loop (via asetTimeout
) or when getRegister is called. See code below for details:systemjs/src/extras/named-register.js
Lines 37 to 40 in 98609db
The bug in your code sandbox occurs when
define('a', ...)
occurs in the same tick of the event loop as thedefine('b', ...)
. In that case, the setTimeout to clear the firstNamedDefine has not executed yet, which results in the bug. A workaround is to callgetRegister()
right after thedefine('a', ...)
, which forcibly clears thefirstNamedDefine
variable. See https://codesandbox.io/s/competent-wright-kt6ur?file=/index.html which shows that.That setTimeout is the root of all of the race conditions. We started it as setTimeout, switched it to Promise.resolve, switched it back to setTimeout, and have reworked the named-register various times to try to avoid race conditions. However, we do not know of a way to implement the namedRegister without that
firstNamedDefine
variable, which is just inherently not a great approach. The ideal would be to clear it right before executing the code for a module loaded with System.import, then call getRegister to clear it immediately after the code is executed. The majority of SystemJS loading is done with<script>
elements since they're more performant than fetch + eval, and with script loading we callgetRegister()
during the load event. However, I do not think there is abeforeload
event that is usable for script tags. For fetch + eval, you could just callgetRegister()
right beforeeval()
in order to first clear the firstNamedDefine variable before executing the code.If you have ideas about how this could implemented in a general way that works with script loading (not just fetch + eval), that would be very helpful. I think I've exhausted my ideas on the topic and don't know of any new things that could attempt to solve the problem. I'll look over your code sandbox a bit more and see if anything comes to mind, but do not know how it could be fixed.
jolyndenning commentedon Oct 4, 2021
I personally do not use the named-register for the reasons described above, preferring just to use anonymous modules which do not have these race conditions.
bmpolonsky commentedon Oct 8, 2021
Thank you for your reply! I am on vacation now, but I will return to this problem soon. Btw I don't understand why you changed the promise to the settimout. Isn't microtask better in this case?
jolyndenning commentedon Oct 12, 2021
Yeah Promise.resolve() is better, and I didn't switch from Promise.resolve to setTimeout intentionally in #2352. I'm not sure if Promise.resolve will completely solve the race condition, but am willing to try it out. I think the problem is occurring when the
<script>
appended by System.import() executes in the same microtick of the event loop as the manual call todefine()
in your code sandbox. So adding a microtick after that microtick might still be too late. However, I'm not 100% sure if that's the case or if that's even possible with the browser's implementaiton of script tags.In testing your code sandbox with Promise.resolve(), it seems to have fixed it, so maybe it is a microtick that's the difference. I'll send in a pull request with it.
Use Promise.resolve in named-register. Resolves #2359.
jolyndenning commentedon Oct 12, 2021
I've created #2363 with the fix, switching to Promise.resolve
bmpolonsky commentedon Oct 18, 2021
Thank you! By the way I'm not sure we can write tests for this case. What do you think?
bmpolonsky commentedon Oct 18, 2021
Or can we use tha sandbox example somehow?
And I've noticed that there is no need for
Promise.resolve
arounddefine
in my sandbox.Just a "synchronous"
define
insidesetInterval
is enough.https://codesandbox.io/s/keen-firefly-0bomu
jolyndenning commentedon Oct 18, 2021
Fix released in https://github.com/systemjs/systemjs/releases/tag/6.11.0