Skip to content

AMD named define works strangely #2359

Closed
@bmpolonsky

Description

@bmpolonsky

Intro

Hi guys! We have a huge microfront SPA with hundreds of scripts. There is a custom loading system and it uses fetch + eval. And now we want to migrate to SystemJS with AMD for backward compatibility.
I've tried following scenario:

  • Loading each microservice with System.import().
  • Using define(['sharedDep1'], function(sharedDep1){}) inside js files
  • And using define('sharedDep1', function(){}) for shared dependencies in a core script.

And then I got a very strange bug. Sometimes some of modules export a wrong value, as if the different modules are messed up.
Something like that:

System.import('moduleA').then(console.log) // And there is a moduleB in the console

After some investigation I figured out that it's because of a race condition. So it's very hard to reproduce and I've got it just because I have a huge SPA.
Here's what's happening:

  1. System.import()  adds a script into the document
  2. It sets onload event listener that will call getRegister after loading
  3. Another script finished its loading and that caused my code with amd define to be called
  4. define is working synchronous, it sets firstName and firstNamedDefine
  5. It sets setTimeout that clear them in the end
  6. Previous script is loaded and it fires onload event, then getRegister will be called and if (firstName && url) { is true

After that for some reason systemjs will use one module instead of the other.
I think the problem in this PR #2352
I guess there is a race condition between eventloop macrotasks: setTimeout and onload event.

Demonstration

Because of the race condition it's hard to reproduce apart of our code. So I wrote a synthetic example with multiple fake requests.
I made a loop with random choice of import or define. Check it out:
Code Sandbox: https://codesandbox.io/s/modest-chandrasekhar-lb7z6?file=/index.html

Expected Behavior

define call doesn't affect on System.import()
Every import returns its own module.

In my example it will be message We couldn't find the bug in the console

Actual Behavior

If we call define at the wrong moment, some modules will be mixed up.

In my example it will be message expect module b: {a: "a"} We've finally found the bug! in the console

Activity

bmpolonsky

bmpolonsky commented on Sep 29, 2021

@bmpolonsky
Author

My first clue was this.
I noticed that the modules have aliases of completely different services

image

bmpolonsky

bmpolonsky commented on Sep 29, 2021

@bmpolonsky
Author

The same example but with the previous version of systemjs (6.10.2):

https://codesandbox.io/s/dawn-dawn-hevnx?file=/index.html

jolyndenning

jolyndenning commented on Oct 4, 2021

@jolyndenning
Collaborator

The named-register extra (used to process define('name', ...) has always had race conditions in every version. I've fixed more than five in the last two years, and added tests, but it's quite possible there are still race conditions. The change in 6.10.3 is was very related to it, reworking much of how named registers work. See #2352 for more detail.

Thanks for providing the code sandbox, I'll look into it and see if I can identify the issue.

jolyndenning

jolyndenning commented on Oct 4, 2021

@jolyndenning
Collaborator

For some background info, the named-register extra has no way of differentiating a define() that occurs as a result from System.import vs one that does not. The job of the named-register extra is to implement the System.getRegister() function so that it returns the first named register within a module loaded via System.import. It is supposed to ignore named registers that didn't originate from System.import(), and also is supposed to be concurrent safe for multiple System.imports that occur simultaneously.

The way that named-register implements this is by storing the firstNamedDefine variable. When the firstNamedDefine variable exists, it is returned from getRegister. The firstNamedDefine is set synchronously at the moment that define is called, and then is cleared in the next tick of the event loop (via a setTimeout) or when getRegister is called. See code below for details:

setTimeout(function () {
firstNamedDefine = null;
firstName = null;
});

The bug in your code sandbox occurs when define('a', ...) occurs in the same tick of the event loop as the define('b', ...). In that case, the setTimeout to clear the firstNamedDefine has not executed yet, which results in the bug. A workaround is to call getRegister() right after the define('a', ...), which forcibly clears the firstNamedDefine variable. See https://codesandbox.io/s/competent-wright-kt6ur?file=/index.html which shows that.

That setTimeout is the root of all of the race conditions. We started it as setTimeout, switched it to Promise.resolve, switched it back to setTimeout, and have reworked the named-register various times to try to avoid race conditions. However, we do not know of a way to implement the namedRegister without that firstNamedDefine variable, which is just inherently not a great approach. The ideal would be to clear it right before executing the code for a module loaded with System.import, then call getRegister to clear it immediately after the code is executed. The majority of SystemJS loading is done with <script> elements since they're more performant than fetch + eval, and with script loading we call getRegister() during the load event. However, I do not think there is a beforeload event that is usable for script tags. For fetch + eval, you could just call getRegister() right before eval() in order to first clear the firstNamedDefine variable before executing the code.

If you have ideas about how this could implemented in a general way that works with script loading (not just fetch + eval), that would be very helpful. I think I've exhausted my ideas on the topic and don't know of any new things that could attempt to solve the problem. I'll look over your code sandbox a bit more and see if anything comes to mind, but do not know how it could be fixed.

jolyndenning

jolyndenning commented on Oct 4, 2021

@jolyndenning
Collaborator

I personally do not use the named-register for the reasons described above, preferring just to use anonymous modules which do not have these race conditions.

bmpolonsky

bmpolonsky commented on Oct 8, 2021

@bmpolonsky
Author

Thank you for your reply! I am on vacation now, but I will return to this problem soon. Btw I don't understand why you changed the promise to the settimout. Isn't microtask better in this case?

jolyndenning

jolyndenning commented on Oct 12, 2021

@jolyndenning
Collaborator

Yeah Promise.resolve() is better, and I didn't switch from Promise.resolve to setTimeout intentionally in #2352. I'm not sure if Promise.resolve will completely solve the race condition, but am willing to try it out. I think the problem is occurring when the <script> appended by System.import() executes in the same microtick of the event loop as the manual call to define() in your code sandbox. So adding a microtick after that microtick might still be too late. However, I'm not 100% sure if that's the case or if that's even possible with the browser's implementaiton of script tags.

In testing your code sandbox with Promise.resolve(), it seems to have fixed it, so maybe it is a microtick that's the difference. I'll send in a pull request with it.

added a commit that references this issue on Oct 12, 2021
92e9a59
jolyndenning

jolyndenning commented on Oct 12, 2021

@jolyndenning
Collaborator

I've created #2363 with the fix, switching to Promise.resolve

bmpolonsky

bmpolonsky commented on Oct 18, 2021

@bmpolonsky
Author

Thank you! By the way I'm not sure we can write tests for this case. What do you think?

bmpolonsky

bmpolonsky commented on Oct 18, 2021

@bmpolonsky
Author

Or can we use tha sandbox example somehow?

And I've noticed that there is no need for Promise.resolve around define in my sandbox.
Just a "synchronous" define inside setInterval is enough.
https://codesandbox.io/s/keen-firefly-0bomu

jolyndenning

jolyndenning commented on Oct 18, 2021

@jolyndenning
Collaborator
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

      AMD named define works strangely · Issue #2359 · systemjs/systemjs