Add specification for Synthetic Module Records #44

domenic · 2019-02-26T18:12:13Z

Hey folks!

I thought I'd get things started with a spec for module records that seem like they'd be useful for built-in modules, JSON modules, CSS modules, etc. As we work through these ideas on the web platform side, we've found some confusions that would be prevented with a more concrete spec text, so I wanted to write something up we could point to.

I've deployed a preview of the built product at https://proposal-javascript-standard-library-domenic.now.sh/. Thoughts welcome!

littledan · 2019-02-27T22:15:25Z

This looks great to me. I don't see any particular technical issues, just some one editorial comment (which would not be any sort of normative change):

All use cases I can think of for synthetic modules would have fixed exports once the module is "parsed". What if CreateSyntheticModule instead took two arguments, the realm and a mapping from strings to JavaScript values (or, since we don't have infra, a List of Records { [[Name]]: String, [[Value]]: JS value }), and set these up as immutable bindings during the Initialize() phase? We can consider generalizing the mechanism if there's a particular use case for it.

cc @Ms2ger

domenic · 2019-02-27T22:26:28Z

Yeah, that's a possible approach. However, my thinking went toward the current approach like so:

We need custom evaluation steps anyway for side effects. (E.g., KV storage throws in non-secure contexts, or built-in module might define a custom element, or apply a stylesheet to the page, or something.) So that hook needs to stay anyway.
It's weird to create objects before the module is evaluated. For example, KV storage isn't supposed to be usable in non-secure contexts. We could create all its exports ahead of time, export them as immutable bindings, but then have the module just throw when it's evaluated, despite exporting things. But that seems strange; I'd prefer to emulate how JS code works, in that it establishes the bindings' values at evaluation time, after any checks it might want to perform. I think they are observably equivalent though, i.e. I think you can't observe a throwing-when-evaluated module's bindings.

(Note: there are other ways to achieve the secure context restriction, e.g.: not putting the module in the module map at all, or exporting things that throw when you try to use them, or exporting undefined for all exports, or exporting zero bindings in that case. I don't claim the current design is the best one for KV storage to choose, and welcome discussion over on that issue tracker. But, I think taking it as a given for this discussion is a helpful way of illustrating the things a built-in module might want to do, and that any generic system like this PR should probably accomodate.)
It seems at least possible that you might want to export, either mutable bindings, or bindings whose values can't be determined until evaluation time. (For example, given a JSON module, is it OK to execute the JSON parsing algorithm at instantiation time? It seems like it runs JS script to me.) Thus something like SetSyntheticModuleExport() seems useful to include. But if we're going to have that anyway, then why create a dual system where you also have an [[ExportValues]] List, or a { [[Name]], [[Value]] } List, for the initial values, and use SetSyntheticModuleExport() for the runtime ones?

I'm open to changing this, but I think the current approach is more flexible, so I'm inclined toward that one.

domenic · 2019-02-27T22:29:31Z

To be more concrete, with the JSON module example, with the OP design I think it would have straightforward evaluation steps:

Let result be ? Call(%JSONParse%, undefined, « jsonText »).
Perform SetSyntheticModuleExport(module, "default", result).

But with a static-exports approach, it would instead need to be something like:

Parse the JSON during instantiation time. (Assume this is legal; we can probably make it work.)
If that throws an exception, store the result in module.[[HostDefined]] as { [[Exception]]: e }
Otherwise, set [[ExportNames]] to « "default" » and [[ExportValues]] to « result ».
Have evaluation steps that check if module.[[HostDefined]] contains an [[Exception]] field, and if so, rethrows it.

littledan · 2019-02-27T22:41:13Z

Thanks for walking through these error cases; that makes the motivation for this design a lot clearer. The strategy you describe for JSON parse errors is sort of what I expected (that's what we do for JS early errors phase-wise, right?), but I can see how it's a lot more complicated to write down.

Actually, now, we came to an observable difference, didn't we? The find the first parse error algorithm would stop any module from executing if it hit a JSON parse error if these are treated as parse errors. But, if they are treated as errors during the execution phase, it might throw the exception after executing some other earlier sibling modules. (Maybe we should move this discussion back to whatwg/html#4315 (comment) .)

All I could find on import-maps for why it throws an exception on evaluation rather than being absent from the import-map (which is what I would've expected, but don't feel strongly about) was WICG/kv-storage#16 . Was there more context somewhere, or should I file a different issue?

This patch provides JSON modules as a single default export, with parse errors checked before instantiating the module graph. Note, editorially, it's unclear whether JSON modules should be considered a type of "module script", with a settings object, fetch options, base URL, etc or not. This patch considers them "module scripts", but leaves those record fields unset (as they are unused). This patch is based on tc39/proposal-built-in-modules#44 which hasn't landed yet, so the references are a bit awkward, and this patch should not land until that one does. Closes whatwg#4315

littledan · 2019-03-03T21:29:17Z

Thinking about this more: Even if I am not sure about those semantic details for the use cases, I think it's good to give module evaluation steps, to do the equivalent of executing class declarations, etc. This would make it very practical to have a real implementation which executes synthetic modules based on JavaScript modules. So, +1 to all of this PR from me.

From that frame, I'm wondering: Should we leave the step of initializing bindings to the module, rather than initializing them to undefined?

This PR defines the ability in WebIDL to define JavaScript built-in modules, based on the proposed infrastructure at tc39/proposal-built-in-modules#44 These semantics set the module map, which makes sense in the context of the import maps draft specification https://wicg.github.io/import-maps/

@imports

This patch provides CSS modules as a single default export, of a CSSStyleSheet object, which is not added to the document. Edge cases which might be worth reconsidering: - @imports are recursively fetched together with the module graph, blocking script execution. Network errors reached prevent the execution of the entire module graph. - Any MIME type whose essence is "text/css" is accepted; this appears to be weaker checking than elsewhere in the specification. - Although the Constructable Stylesheet Objects proposal is used for infrastructure, the resulting CSSStyleSheet object acts as if it were not constructed (i.e., you can't call the replace() method). This patch is based on tc39/proposal-built-in-modules#44 Closes whatwg#4315

@imports

This patch provides CSS modules as a single default export, of a CSSStyleSheet object, which is not added to the document. Edge cases which I didn't see discussed elsewhere: - @imports are recursively fetched together with the module graph, blocking script execution. Network errors reached prevent the execution of the entire module graph. - Any MIME type whose essence is "text/css" is accepted; this appears to be weaker checking than elsewhere in the specification. - Although the Constructable Stylesheet Objects proposal is used for infrastructure, the resulting CSSStyleSheet object acts as if it were not constructed (i.e., you can't call the replace() method). Note, the Constructable Stylesheet Objects proposal makes important steps to specifying loading of @import, but there may still be room for more precise plumbing with HTML. This text ensures that style sheet module scripts have a base URL and fetch options, which might be referenced by the definition of @import in the future. This patch is based on tc39/proposal-built-in-modules#44 Closes whatwg#4315 Closes WICG/webcomponents#759

This PR defines the ability in WebIDL to define JavaScript built-in modules, based on the proposed infrastructure at tc39/proposal-built-in-modules#44 These semantics set the module map, which makes sense in the context of the import maps draft specification https://wicg.github.io/import-maps/

[[Status]] is a field of Cyclic Module Records; Abstract Module Records don't keep track of it. This patch avoids checking the [[Status]] of modules that don't have one, instead first checking whether the module is cyclic. All of the use cases seemed to be when modules were in dependency chains and not leaves. Given the proposed Synthetic Module Records tc39/proposal-built-in-modules#44 and their possible usage in JSON modules, CSS modules, and WebIDL modules, it makes sense to avoid these code paths for those cases.

Ms2ger · 2019-03-26T16:37:47Z

spec.html

+        1. Set the LexicalEnvironment of _moduleCxt_ to _module_.[[Environment]].
+        1. Suspend the currently running execution context.
+        1. Push _moduleCxt_ on to the execution context stack; _moduleCxt_ is now the running execution context.
+        1. Let _result_ be the result of performing ? _module_.[[EvaluationSteps]](_module_).


Should this ? be here? It seems like the execution context stack will be messed up if this throws.

? seems right to me--the module evaluation algorithms are all written around Evaluate() maybe throwing. See tc39/ecma262#916 for more context.

Or, step 5 of https://tc39.github.io/ecma262/#sec-moduleevaluation

I think @Ms2ger is right. The ? is wrong; the exception gets re-thrown (or the normal completion gets returned) 3 steps down, but before doing that we need to clean up the execution context stack stuff.

This PR defines the ability in WebIDL to define JavaScript built-in modules, based on the proposed infrastructure at tc39/proposal-built-in-modules#44 These semantics set the module map, which makes sense in the context of the import maps draft specification https://wicg.github.io/import-maps/

[[Status]] is a field of Cyclic Module Records; Abstract Module Records don't keep track of it. This patch avoids checking the [[Status]] of modules that don't have one, instead first checking whether the module is cyclic. All of the use cases seemed to be when modules were in dependency chains and not leaves. Given the proposed Synthetic Module Records tc39/proposal-built-in-modules#44 and their possible usage in JSON modules, CSS modules, and WebIDL modules, it makes sense to avoid these code paths for those cases.

This patch provides JSON modules as a single default export, with parse errors checked before instantiating the module graph. Note, editorially, it's unclear whether JSON modules should be considered a type of "module script", with a settings object, fetch options, base URL, etc or not. This patch considers them "module scripts", but leaves those record fields unset (as they are unused). This patch is based on tc39/proposal-built-in-modules#44 which hasn't landed yet, so the references are a bit awkward, and this patch should not land until that one does. Closes whatwg#4315

Ms2ger · 2019-05-08T15:46:35Z

Hey folks, I'm planning to look into adding this text to WebIDL for the time being, so we can move forward with whatwg/html#4407. I'd be happy to help with moving this into Ecma262 or any other place in the future, if that's what we decide on. I'd certainly want to make sure it remains useful for the work you're doing here.

Based on text written by Domenic at <tc39/proposal-built-in-modules#44>.

Ms2ger · 2019-05-09T12:00:36Z

Please see whatwg/webidl#722

This PR defines the ability in WebIDL to define JavaScript built-in modules, based on the proposed infrastructure at tc39/proposal-built-in-modules#44 These semantics set the module map, which makes sense in the context of the import maps draft specification https://wicg.github.io/import-maps/

Based on text written by Domenic at tc39/proposal-built-in-modules#44. These will be used by HTML in whatwg/html#4407.

@imports

This patch provides CSS modules as a single default export, of a CSSStyleSheet object, which is not added to the document. Edge cases which I didn't see discussed elsewhere: - @imports are recursively fetched together with the module graph, blocking script execution. Network errors reached prevent the execution of the entire module graph. - Any MIME type whose essence is "text/css" is accepted; this appears to be weaker checking than elsewhere in the specification. - Although the Constructable Stylesheet Objects proposal is used for infrastructure, the resulting CSSStyleSheet object acts as if it were not constructed (i.e., you can't call the replace() method). Note, the Constructable Stylesheet Objects proposal makes important steps to specifying loading of @import, but there may still be room for more precise plumbing with HTML. This text ensures that style sheet module scripts have a base URL and fetch options, which might be referenced by the definition of @import in the future. This patch is based on tc39/proposal-built-in-modules#44 Closes whatwg#4315 Closes WICG/webcomponents#759

This PR defines the ability in WebIDL to define JavaScript built-in modules, based on the proposed infrastructure at tc39/proposal-built-in-modules#44 These semantics set the module map, which makes sense in the context of the import maps draft specification https://wicg.github.io/import-maps/

Add specification for Synthetic Module Records

61174f8

domenic force-pushed the draft-spec branch from 818e3c3 to 61174f8 Compare February 26, 2019 18:16

littledan mentioned this pull request Feb 27, 2019

JSON modules whatwg/html#4315

Closed

This was referenced Mar 2, 2019

JSON module support whatwg/html#4407

Merged

Declaring modules in WebIDL whatwg/webidl#592

Open

littledan mentioned this pull request Mar 3, 2019

Define Web IDL Modules whatwg/webidl#675

Closed

littledan mentioned this pull request Mar 8, 2019

[WIP] Add style sheet module scripts whatwg/html#4423

Closed

domenic mentioned this pull request Mar 14, 2019

JavaScript Standard Library Proposal (Builtin Modules) mozilla/standards-positions#147

Open

littledan mentioned this pull request Mar 19, 2019

Editorial: Don't read the [[Status]] of non-cyclic modules tc39/ecma262#1486

Merged

Ms2ger reviewed Mar 26, 2019

View reviewed changes

Ms2ger added a commit to whatwg/webidl that referenced this pull request May 9, 2019

Define Synthetic Module Records

5c7afbe

Based on text written by Domenic at <tc39/proposal-built-in-modules#44>.

Ms2ger mentioned this pull request May 9, 2019

Define Synthetic Module Records whatwg/webidl#722

Merged

domenic pushed a commit to whatwg/webidl that referenced this pull request May 17, 2019

Define Synthetic Module Records

495ff31

Based on text written by Domenic at tc39/proposal-built-in-modules#44. These will be used by HTML in whatwg/html#4407.

domenic closed this Jun 2, 2023

ljharb deleted the draft-spec branch June 2, 2023 05:05

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add specification for Synthetic Module Records #44

Add specification for Synthetic Module Records #44

domenic commented Feb 26, 2019

littledan commented Feb 27, 2019

domenic commented Feb 27, 2019

domenic commented Feb 27, 2019

littledan commented Feb 27, 2019

littledan commented Mar 3, 2019 •

edited

Loading

Ms2ger Mar 26, 2019

littledan Mar 26, 2019

littledan Mar 26, 2019

domenic May 8, 2019

Ms2ger commented May 8, 2019

Ms2ger commented May 9, 2019

Add specification for Synthetic Module Records #44

Add specification for Synthetic Module Records #44

Conversation

domenic commented Feb 26, 2019

littledan commented Feb 27, 2019

domenic commented Feb 27, 2019

domenic commented Feb 27, 2019

littledan commented Feb 27, 2019

littledan commented Mar 3, 2019 • edited Loading

Ms2ger Mar 26, 2019

Choose a reason for hiding this comment

littledan Mar 26, 2019

Choose a reason for hiding this comment

littledan Mar 26, 2019

Choose a reason for hiding this comment

domenic May 8, 2019

Choose a reason for hiding this comment

Ms2ger commented May 8, 2019

Ms2ger commented May 9, 2019

littledan commented Mar 3, 2019 •

edited

Loading