Might need to lock down fragment usage

[bmeck]

If WICG/webpackage#172 doesn't allow differing sources per fragment and the browser network stack also has the same constraint, we might want to impose the same constraint for fragments in our loaders. Where we would treat fragments as resolving to the URL without the fragment and having users rely on import.meta.url to differentiate.

[devsnek]

I'm fine with that behaviour, in fact I would encourage it.

[guybedford]

Great catch, for some reason I assumed we already did this. I'd even be for sanitizing it out immediately after resolution entirely.

[SMotaal]

Revised I might have rushed... It seems Chrome (and Safari) still sends a new request for a previously loaded module when the fragment changes.

This might be useful to better elaborate what I am trying to say:

// server.js
const http = require('http');

let request = 0;

const createModule = (id = 'module') => `
  var global = typeof global !== 'undefined' && global || window;
  const id = '${id}.js';
  const symbol = Symbol.for(id);
  const instance = global[symbol] = (global[symbol] || 0) + 1;
  export default (id + ' #' + instance + '/' + '${++request}');
`;

const server = http.createServer(
  (request, response) => {
    const source = createModule(request.url);
    response.setHeader('content-type', 'text/javascript');
    response.setHeader('Access-Control-Allow-Origin', '*');
    response.end(source);
  }
).listen(8811);

Launching the above server makes it possible to track when a module is loaded (requested), instantiated (and evaluate) in Chrome's console, just make sure to load a blank page without any service workers.

Here are some results:

import('http://localhost:8811/module1').then(imported => console.log(imported.default));
//  /module1.js #1/1

import('http://localhost:8811/module1').then(imported => console.log(imported.default));
//  /module1.js #1/1

Same URL: resolved/loaded first time & instantiated first time

import('http://localhost:8811/module1#1').then(imported => console.log(imported.default));
//  /module1.js #2/2

import('http://localhost:8811/module1#1').then(imported => console.log(imported.default));
//  /module1.js #2/2

Different fragment: resolved/loaded (again) first time & instanced (again) first time

import('http://localhost:8811/module1#2').then(imported => console.log(imported.default));
//  /module1.js #3/3

import('http://localhost:8811/module1#1').then(imported => console.log(imported.default));
//  /module1.js #2/2

import('http://localhost:8811/module1').then(imported => console.log(imported.default));
//  /module1.js #1/1

Hope this elaborates it.

[guybedford]

Interesting, we may have an obligation to follow browsers here then. Perhaps we can get some spec clarification.

[bmeck]

I'm getting a different result on Chrome 65, Safari 11.1, and with a static HTTP server showing 1 request for the import regardless of fragment:

index.html:

<script type=module>
import './foo.mjs';
import './foo.mjs#1';
import './foo.mjs#2';
</script>

foo.mjs

console.log(import.meta.url);

$ http-server
Starting up http-server, serving ./
Available on:
  http://127.0.0.1:8081
  http://192.168.1.184:8081
Hit CTRL-C to stop the server
[Tue Apr 10 2018 14:13:05 GMT-0500 (CDT)] "GET /" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36"
[Tue Apr 10 2018 14:13:05 GMT-0500 (CDT)] "GET /foo.mjs" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36"
[Tue Apr 10 2018 14:13:07 GMT-0500 (CDT)] "GET /" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/11.1 Safari/605.1.15"
[Tue Apr 10 2018 14:13:07 GMT-0500 (CDT)] "GET /foo.mjs" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/11.1 Safari/605.1.15"

[jdalton]

In Chrome 65 and Chrome 67 this jsbin, which uses the same URL with different fragments, creates 3 different executions:

Also interesting the import.meta.url doesn't include the fragment in either Chrome versions.

Safari Technical Preview (Release 53; Safari 11.2) has 3 different executions too, but also includes the fragment in import.meta.url.

While Edge (17 preview) and Firefox Nightly (61.0a1) don't support import.meta yet, they do perform 3 separate executions.

Edge (17 preview)

Firefox Nightly (61.0a1)

[SMotaal]

@bmeck Actually import.meta.url is a little tricky, it can be returning either the (a) URL that is derived from an absolute specifier or a relative (to an absolute referrer) or (b) URI that is used for the network request (which does not include a hash since it is a client side extension, but would a search string). So my results showed that all request for the same URI but with different hashes resulted in requesting the same URI for each hash (where it would be expected to only be requested once) and each URL resulted in a different module instance which was created from those separate module requests only the first time the unique URL was imported.

Please advise: is it okay to pitch in (or is there a place where it is more appropriate for non-members to take part in a more moderated setting.

[domenic]

In the browser, different fragments are supposed to result in multiple requests; see the large example in https://html.spec.whatwg.org/#module-map. The module map is keyed by request URL.

However, they are not supposed to survive to import.meta.url, since import.meta.url is the response URL, and response URLs don't contain fragments. See whatwg/html#3622.

[domenic]

Note that whether your server gets hit 3 times, when 3 requests are issued, is dependent on your server's caching headers. This may explain the disparity between @bmeck's server logs and @jdalton's tests.

[bmeck]

@domenic I have caches disabled and devtools open on that log.

[jdalton]

@domenic I noticed in Chrome 65/67 if I mess around with the jsbin and add a fresh query+fragment I can get the fragment to be reported in import.meta.url.

[guybedford]

I think this can be closed as it turns out we do follow the same model as the web here.

[jdalton]

Would it make sense to drop the fragment for now and add it back when/if the spec changes?
(This way it at least matches Chrome)

[guybedford]

We wouldn't be matching Chrome if we drop the fragment, as we need to have 3 separate executions. Browser request caching / coalescing may be slightly inconsistent here, but the underlying module map is still fragment-based.

[jdalton]

The import.meta.url value doesn't indicate cache keys (not intended to) so we can align with Chrome on the import.meta.url value without having to adjust caching keys. We shouldn't try to shoehorn import.meta.url into the cache key box. If we need to expose a cache key there are likely better possible APIs to do it.

[ljharb]

Totally agree, especially since the conceptual purpose of import.meta.url is “the url that relative paths will be relative to” - the cache key is something else.

[guybedford]

import.meta.url is supposed to include the fragment according to the WhatWG specification of the response URL - whatwg/html#3622 (comment).

[jdalton]

I think github comments and spec are different. I rather make changes align with implementations or specs and not github comments.

[guybedford]

The thread explicitly states that this is the specification behaviour pending web platform tests.

[jdalton]

Pending is pending and Chrome has this behavior. Even if fragments are allowed (I'd prefer them too) I'd still want a better way to get a cache key than import.meta.url which may or may not incidentally be a similar value. And if we ship import.meta.url I think, since Node is v8 centric, aligning with whatever the Chrome behavior is (is a nice touch).

[domenic]

The specification includes fragments in import.meta.url. I thought that was a mistake, but apparently it's correct. We will write web platform tests for that aspect of the (current) specification, and file bugs on any browsers that don't follow them.

[guybedford]

@jdalton I'm not sure why you think I'm interested in cache keys, that really doesn't concern me - I'm only trying to follow the specification.

[jdalton]

@guybedford Great!

[guybedford]

So to summarize our current implementation - (a) fragments are set in the module map resulting in repeated execution (b) fragments are maintained in import.meta.url, and this is handled even when following symlinks.

Both of these behaviors are to the specification.

Perhaps we should just look at ensuring this is tested though in Node before closing this?

[benjamingr]

@guybedford are you interested in following up with a test PR?

[SMotaal]

@guybedford

(a) fragments are set in the module map resulting in repeated execution

Does this mean reloading the source (pickups a changed code in new module instance), or even more thoroughly, re-resolving the specifier altogether (change to files structure not just to content of the same file)? I guessing where the line would be drawn would have great implications.

[guybedford]

@benjamingr I will keep this on my list, but am a little constrained for time right now.

Does this mean reloading the source (pickups a changed code in new module instance), or even more thoroughly, re-resolving the specifier altogether (change to files structure not just to content of the same file)? I guessing where the line would be drawn would have great implications.

await import('./a.js'), await import('./a.js#2') would resolve to two separate sources at file:///path/to/a.js and file:///path/to/a.js#2, then fetching and executing them as if they were separate files.

[bmeck]

closing based upon #60 (comment) and nothing of concern showing up after that.