esm: phase two of new esm implementation

This PR updates the current `--experimental-modules` implementation
based on the work of the modules team  and reflects Phase 2 of our
new modules plan.

The largest differences from the current implementation include

* `packge.type` which can be either `module` or `commonjs`
  - `type: "commonjs"`:
    - `.js` is parsed as commonjs
    - default for entry point without an extension is commonjs
  - `type: "module"`:
    - `.js` is parsed as esm
    - does not support loading JSON or Native Module by default
    - default for entry point without an extension is esm
* `--entry-type=[mode]`
  - allows you set the type on entry point.
* A new file extension `.cjs`.
  - this is specifically to support importing commonjs in the
    `module` mode.
  - this is only in the esm loader, the commonjs loader remains
    untouched, but the extension will work in the old loader if you use
    the full file path.
* `--es-module-specifier-resolution=[type]`
  - options are `explicit` (default) and `node`
  - by default our loader will not allow for optional extensions in
    the import, the path for a module must include the extension if
    there is one
  - by default our loader will not allow for importing directories that
    have an index file
  - developers can use `--es-module-specifier-resolution=node` to
    enable the commonjs specifier resolution algorithm
  - This is not a “feature” but rather an implementation for
    experimentation. It is expected to change before the flag is
    removed
* `--experimental-json-loader`
  - the only way to import json when `"type": "module"`
  - when enable all `import 'thing.json'` will go through the
    experimental loader independent of mode
  - based on whatwg/html#4315
* You can use `package.main` to set an entry point for a module
  - the file extensions used in main will be resolved based on the
    `type` of the module

Refs: https://github.com/nodejs/modules/blob/master/doc/plan-for-new-modules-implementation.md
Refs: https://github.com/GeoffreyBooth/node-import-file-specifier-resolution-proposal
Refs: nodejs/modules#180
Refs: nodejs/ecmascript-modules#6
Refs: nodejs/ecmascript-modules#12
Refs: nodejs/ecmascript-modules#28
Refs: nodejs/modules#255
Refs: whatwg/html#4315
Refs: WICG/webcomponents#770
Co-authored-by: Myles Borins <[email protected]>
Co-authored-by: John-David Dalton <[email protected]>
Co-authored-by: Evan Plaice <[email protected]>
Co-authored-by: Geoffrey Booth <[email protected]>
Co-authored-by: Michaël Zasso <[email protected]>

PR-URL: #26745
Reviewed-By: Gus Caplan <[email protected]>
Reviewed-By: Guy Bedford <[email protected]>
Reviewed-By: Ben Coe <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Joyee Cheung <[email protected]>
Reviewed-By: Ruben Bridgewater <[email protected]>
Reviewed-By: Сковорода Никита Андреевич <[email protected]>

Author: guybedford

.eslintrc.js

@@ -38,6 +38,8 @@ module.exports = {
     {
       files: [
         'doc/api/esm.md',
+        'test/es-module/test-esm-type-flag.js',
+        'test/es-module/test-esm-type-flag-alias.js',
         '*.mjs',
         'test/es-module/test-esm-example-loader.js',
       ],

doc/api/cli.md

@@ -131,9 +131,43 @@ conjunction with native stack and other runtime environment data.
 added: v6.0.0
 -->
 
+### `--entry-type=type`
+<!-- YAML
+added: REPLACEME
+-->
+
+Used with `--experimental-modules`, this configures Node.js to interpret the
+initial entry point as CommonJS or as an ES module.
+
+Valid values are `"commonjs"` and `"module"`. The default is to infer from
+the file extension and the `"type"` field in the nearest parent `package.json`.
+
+Works for executing a file as well as `--eval`, `--print`, `STDIN`.
+
 Enable FIPS-compliant crypto at startup. (Requires Node.js to be built with
 `./configure --openssl-fips`.)
 
+### `--es-module-specifier-resolution=mode`
+<!-- YAML
+added: REPLACEME
+-->
+
+To be used in conjunction with `--experimental-modules`. Sets the resolution
+algorithm for resolving specifiers. Valid options are `explicit` and `node`.
+
+The default is `explicit`, which requires providing the full path to a
+module. The `node` mode will enable support for optional file extensions and
+the ability to import a directory that has an index file.
+
+Please see [customizing esm specifier resolution][] for example usage.
+
+### `--experimental-json-modules`
+<!-- YAML
+added: REPLACEME
+-->
+
+Enable experimental JSON support for the ES Module loader.
+
 ### `--experimental-modules`
 <!-- YAML
 added: v8.5.0
@@ -927,9 +961,10 @@ greater than `4` (its current default value). For more information, see the
 [REPL]: repl.html
 [ScriptCoverage]: https://chromedevtools.github.io/devtools-protocol/tot/Profiler#type-ScriptCoverage
 [V8 JavaScript code coverage]: https://v8project.blogspot.com/2017/12/javascript-code-coverage.html
+[customizing esm specifier resolution]: esm.html#esm_customizing_esm_specifier_resolution_algorithm
 [debugger]: debugger.html
 [debugging security implications]: https://nodejs.org/en/docs/guides/debugging-getting-started/#security-implications
 [emit_warning]: process.html#process_process_emitwarning_warning_type_code_ctor
-[experimental ECMAScript Module]: esm.html#esm_loader_hooks
+[experimental ECMAScript Module]: esm.html#esm_resolve_hook
 [libuv threadpool documentation]: http://docs.libuv.org/en/latest/threadpool.html
 [remote code execution]: https://www.owasp.org/index.php/Code_Injection

doc/api/errors.md

@@ -844,6 +844,18 @@ provided.
 Encoding provided to `TextDecoder()` API was not one of the
 [WHATWG Supported Encodings][].
 
+<a id="ERR_ENTRY_TYPE_MISMATCH"></a>
+#### ERR_ENTRY_TYPE_MISMATCH
+
+> Stability: 1 - Experimental
+
+The `--entry-type=commonjs` flag was used to attempt to execute an `.mjs` file
+or a `.js` file where the nearest parent `package.json` contains
+`"type": "module"`; or
+the `--entry-type=module` flag was used to attempt to execute a `.cjs` file or
+a `.js` file where the nearest parent `package.json` either lacks a `"type"`
+field or contains `"type": "commonjs"`.
+
 <a id="ERR_FALSY_VALUE_REJECTION"></a>
 ### ERR_FALSY_VALUE_REJECTION
 
@@ -1267,6 +1279,11 @@ An invalid or unexpected value was passed in an options object.
 
 An invalid or unknown file encoding was passed.
 
+<a id="ERR_INVALID_PACKAGE_CONFIG"></a>
+### ERR_INVALID_PACKAGE_CONFIG
+
+An invalid `package.json` file was found which failed parsing.
+
 <a id="ERR_INVALID_PERFORMANCE_MARK"></a>
 ### ERR_INVALID_PERFORMANCE_MARK
 
@@ -1440,7 +1457,7 @@ strict compliance with the API specification (which in some cases may accept
 
 > Stability: 1 - Experimental
 
-An [ES6 module][] loader hook specified `format: 'dynamic'` but did not provide
+An [ES Module][] loader hook specified `format: 'dynamic'` but did not provide
 a `dynamicInstantiate` hook.
 
 <a id="ERR_MISSING_MESSAGE_PORT_IN_TRANSFER_LIST"></a>
@@ -1449,26 +1466,19 @@ a `dynamicInstantiate` hook.
 A `MessagePort` was found in the object passed to a `postMessage()` call,
 but not provided in the `transferList` for that call.
 
-<a id="ERR_MISSING_MODULE"></a>
-### ERR_MISSING_MODULE
-
-> Stability: 1 - Experimental
-
-An [ES6 module][] could not be resolved.
-
 <a id="ERR_MISSING_PLATFORM_FOR_WORKER"></a>
 ### ERR_MISSING_PLATFORM_FOR_WORKER
 
 The V8 platform used by this instance of Node.js does not support creating
 Workers. This is caused by lack of embedder support for Workers. In particular,
 this error will not occur with standard builds of Node.js.
 
-<a id="ERR_MODULE_RESOLUTION_LEGACY"></a>
-### ERR_MODULE_RESOLUTION_LEGACY
+<a id="ERR_MODULE_NOT_FOUND"></a>
+### ERR_MODULE_NOT_FOUND
 
 > Stability: 1 - Experimental
 
-A failure occurred resolving imports in an [ES6 module][].
+An [ES Module][] could not be resolved.
 
 <a id="ERR_MULTIPLE_CALLBACK"></a>
 ### ERR_MULTIPLE_CALLBACK
@@ -1555,7 +1565,7 @@ A given value is out of the accepted range.
 
 > Stability: 1 - Experimental
 
-An attempt was made to `require()` an [ES6 module][].
+An attempt was made to `require()` an [ES Module][].
 
 <a id="ERR_SCRIPT_EXECUTION_INTERRUPTED"></a>
 ### ERR_SCRIPT_EXECUTION_INTERRUPTED
@@ -2220,10 +2230,17 @@ A non-specific HTTP/2 error has occurred.
 Used in the `repl` in case the old history file is used and an error occurred
 while trying to read and parse it.
 
+<a id="ERR_INVALID_REPL_TYPE"></a>
+#### ERR_INVALID_REPL_TYPE
+
+> Stability: 1 - Experimental
+
+The `--entry-type=...` flag is not compatible with the Node.js REPL.
+
 <a id="ERR_MISSING_DYNAMIC_INSTANTIATE_HOOK"></a>
 #### ERR_MISSING_DYNAMIC_INSTANTIATE_HOOK
 
-Used when an [ES6 module][] loader hook specifies `format: 'dynamic'` but does
+Used when an [ES Module][] loader hook specifies `format: 'dynamic'` but does
 not provide a `dynamicInstantiate` hook.
 
 <a id="ERR_STREAM_HAS_STRINGDECODER"></a>
@@ -2250,7 +2267,6 @@ size.
 This `Error` is thrown when a read is attempted on a TTY `WriteStream`,
 such as `process.stdout.on('data')`.
 
-
 [`'uncaughtException'`]: process.html#process_event_uncaughtexception
 [`--force-fips`]: cli.html#cli_force_fips
 [`Class: assert.AssertionError`]: assert.html#assert_class_assert_assertionerror
@@ -2293,7 +2309,7 @@ such as `process.stdout.on('data')`.
 [`subprocess.kill()`]: child_process.html#child_process_subprocess_kill_signal
 [`subprocess.send()`]: child_process.html#child_process_subprocess_send_message_sendhandle_options_callback
 [`zlib`]: zlib.html
-[ES6 module]: esm.html
+[ES Module]: esm.html
 [ICU]: intl.html#intl_internationalization_support
 [Node.js Error Codes]: #nodejs-error-codes
 [V8's stack trace API]: https://github.com/v8/v8/wiki/Stack-Trace-API