[v22.x] backport unflagging of --experimental-require-module and related fixes/refactorings

doc/api/cli.md

@@ -1054,6 +1054,10 @@
 
 <!-- YAML
 added: v22.0.0
+changes:
+  - version: REPLACEME
+    pr-url: https://github.com/nodejs/node/pull/55085
+    description: This is now true by default.
 -->
 
 > Stability: 1.1 - Active Development
@@ -1695,6 +1699,22 @@
 
 Use this flag to disable top-level await in REPL.
 
+### `--no-experimental-require-module`
+
+<!-- YAML
+added: v22.0.0
+changes:
+  - version: REPLACEME
+    pr-url: https://github.com/nodejs/node/pull/55085
+    description: This is now false by default.
+-->
+
+> Stability: 1.1 - Active Development
+
+Disable support for loading a synchronous ES module graph in `require()`.
+
+See [Loading ECMAScript modules using `require()`][].
+
 ### `--no-experimental-websocket`
 
 <!-- YAML
@@ -1900,9 +1920,7 @@
 added: v22.0.0
 -->
 
-This flag is only useful when `--experimental-require-module` is enabled.
-
-If the ES module being `require()`'d contains top-level await, this flag
+If the ES module being `require()`'d contains top-level `await`, this flag
 allows Node.js to evaluate the module, try to locate the
 top-level awaits, and print their location to help users find them.
 

doc/api/errors.md

@@ -2478,8 +2478,8 @@
 
 > Stability: 1 - Experimental
 
-When trying to `require()` a [ES Module][] under `--experimental-require-module`,
-a CommonJS to ESM or ESM to CommonJS edge participates in an immediate cycle.
+When trying to `require()` a [ES Module][], a CommonJS to ESM or ESM to CommonJS edge
+participates in an immediate cycle.
 This is not allowed because ES Modules cannot be evaluated while they are
 already being evaluated.
 
@@ -2493,8 +2493,8 @@
 
 > Stability: 1 - Experimental
 
-When trying to `require()` a [ES Module][] under `--experimental-require-module`,
-the module turns out to be asynchronous. That is, it contains top-level await.
+When trying to `require()` a [ES Module][], the module turns out to be asynchronous.
+That is, it contains top-level await.
 
 To see where the top-level await is, use
 `--experimental-print-required-tla` (this would execute the modules
@@ -2504,12 +2504,20 @@
 
 ### `ERR_REQUIRE_ESM`
 
-> Stability: 1 - Experimental
+<!-- YAML
+changes:
+  - version: REPLACEME
+    pr-url: https://github.com/nodejs/node/pull/55085
+    description: require() now supports loading synchronous ES modules by default.
+-->
+
+> Stability: 0 - Deprecated
 
 An attempt was made to `require()` an [ES Module][].
 
-To enable `require()` for synchronous module graphs (without
-top-level `await`), use `--experimental-require-module`.
+This error has been deprecated since `require()` now supports loading synchronous
+ES modules. When `require()` encounters an ES module that contains top-level
+`await`, it will throw [`ERR_REQUIRE_ASYNC_MODULE`][] instead.
 
 <a id="ERR_SCRIPT_EXECUTION_INTERRUPTED"></a>
 
@@ -4123,6 +4131,7 @@
 [`ERR_INVALID_ARG_TYPE`]: #err_invalid_arg_type
 [`ERR_MISSING_MESSAGE_PORT_IN_TRANSFER_LIST`]: #err_missing_message_port_in_transfer_list
 [`ERR_MISSING_TRANSFERABLE_IN_TRANSFER_LIST`]: #err_missing_transferable_in_transfer_list
+[`ERR_REQUIRE_ASYNC_MODULE`]: #err_require_async_module
 [`EventEmitter`]: events.md#class-eventemitter
 [`MessagePort`]: worker_threads.md#class-messageport
 [`Object.getPrototypeOf`]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/getPrototypeOf

doc/api/esm.md

@@ -468,7 +468,7 @@ compatibility.
 ### `require`
 
 The CommonJS module `require` currently only supports loading synchronous ES
-modules when `--experimental-require-module` is enabled.
+modules (that is, ES modules that do not use top-level `await`).
 
 See [Loading ECMAScript modules using `require()`][] for details.
 

doc/api/modules.md

@@ -172,20 +172,20 @@
 
 <!-- YAML
 added: v22.0.0
+changes:
+  - version: REPLACEME
+    pr-url: https://github.com/nodejs/node/pull/55085
+    description: This feature is no longer behind the `--experimental-require-module` CLI flag.
 -->
 
 > Stability: 1.1 - Active Development. Enable this API with the
 > [`--experimental-require-module`][] CLI flag.
 
 The `.mjs` extension is reserved for [ECMAScript Modules][].
-Currently, if the flag `--experimental-require-module` is not used, loading
-an ECMAScript module using `require()` will throw a [`ERR_REQUIRE_ESM`][]
-error, and users need to use [`import()`][] instead. See
-[Determining module system][] section for more info
+See [Determining module system][] section for more info
 regarding which files are parsed as ECMAScript modules.
 
-If `--experimental-require-module` is enabled, and the ECMAScript module being
-loaded by `require()` meets the following requirements:
+`require()` only supports loading ECMAScript modules that meet the following requirements:
 
 * The module is fully synchronous (contains no top-level `await`); and
 * One of these conditions are met:
@@ -194,8 +194,8 @@
   3. The file has a `.js` extension, the closest `package.json` does not contain
      `"type": "commonjs"`, and the module contains ES module syntax.
 
-`require()` will load the requested module as an ES Module, and return
-the module namespace object. In this case it is similar to dynamic
+If the ES Module being loaded meet the requirements, `require()` can load it and
+return the module namespace object. In this case it is similar to dynamic
 `import()` but is run synchronously and returns the name space object
 directly.
 
@@ -214,7 +214,7 @@
 export default Point;
 ```
 
-A CommonJS module can load them with `require()` under `--experimental-detect-module`:
+A CommonJS module can load them with `require()`:
 
 ```cjs
 const distance = require('./distance.mjs');
@@ -243,13 +243,18 @@
 If the module being `require()`'d contains top-level `await`, or the module
 graph it `import`s contains top-level `await`,
 [`ERR_REQUIRE_ASYNC_MODULE`][] will be thrown. In this case, users should
-load the asynchronous module using `import()`.
+load the asynchronous module using [`import()`][].
 
 If `--experimental-print-required-tla` is enabled, instead of throwing
 `ERR_REQUIRE_ASYNC_MODULE` before evaluation, Node.js will evaluate the
 module, try to locate the top-level awaits, and print their location to
 help users fix them.
 
+Support for loading ES modules using `require()` is currently
+experimental and can be disabled using `--no-experimental-require-module`.
+When `require()` actually encounters an ES module for the
+first time in the process, it will emit an experimental warning. The
+warning is expected to be removed when this feature stablizes.
 This feature can be detected by checking if
 [`process.features.require_module`][] is `true`.
 
@@ -282,8 +287,7 @@
 
 MAYBE_DETECT_AND_LOAD(X)
 1. If X parses as a CommonJS module, load X as a CommonJS module. STOP.
-2. Else, if `--experimental-require-module` is
-  enabled, and the source code of X can be parsed as ECMAScript module using
+2. Else, if the source code of X can be parsed as ECMAScript module using
   <a href="esm.md#resolver-algorithm-specification">DETECT_MODULE_SYNTAX defined in
   the ESM resolver</a>,
   a. Load X as an ECMAScript module. STOP.
@@ -1201,7 +1205,6 @@
 [`"type"`]: packages.md#type
 [`--experimental-require-module`]: cli.md#--experimental-require-module
 [`ERR_REQUIRE_ASYNC_MODULE`]: errors.md#err_require_async_module
-[`ERR_REQUIRE_ESM`]: errors.md#err_require_esm
 [`ERR_UNSUPPORTED_DIR_IMPORT`]: errors.md#err_unsupported_dir_import
 [`MODULE_NOT_FOUND`]: errors.md#module_not_found
 [`__dirname`]: #__dirname

doc/api/packages.md

@@ -172,8 +172,7 @@ There is the CommonJS module loader:
 * It treats all files that lack `.json` or `.node` extensions as JavaScript
   text files.
 * It can only be used to [load ECMASCript modules from CommonJS modules][] if
-  the module graph is synchronous (that contains no top-level `await`) when
-  `--experimental-require-module` is enabled.
+  the module graph is synchronous (that contains no top-level `await`).
   When used to load a JavaScript text file that is not an ECMAScript module,
   the file will be loaded as a CommonJS module.
 
@@ -662,8 +661,7 @@ specific to least specific as conditions should be defined:
 * `"require"` - matches when the package is loaded via `require()`. The
   referenced file should be loadable with `require()` although the condition
   matches regardless of the module format of the target file. Expected
-  formats include CommonJS, JSON, native addons, and ES modules
-  if `--experimental-require-module` is enabled. _Always mutually
+  formats include CommonJS, JSON, native addons, and ES modules. _Always mutually
   exclusive with `"import"`._
 * `"module-sync"` - matches no matter the package is loaded via `import`,
   `import()` or `require()`. The format is expected to be ES modules that does

lib/internal/modules/cjs/loader.js

@@ -124,6 +124,8 @@ const { pathToFileURL, fileURLToPath, isURL } = require('internal/url');
 const {
   pendingDeprecate,
   emitExperimentalWarning,
+  isInsideNodeModules,
+  isUnderNodeModules,
   kEmptyObject,
   setOwnProperty,
   getLazy,
@@ -146,7 +148,6 @@ const { safeGetenv } = internalBinding('credentials');
 const {
   getCjsConditions,
   initializeCjsConditions,
-  isUnderNodeModules,
   loadBuiltinModule,
   makeRequireFunction,
   setHasStartedUserCJSExecution,
@@ -438,7 +439,6 @@ function initializeCJS() {
     Module._extensions['.ts'] = loadTS;
   }
   if (getOptionValue('--experimental-require-module')) {
-    emitExperimentalWarning('Support for loading ES Module in require()');
     Module._extensions['.mjs'] = loadESMFromCJS;
     if (tsEnabled) {
       Module._extensions['.mts'] = loadESMFromCJS;
@@ -1352,6 +1352,7 @@ let resolvedArgv;
 let hasPausedEntry = false;
 /** @type {import('vm').Script} */
 
+let emittedRequireModuleWarning = false;
 /**
  * Resolve and evaluate it synchronously as ESM if it's ESM.
  * @param {Module} mod CJS module instance
@@ -1375,10 +1376,27 @@ function loadESMFromCJS(mod, filename) {
     // ESM won't be accessible via process.mainModule.
     setOwnProperty(process, 'mainModule', undefined);
   } else {
+    const parent = mod[kModuleParent];
+    if (!emittedRequireModuleWarning) {
+      let shouldEmitWarning = false;
+      // Check if the require() comes from node_modules.
+      if (parent) {
+        shouldEmitWarning = !isUnderNodeModules(parent.filename);
+      } else if (mod[kIsCachedByESMLoader]) {
+        // It comes from the require() built for `import cjs` and doesn't have a parent recorded
+        // in the CJS module instance. Inspect the stack trace to see if the require()
+        // comes from node_modules.
+        shouldEmitWarning = !isInsideNodeModules();
+      }
+      if (shouldEmitWarning) {
+        emitExperimentalWarning('Support for loading ES Module in require()');
+        emittedRequireModuleWarning = true;
+      }
+    }
     const {
       wrap,
       namespace,
-    } = cascadedLoader.importSyncForRequire(mod, filename, source, isMain, mod[kModuleParent]);
+    } = cascadedLoader.importSyncForRequire(mod, filename, source, isMain, parent);
     // Tooling in the ecosystem have been using the __esModule property to recognize
     // transpiled ESM in consuming code. For example, a 'log' package written in ESM:
     //

lib/internal/modules/esm/load.js

@@ -3,7 +3,10 @@
 const {
   RegExpPrototypeExec,
 } = primordials;
-const { kEmptyObject } = require('internal/util');
+const {
+  isUnderNodeModules,
+  kEmptyObject,
+} = require('internal/util');
 
 const { defaultGetFormat } = require('internal/modules/esm/get_format');
 const { validateAttributes, emitImportAssertionWarning } = require('internal/modules/esm/assert');
@@ -14,9 +17,6 @@ const defaultType =
   getOptionValue('--experimental-default-type');
 
 const { Buffer: { from: BufferFrom } } = require('buffer');
-const {
-  isUnderNodeModules,
-} = require('internal/modules/helpers');
 
 const { URL } = require('internal/url');
 const {
@@ -131,11 +131,6 @@ async function defaultLoad(url, context = kEmptyObject) {
 
   validateAttributes(url, format, importAttributes);
 
-  // Use the synchronous commonjs translator which can deal with cycles.
-  if (format === 'commonjs' && getOptionValue('--experimental-require-module')) {
-    format = 'commonjs-sync';
-  }
-
   if (getOptionValue('--experimental-strip-types') &&
     (format === 'module-typescript' || format === 'commonjs-typescript') &&
     isUnderNodeModules(url)) {
@@ -191,11 +186,6 @@ function defaultLoadSync(url, context = kEmptyObject) {
 
   validateAttributes(url, format, importAttributes);
 
-  // Use the synchronous commonjs translator which can deal with cycles.
-  if (format === 'commonjs' && getOptionValue('--experimental-require-module')) {
-    format = 'commonjs-sync';
-  }
-
   return {
     __proto__: null,
     format,

lib/internal/modules/esm/loader.js

@@ -373,15 +373,15 @@ class ModuleLoader {
 
     defaultLoadSync ??= require('internal/modules/esm/load').defaultLoadSync;
     const loadResult = defaultLoadSync(url, { format, importAttributes });
-    const {
-      format: finalFormat,
-      source,
-    } = loadResult;
+
+    // Use the synchronous commonjs translator which can deal with cycles.
+    const finalFormat = loadResult.format === 'commonjs' ? 'commonjs-sync' : loadResult.format;
 
     if (finalFormat === 'wasm') {
       assert.fail('WASM is currently unsupported by require(esm)');
     }
 
+    const { source } = loadResult;
     const isMain = (parentURL === undefined);
     const wrap = this.#translate(url, finalFormat, source, isMain);
     assert(wrap instanceof ModuleWrap, `Translator used for require(${url}) should not be async`);

lib/internal/modules/esm/module_job.js

@@ -361,6 +361,7 @@ class ModuleJobSync extends ModuleJobBase {
   }
 
   runSync() {
+    // TODO(joyeecheung): add the error decoration logic from the async instantiate.
     this.module.instantiateSync();
     setHasStartedUserESMExecution();
     const namespace = this.module.evaluateSync();

lib/internal/modules/esm/utils.js

@@ -75,17 +75,15 @@ function initializeDefaultConditions() {
   const userConditions = getOptionValue('--conditions');
   const noAddons = getOptionValue('--no-addons');
   const addonConditions = noAddons ? [] : ['node-addons'];
-
+  const moduleConditions = getOptionValue('--experimental-require-module') ? ['module-sync'] : [];
   defaultConditions = ObjectFreeze([
     'node',
     'import',
+    ...moduleConditions,
     ...addonConditions,
     ...userConditions,
   ]);
   defaultConditionsSet = new SafeSet(defaultConditions);
-  if (getOptionValue('--experimental-require-module')) {
-    defaultConditionsSet.add('module-sync');
-  }
 }
 
 /**