Skip to content

Commit

Permalink
feat(compartment-mapper): Support dynamic require/importNow (#2310)
Browse files Browse the repository at this point in the history
## Description

- This PR adds support for dynamic requires via `loadFromMap()` and
`link()` (in `import-lite.js` and `link.js`, respectively).
`importLocation()`'s signature has also been modified for support.

    To use this feature in either function, the following must be true:

1. The `moduleTransforms` option (in the appropriate `options`
parameter) must not be present. These are _asynchronous_ module
transforms, which cannot be used by dynamic require.
2. The `ReadPowers` param must be a proper `ReadPowers` object (not just
a `ReadFn`) _and_ must contain both the _new_ `maybeReadNow()`, _new_
`isAbsolute`, and `fileURLToPath` functions. The new type representing
this is `ReadNowPowers`.
    
If all of the above are true, then a compartment will be allowed to
dynamically require something. If that thing still cannot be found in
the compartment map, the sync "fallback" exit hook (see next item) will
be executed.
- The new `importHookNow` property can be provided via options, which is
a synchronous exit module import hook.
- About `ReadNowPowers`: 
- `ReadNowPowers.maybeReadNow()` is necessary to read files
synchronously which is necessary to load them synchronously. This fails
gracefully if the file is not found; the implementation is
platform-dependent.
- `ReadNowPowers.isAbsolute()` is necessary to determine if the module
specifier of a dynamic require is absolute. If it is, it could be just
about anything, and must be loaded via the user-provided `importNowHook`
(the sync exit module import hook).
- `ReadNowPowers.fileURLToPath()` is needed for `__filename` and
`__dirname`, which is often used to create absolute paths for requiring
dynamically.
- As an alternative to `moduleTransforms`, _synchronous_ module
transforms may be provided via the _new_ `syncModuleTransforms` object.
In a non-dynamic-require use-case, if present, `syncModuleTransforms`
are combined with the `moduleTransforms` option; all sync module
transforms are module transforms, but not all module transforms are sync
module transforms.
- **All builtin parsers are now synchronous**. User-defined parsers
_can_ be async, but this will disable dynamic require support.
- `@endo/evasive-transform` now exports `evadeCensorSync()` in addition
to `evadeCensor()`. This is possible because I've swapped the async-only
[source-map](https://npm.im/source-map) with
[source-map-js](https://npm.im/source-map-js), which is a fork of the
former before it went async-only. `source-map-js` claims comparable
performance.

### Security Considerations

Dynamically requiring an exit module (e.g., a Node.js builtin) requires
a user-defined hook, which has the same security considerations as a
user-defined exit module hook.

Swapping out a dependency (`source-map-js` for `source-map`) incurs
risk.

### Scaling Considerations

n/a

### Documentation Considerations

Should be announced as a user-facing feature

### Testing Considerations

I've added some fixtures and tested around the conditionals I've added,
but am open to any suggestions for additional coverage.

### Compatibility Considerations

This increases ecosystem compatibility considerably; use of dynamic
require in the ecosystem is not rare.

For example, most packages which ship a native module will be using
dynamic require, because the filepath of the build artifact is dependent
upon the platform and architecture.

### Upgrade Considerations

Everything else _should_ be backwards-compatible, as long as
`source-map-js` does as it says on the tin.

Users of `@endo/evasive-transform` may note that native modules are
neither downloaded/compiled (due to the switch from `source-map` to
`source-map-js`).

* * *

## UPDATE: Aug 18 2024 

This PR now targets the `boneskull/supertramp` branch, which is PR #2263
  • Loading branch information
boneskull authored Oct 1, 2024
2 parents 797d4b5 + 572589c commit a74fcf3
Show file tree
Hide file tree
Showing 57 changed files with 2,318 additions and 463 deletions.
7 changes: 7 additions & 0 deletions packages/compartment-mapper/NEWS.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,12 @@
User-visible changes to `@endo/compartment-mapper`:

# Next release

- Adds support for dynamic requires in CommonJS modules. This requires specific
configuration to be passed in (including new read powers), and is _not_
enabled by default. See the signature of `loadFromMap()` in `import-lite.js`
for details.

# v1.2.0 (2024-07-30)

- Fixes incompatible behavior with Node.js package conditional exports #2276.
Expand Down
6 changes: 5 additions & 1 deletion packages/compartment-mapper/node-powers.js
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
export { makeReadPowers, makeWritePowers } from './src/node-powers.js';
export {
makeReadPowers,
makeWritePowers,
makeReadNowPowers,
} from './src/node-powers.js';
// Deprecated:
export { makeNodeReadPowers, makeNodeWritePowers } from './src/node-powers.js';
1 change: 1 addition & 0 deletions packages/compartment-mapper/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,7 @@
"dependencies": {
"@endo/cjs-module-analyzer": "workspace:^",
"@endo/module-source": "workspace:^",
"@endo/trampoline": "workspace:^",
"@endo/zip": "workspace:^",
"ses": "workspace:^"
},
Expand Down
Loading

0 comments on commit a74fcf3

Please sign in to comment.