Arbitrary dependencies syntax+approach

[mpeterdev]

Arbitrary dependency support is one of the most impactful features planned for BWE. The implementation could take a number of different forms. Let's get started on syntax design

The solution should:

1. enable importing any NPM dependency without vetting or an allowlist of any sort
2. leverage a package provider CDN which enables decentralization. bonus points if the implementation can be provider-agnostic
3. ideally not include syntax which does not look like sane JS/TS and would require a custom language syntax plugin to load component files in code editors like VS Code

The solution need not:

1. enable importing packages which conflict with the architecture of BWE (e.g. passing a function as a prop across a sandbox boundary). If a package does conflict we do not need to prevent import but behavior would be undefined
2. support alternative frontend frameworks—i.e. arbitrary dependencies is not a path to using Vue over React

[mpeterdev]

we may be able to draw inspiration from Deno since it relies on url-based npm imports

https://docs.deno.com/runtime/manual/node/cdns
https://docs.deno.com/runtime/manual/basics/import_maps

[mpeterdev]

Overall BOS Component Syntax

The design of imports will be heavily affected by the decision on whether to continue writing BOS components as only the body of a function component or move to writing the full function declaration. Imports must be declared at the top level in JS, so there is no obvious syntax when writing components in their current form

Writing the full function declaration would offer some unrelated advantages (e.g. TypeScript, prop handling) so it is already under consideration

Full URL vs Package-only

For each syntax variation, we can consider two cases: specifying the full dependency URL or specifying only the package+version. If specifying only the package+version, the full dependency URL would be built by either using a package provider configured elsewhere (i.e. gateway settings) or by having the developer create an import map to be applied to their component code.

import map example:

{
  "imports": {
    "moment": "https://esm.sh/[email protected]"
  }
}

Specifying the full URL has simplicity advantages and provides flexibility for developers, but package-only would likely be more robust since multiple package-providers could be used leading to stronger decentralization of the system.

If a single package provider going offline or being inaccessible in a certain region prevents a BOS component from rendering, then that negates much of the decentralization benefit of BOS.

Brainstorming

Function body only

All of the following would be stripped from the source before loading it into the iframe

Mock function call

Treats import like a function taking a string param of the package

const moment = import('[email protected]');
//or
const moment = import('https://esm.sh/[email protected]');

Could be confusing since users may try to use a variable as the param but that would not be allowed since imports must be handled statically

Comment

Specifies the package in a comment above a variable which will reference the import after compilation

// @import [email protected]
const moment;

//or

// @import https://esm.sh/[email protected]
const moment;

Full function declaration

This approach is pretty straightforward and fulfills our goal of being minimally different from vanilla React. Essentially, BOS component source would become a full module and look like a JSX file.

import moment from "moment";
// or
import moment from "https://esm.sh/[email protected]"

export function Component(props) {
  return <div>{moment().format("[Today is] dddd")}</div>;
}

[andy-haynes]

Excellent writeup ❤️

Full URL vs Package-only

I like leaving the CDN host as runtime configuration, but I'm concerned there could be too much variability in how bundles are served and where they are available. It may not make sense for us to maintain compatibility matrices if that's what it comes down to.

In any case, giving developers the option to override with an explicit URL would be my preferred approach here. Also a big fan of developer-maintained importmaps that get injected at compile time. Providing developers the option to maintain dependencies centrally across all their Components simplifies maintenance and promotes consistency between interdependent Components.

Mock function call

I agree that this approach could be confusing. Invoking it gives the impression that it's a dynamic import, which is basically the one thing our implementation won't be 😄

Comment

I like this approach from an implementation perspective - it's easily parseable and easy to reconcile. But it does feel a bit too much like an interim solution.

Full function declaration

I think this is my favorite. It requires some exploration in Component compilation but seems viable. The export would be meaningless in our context but we should be able to use the AST to derive the root Component.

I am a little concerned about what happens in module scope between the imports and function declaration, but this can always be ignored (to the detriment of any Component code that does reference it). However supporting this also opens the door to defining multiple React components inside a single BOS Component's source, which could be useful as a DX enhancement.

[calebjacob]

Great writeup! I'm on the same page as @andy-haynes. Full function declaration would definitely be my vote:

import moment from "moment";
// or
import moment from "https://esm.sh/[email protected]"

In general, I think it's really important to match the syntax of a typical React application as much as possible.

Definitely agree with Andy here on import maps and explicit URL imports. Supporting both would be great:

In any case, giving developers the option to override with an explicit URL would be my preferred approach here. Also a big fan of developer-maintained importmaps that get injected at compile time. Providing developers the option to maintain dependencies centrally across all their Components simplifies maintenance and promotes consistency between interdependent Components.

[mpeterdev]

Seems like we're all on the same page here that full function declaration is the preferred solution and can proceed with implementation exploration

[andy-haynes]

Now that the import syntax is supported, I'm ready to get this implemented. Given that this isn't a real export functionality I think it makes sense to support a simple subset of the syntax (the rules are exhaustive). Then the BOS Components can at least be valid JS, even if it's not 100% true the other way around.

I'm thinking we support only the following for this implementation:

• export default [function] X; this is my preferred and would be ideal if we want to support only one syntax; added bonus that this can only appear once in valid JS
• export function X... also straightforward to parse, though it introduces an incongruity with JS (which can specify multiple exports) as BOS Components would only export one
• export const X = ... same as above

[calebjacob]

If BOS components can only have a single export, I almost wonder if we should only allow the export default ... syntax.

Whenever we support TypeScript, something to consider would be the common patterns of exporting and importing types across different components/files. Not sure if that needs to affect any of our decisions here, but just something to consider.

[andy-haynes]

I think you're right that the export default approach is the most appropriate for our use case. Named exports could work if we require that the module exports a Component named BWEComponent, which would probably be a useful interface for developers, e.g. CI bundling for BOS.

We do actually support TypeScript now, and that's an excellent point. The current implementation does things a little out of order to handle JSX bodies instead of functions in that it wraps the function body before being transpiled. Now that we're working with modules, I imagine the entire thing needs to be transpiled before being wrapped in a function (which is now just a closure to prevent module scope collision between Components).

[andy-haynes]

@calebjacob #158 has this implemented if you want to take a look: https://bos-web-engine-git-feat-require-export-near-developer-console.vercel.app/andyh.near/Export