Merge branch 'main' into fil/duckdb-extension

Author: Fil

docs/deploying.md

@@ -1,40 +1,48 @@
 # Deploying
 
-When time comes to share your project, you have many options for deploying it for others to see. Framework is compatible with many static site hosts and automation environments. In this guide we’ll focus on deploying to Observable manually, then with GitHub Actions.
+You can host your built Framework project on any static site hosting service, or self-host it with any static site server. This guide covers deploying to [Observable](https://observablehq.com), which is the easiest way to host your Framework project as support is built-in. We’ll also cover setting up automated deploys with GitHub Actions.
 
-## Manually deploying to Observable
+<div class="tip">
 
-If you don’t already have a project to deploy, you can create one by following [getting-started](./getting-started). First, make sure that your project builds without error:
+If you don’t already have a project ready to deploy, create one by following our [Getting started guide](./getting-started).
+
+</div>
+
+## Manual deploys
+
+First, make sure that your project builds without error:
 
 ```sh
-$ npm run build
+npm run build
 ```
 
-Once that is done you can deploy to Observable with the command
+Once that is done you can deploy to Observable:
 
 ```sh
-$ npm run deploy
+npm run deploy
 ```
 
-The first time you run this command, you will be prompted for details needed to set up the project on the server, such as the project's _slug_ (which determines its URL), and the access level. If you don’t already have an Observable account or aren’t signed in, this command will also guide you through setting that up.
+The first time you deploy a project, you will be prompted to configure the project’s _slug_ (which determines its URL), access level, and other details. If you don’t yet have an Observable account or aren’t signed-in, you will also be prompted to sign-up or sign-in.
 
-When the deploy command finishes, it prints a link to observablehq.cloud where you can view your deployed project. If you choose “private” as the access level, you can now share that link with anyone who is a member of  your workspace. If you chose “public”, you can share that link with anyone and they’ll be able to see your Framework project.
+When the deploy command finishes, it prints a link to observablehq.cloud where you can view your deployed project. If you choose *private* as the access level, that link will only be accessible to members of your Observable workspace. (You can invite people to your workspace by going to observablehq.com.) If you chose *public*, you can share your project link with anyone.
 
-<div class="note">The deploy command creates a file at <code>docs/.observablehq/deploy.json</code> with information on where to deploy the project. It is required for automated deploys. You should commit it to git to make it available to GitHub Actions. (If you have configured a source root besides <code>docs/</code>, the file will be placed there instead.)</div>
+<div class="note">The deploy command creates a file at <code>docs/.observablehq/deploy.json</code> with information on where to deploy the project. This file is required for automated deploys. You will need to commit this file to git to deploy via GitHub Actions. (If you have configured a source root besides <code>docs</code>, the file will be placed there instead.)</div>
 
-## Automated deploys to Observable
+## Automated deploys
 
-To set up automatic deploys, we’ll be using [GitHub actions](https://github.com/features/actions). In your git repository, create and commit a file at `.github/workflows/deploy.yml`. Here is a starting example:
+To set up automatic deploys (also known as *continuous deployment* or *CD*), we recommend [GitHub Actions](https://github.com/features/actions). In your git repository, create and commit a file at `.github/workflows/deploy.yml`. Here is a starting example:
 
 ```yaml
 name: Deploy
+
 on:
   # Run this workflow whenever a new commit is pushed to main.
   push: {branches: [main]}
   # Run this workflow once per day, at 10:15 UTC
   schedule: [{cron: "15 10 * * *"}]
-  # Run this workflow when triggered manually in GitHub's UI.
+  # Run this workflow when triggered manually in GitHub’s UI.
   workflow_dispatch: {}
+
 jobs:
   deploy:
     runs-on: ubuntu-latest
@@ -54,15 +62,39 @@ jobs:
           OBSERVABLE_TOKEN: ${{ secrets.OBSERVABLE_TOKEN }}
 ```
 
-When deploying automatically, you won’t be able to login with a browser the way you did for manual deploys. Instead, you will authenticate via the environment variable `OBSERVABLE_TOKEN`, using an API key from Observable.
+When deploying automatically, you can’t sign-in in your browser the way you did for manual deploys; instead, your GitHub action will authenticate using an Observable API key (also known as a *token* and referred to as `OBSERVABLE_TOKEN` above).
+
+To create an API key:
+
+1. Go to [observablehq.com](https://observablehq.com).
+2. In the left sidebar, click **Settings**.
+3. In the right sidebar, click **API / Notebook Keys**.
+4. Click **New API Key**.
+5. Check the **Deploy new versions of projects** checkbox.
+6. Give your key a description, such as “Deploy via GitHub Actions”.
+7. Click **Create API Key**.
 
-To create a token, go to https://observablehq.com and open your workspace settings. Choose “API keys”. From there, create a new key, and assign it the "Deploy new versions of projects" scope.
+<div class="caution">
 
-That token is the equivalent of a password giving write access to your hosted project. **Do not commit it to git** (and, if it is exposed in any way, take a minute to revoke it and create a new one instead—or contact support).
+The token you create is the equivalent of a password giving write access to your hosted project. **Do not commit it to git** or share it with anyone you don’t trust. If you accidentally expose your key, you can go back to your settings to immediately revoke it (and create a new key).
 
-To pass this information securely to the Github action (so it can effectively be authorized to deploy the project to Observable), we’ll use GitHub secrets. Sign in to the settings of your GitHub account, and add a secret named `OBSERVABLE_TOKEN`. See [GitHub’s documentation](https://docs.github.com/en/actions/security-guides/using-secrets-in-github-actions) for more information about secrets.
+</div>
 
-This `deploy.yml` will automatically build and deploy your project once per day (to keep your data up-to-date), as well as whenever you push a new version of the code to your repository (so you can make changes at any time).
+In a moment, you’ll copy-paste your new Observable API key, so leave this window open for now. (If you accidentally close the window, you can delete the old key and create a new one. For security, API keys are only shown once upon creation.)
+
+To authenticate with Observable and to access the Observable API key securely from our Github action, we’ll use a [GitHub secret](https://docs.github.com/en/actions/security-guides/using-secrets-in-github-actions).
+
+To create a GitHub secret, in a new window:
+
+1. Go to your GitHub repository.
+2. In the top navigation, click **Settings**.
+3. In the left sidebar, click **Secrets and variables**, then **Actions**.
+4. Click **New repository secret**.
+5. In the **Name** field, enter `OBSERVABLE_TOKEN`.
+6. In the **Secret** field, paste the API key you created on Observable.
+7. Click **Add secret**.
+
+After you’ve performed these steps, the `deploy.yml` above will automatically build and deploy your project once per day (to keep your data up-to-date), as well as whenever you push a new version of the code to your repository (so you can make changes at any time).
 
 ### Caching
 
@@ -85,16 +117,22 @@ jobs:
       # ...
 ```
 
-This uses one cache per calendar day (in the “America/Los_Angeles” time zone). If you deploy multiple times in a day, the results of your data loaders will be reused on the second and subsequent runs. You can customize the `date` and `cache-data` steps to change the cadence of the caching. For example you could use `date +'%Y-%U'` to cache data for a week or `date +'%Y-%m-%dT%H` to cache it for only an hour.
+This uses one cache per calendar day (in the `America/Los_Angeles` time zone). If you deploy multiple times in a day, the results of your data loaders will be reused on the second and subsequent runs. You can customize the `date` and `cache-data` steps to change the cadence of the caching. For example you could use `date +'%Y-%U'` to cache data for a week or `date +'%Y-%m-%dT%H` to cache it for only an hour.
 
-<div class="note">You’ll need to change the paths used in this config if <code>observablehq.config.js</code> points to a different <code>root</code>.</div>
+<div class="note">You’ll need to edit the paths above if you’ve configured a source root other than <code>docs</code>.</div>
 
-## Deploying to other services
+## Other hosting services
 
-The output of Observable Framework is set of static files that can be hosted by many services. To build a hostable copy of your project, run:
+Observable Framework builds a set of static files that can be hosted by any static site hosting services. To build your project, run:
 
 ```sh
-$ npm run build
+npm run build
 ```
 
-Then you can upload the contents of your `dist/` directory to your static webhost of choice. Some webhosts may need the `cleanUrls` option <a href="https://github.com/observablehq/framework/releases/tag/v1.3.0" target="_blank" class="observablehq-version-badge" data-version="^1.3.0" title="Added in v1.3.0"></a> set to false in your project configuration file. For details and other options, see [configuration](./config).
+Then upload the contents of your `dist` directory to your static service of choice.
+
+<div class="tip">By default, Framework generates “clean” URLs by dropping the `.html` extension from page links. Not all webhosts support this; some need the <a href="./config#cleanUrls"><b>cleanUrls</b> config option</a> set to false.</div>
+
+<div class="tip">When deploying to GitHub Pages without using GitHub’s related actions (<a href="https://github.com/actions/configure-pages">configure-pages</a>,
+<a href="https://github.com/actions/deploy-pages">deploy-pages</a>, and
+<a href="https://github.com/actions/upload-pages-artifact">upload-pages-artifact</a>), you may need to create a <code>.nojekyll</code> file in your <code>dist</code> folder after building. See GitHub’s documentation on <a href="https://docs.github.com/en/pages/getting-started-with-github-pages/about-github-pages#static-site-generators">static site generators</a> for more.</div>

docs/getting-started.md

@@ -209,7 +209,7 @@ Next, `cd` into your new project folder.
 
 Framework’s local development server lets you preview your site in the browser as you make rapid changes. The preview server generates pages on-the-fly: as you edit files in your editor, changes are instantly streamed to your browser.
 
-<div class="tip">You can work offline with the preview server, but you must be connected to the internet to <a href="./javascript/imports">import libraries</a> from npm. In the future, we intend to support self-hosting imported libraries; please upvote <a href="https://github.com/observablehq/framework/issues/20">#20</a> and <a href="https://github.com/observablehq/framework/issues/360">#360</a> if you are interested in this feature.</div>
+<div class="tip">You can develop offline, but you must connect to the internet to <a href="./javascript/imports">import new libraries</a> from npm. Framework self-hosts imports: modules are downloaded from npm and stored in a local cache the first time you import them, and are included in <code>dist</code> when you build your project. In the future, we also plan to support importing from <code>node_modules</code>; please upvote <a href="https://github.com/observablehq/framework/issues/360">#360</a> if you are interested in this feature.</div>
 
 To start the preview server using npm:
 

docs/javascript/files.md

@@ -10,7 +10,7 @@ Load files — whether static or generated dynamically by a [data loader](../loa
 import {FileAttachment} from "npm:@observablehq/stdlib";
 ```
 
-The `FileAttachment` function takes a path and returns a file handle. This handle exposes the file’s name and [MIME type](https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/MIME_types).
+The `FileAttachment` function takes a path and returns a file handle. This handle exposes the file’s name, [MIME type](https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/MIME_types), and last modification date as the number of milliseconds since UNIX epoch.
 
 ```js echo
 FileAttachment("volcano.json")
@@ -32,7 +32,7 @@ volcano
 
 ### Static analysis
 
-The `FileAttachment` function can _only_ be passed a static string literal; constructing a dynamic path such as `FileAttachment("my" + "file.csv")` is invalid syntax. Static analysis is used to invoke [data loaders](../loaders) at build time, and ensures that only referenced files are included in the generated output during build. In the future [#260](https://github.com/observablehq/framework/issues/260), it will also allow content hashes for cache breaking during deploy.
+The `FileAttachment` function can _only_ be passed a static string literal; constructing a dynamic path such as `FileAttachment("my" + "file.csv")` is invalid syntax. Static analysis is used to invoke [data loaders](../loaders) at build time, and ensures that only referenced files are included in the generated output during build. This also allows a content hash in the file name for cache breaking during deploy.
 
 If you have multiple files, you can enumerate them explicitly like so:
 
@@ -52,6 +52,8 @@ const frames = [
 
 None of the files in `frames` above are loaded until a [content method](#supported-formats) is invoked, for example by saying `frames[0].image()`.
 
+For missing files, `file.lastModified` is undefined. The `file.mimeType` is determined by checking the file extension against the [`mime-db` media type database](https://github.com/jshttp/mime-db); it defaults to `application/octet-stream`.
+
 ## Supported formats
 
 `FileAttachment` supports a variety of methods for loading file contents:

docs/javascript/imports.md

@@ -108,7 +108,7 @@ In addition to resolving versions of directly-imported modules, Observable Frame
 <link rel="modulepreload" href="https://cdn.jsdelivr.net/npm/[email protected]/+esm">
 ```
 
-<div class="note">We’d also like to download imported modules from the CDN during build, making the built site entirely self-contained; see <a href="https://github.com/observablehq/framework/issues/20">#20</a>. This would further enable subresource integrity hashes; see <a href="https://github.com/observablehq/framework/issues/306">#306</a>.</div>
+<div class="note">Framework automatically downloads <code>npm:</code> imports from npm during preview and build, making the built site entirely self-contained. If you prefer not to self-host a module, and instead load it from an external server at runtime, import a full URL instead of using the <code>npm:</code> protocol.</div>
 
 ## Implicit imports
 

docs/routing.md

@@ -157,14 +157,14 @@ The resulting output root is:
 .
 ├─ dist
 │  ├─ _import
-│  │  └─ chart.js
+│  │  └─ chart.c79c2048.js
 │  ├─ _observablehq
 │  │  └─ ... # additional assets for serving the site
 │  └─ index.html
 └─ ...
 ```
 
-The import declaration is automatically rewritten during build to point to `./_import/chart.js` instead of `./chart.js`. (In the future [#260](https://github.com/observablehq/framework/issues/260), Observable Framework will add a content hash to the imported module name for cache-breaking.)
+The import declaration is automatically rewritten during build to point to `./_import/chart.c79c2048.js` instead of `./chart.js`. The content hash `c79c2048` ensures cache-breaking during deploy, and allows assets to be marked as `cache-control: immutable` to improve performance.
 
 Use a leading slash to denote paths relative to the source root, such as `/chart.js` instead of `./chart.js`. This allows you to use the same path to import a module from anywhere, even in nested folders. Observable Framework always generates relative links so that the generated site can be served under a base path.
 
@@ -186,18 +186,18 @@ Any files referenced by `FileAttachment` will automatically be copied to the `_f
 .
 ├─ dist
 │  ├─ _file
-│  │  └─ quakes.csv
+│  │  └─ quakes.e5f2eb94.csv
 │  ├─ _observablehq
 │  │  └─ ... # additional assets for serving the site
 │  └─ index.html
 └─ ...
 ```
 
-`FileAttachment` references are automatically rewritten during build; for example, a reference to `quakes.csv` might be replaced with `_file/quakes.csv`. (In the future [#260](https://github.com/observablehq/framework/issues/260), Observable Framework will add a content hash to the attached file name for cache-breaking.) Only the files you reference statically are copied to the output root (`dist`), so nothing extra or unused is included in the built site.
+`FileAttachment` references are automatically rewritten during build; for example, a reference to `quakes.csv` might be replaced with `_file/quakes.e5f2eb94.csv`. (As with imports, file names are given a content hash, here `e5f2eb94`, to improve performance.) Only the files you reference statically are copied to the output root (`dist`), so nothing extra or unused is included in the built site.
 
 [Imported modules](#imports) can use `FileAttachment`, too. In this case, the path to the file is _relative to the importing module_ in the same fashion as `import`; this is accomplished by resolving relative paths at runtime with [`import.meta.url`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/import.meta).
 
-Some additional assets are automatically promoted to file attachments and copied to `_file`. For example, if you have a `<link rel="stylesheet" href="style.css">` declared statically in a Markdown page, the `style.css` file will be copied to `_file`, too. The HTML elements eligible for file attachments are `audio`, `img`, `link`, `picture`, and `video`.
+Some additional assets are automatically promoted to file attachments and copied to `_file`. For example, if you have a `<link rel="stylesheet" href="style.css">` declared statically in a Markdown page, the `style.css` file will be copied to `_file`, too (and the file name given a content hash). The HTML elements eligible for file attachments are `audio`, `img`, `link`, `picture`, and `video`.
 
 ## Data loaders
 
@@ -223,7 +223,7 @@ This will produce the following output root:
 .
 ├─ dist
 │  ├─ _file
-│  │  └─ quakes.json
+│  │  └─ quakes.99da78d9.json
 │  ├─ _observablehq
 │  │  └─ ... # additional assets for serving the site
 │  └─ index.html
@@ -240,7 +240,7 @@ File attachments can be also be pulled from archives. The following archive exte
 - `.tar` - for [tarballs](<https://en.wikipedia.org/wiki/Tar_(computing)>)
 - `.tar.gz` and `.tgz` - for [compressed tarballs](https://en.wikipedia.org/wiki/Gzip)
 
-For example, say you have a `quakes.zip` archive that includes yearly files for observed earthquakes. If you reference `FileAttachment("quakes/2021.csv")` in code, Observable Framework will pull the `2021.csv` from `quakes.zip`. So this source root:
+For example, say you have a `quakes.zip` archive that includes yearly files for observed earthquakes. If you reference `FileAttachment("quakes/2021.csv")`, Framework will pull the `2021.csv` from `quakes.zip`. So this source root:
 
 ```ini
 .
@@ -257,7 +257,7 @@ Becomes this output:
 ├─ dist
 │  ├─ _file
 │  │  └─ quakes
-│  │     └─ 2021.csv
+│  │     └─ 2021.e5f2eb94.csv
 │  ├─ _observablehq
 │  │  └─ ... # additional assets for serving the site
 │  └─ index.html