Content Layer

.changeset/fresh-fans-study.md

@@ -0,0 +1,18 @@
+---
+'@astrojs/db': minor
+---
+
+Changes how type generation works
+
+The generated `.d.ts` file is now at a new location:
+
+```diff
+- .astro/db-types.d.ts
++ .astro/integrations/astro_db/db.d.ts
+```
+
+The following line can now be removed from `src/env.d.ts`:
+
+```diff
+- /// <reference path="../.astro/db-types.d.ts" />
+```

.changeset/mean-horses-kiss.md

@@ -0,0 +1,35 @@
+---
+'astro': minor
+---
+
+Adds a new [`injectTypes()` utility](https://docs.astro.build/en/reference/integrations-reference/#injecttypes-options) to the Integration API and refactors how type generation works
+
+Use `injectTypes()` in the `astro:config:done` hook to inject types into your user's project by adding a new a `*.d.ts` file.
+
+The `filename` property will be used to generate a file at `/.astro/integrations/<normalized_integration_name>/<normalized_filename>.d.ts` and must end with `".d.ts"`.
+
+The `content` property will create the body of the file, and must be valid TypeScript.
+
+Additionally, `injectTypes()` returns a URL to the normalized path so you can overwrite its content later on, or manipulate it in any way you want.
+
+```js
+// my-integration/index.js
+export default {
+  name: 'my-integration',
+  'astro:config:done': ({ injectTypes }) => {
+    injectTypes({
+      filename: "types.d.ts",
+      content: "declare module 'virtual:my-integration' {}"
+    })
+  }
+};
+```
+
+Codegen has been refactored. Although `src/env.d.ts` will continue to work as is, we recommend you update it:
+
+```diff
+- /// <reference types="astro/client" />
++ /// <reference path="../.astro/types.d.ts" />
+- /// <reference path="../.astro/env.d.ts" />
+- /// <reference path="../.astro/actions.d.ts" />
+```

.changeset/serious-pumas-run.md

@@ -0,0 +1,21 @@
+---
+'astro': minor
+---
+
+Adds support for Intellisense features (e.g. code completion, quick hints) for your content collection entries in compatible editors under the `experimental.contentIntellisense` flag.
+
+```js
+import { defineConfig } from 'astro';
+
+export default defineConfig({
+	experimental: {
+		contentIntellisense: true
+	}
+})
+```
+
+When enabled, this feature will generate and add JSON schemas to the `.astro` directory in your project. These files can be used by the Astro language server to provide Intellisense inside content files (`.md`, `.mdx`, `.mdoc`).
+
+Note that at this time, this also require enabling the `astro.content-intellisense` option in your editor, or passing the `contentIntellisense: true` initialization parameter to the Astro language server for editors using it directly.
+
+See the [experimental content Intellisense docs](https://docs.astro.build/en/reference/configuration-reference/#experimentalcontentintellisense) for more information updates as this feature develops.

.changeset/smooth-chicken-wash.md

@@ -0,0 +1,107 @@
+---
+'astro': minor
+---
+
+Adds experimental support for the Content Layer API.
+
+The new Content Layer API builds upon content collections, taking them beyond local files in `src/content/` and allowing you to fetch content from anywhere, including remote APIs. These new collections work alongside your existing content collections, and you can migrate them to the new API at your own pace. There are significant improvements to performance with large collections of local files. 
+
+### Getting started
+
+To try out the new Content Layer API, enable it in your Astro config:
+
+```js
+import { defineConfig } from 'astro';
+
+export default defineConfig({
+  experimental: {
+    contentLayer: true
+  }
+})
+```
+
+You can then create collections in your `src/content/config.ts` using the Content Layer API.
+
+### Loading your content
+
+The core of the new Content Layer API is the loader, a function that fetches content from a source and caches it in a local data store. Astro 4.14 ships with built-in `glob()` and `file()` loaders to handle your local Markdown, MDX, Markdoc, and JSON files:
+
+```ts {3,7}
+// src/content/config.ts
+import { defineCollection, z } from 'astro:content';
+import { glob } from 'astro/loaders';
+
+const blog = defineCollection({
+  // The ID is a slug generated from the path of the file relative to `base`
+  loader: glob({ pattern: "**/*.md", base: "./src/data/blog" }),
+  schema: z.object({
+    title: z.string(),
+    description: z.string(),
+    publishDate: z.coerce.date(),
+  })
+});
+
+export const collections = { blog };
+```
+
+You can then query using the existing content collections functions, and enjoy a simplified `render()` function to display your content:
+
+```astro
+---
+import { getEntry, render } from 'astro:content';
+
+const post = await getEntry('blog', Astro.params.slug);
+
+const { Content } = await render(entry);
+---
+
+<Content />
+```
+
+### Creating a loader
+
+You're not restricted to the built-in loaders – we hope you'll try building your own. You can fetch content from anywhere and return an array of entries:
+
+```ts
+// src/content/config.ts
+const countries = defineCollection({
+  loader: async () => {
+    const response = await fetch("https://restcountries.com/v3.1/all");
+    const data = await response.json();
+    // Must return an array of entries with an id property,
+    // or an object with IDs as keys and entries as values
+    return data.map((country) => ({
+      id: country.cca3,
+      ...country,
+    }));
+  },
+  // optionally add a schema to validate the data and make it type-safe for users
+  // schema: z.object...
+});
+
+export const collections = { countries };
+```
+
+For more advanced loading logic, you can define an object loader. This allows incremental updates and conditional loading, and gives full access to the data store. It also allows a loader to define its own schema, including generating it dynamically based on the source API. See the [the Content Layer API RFC](https://github.com/withastro/roadmap/blob/content-layer/proposals/0047-content-layer.md#loaders) for more details.
+
+### Sharing your loaders
+
+Loaders are better when they're shared. You can create a package that exports a loader and publish it to npm, and then anyone can use it on their site. We're excited to see what the community comes up with! To get started, [take a look at some examples](https://github.com/ascorbic/astro-loaders/). Here's how to load content using an RSS/Atom feed loader:
+
+```ts
+// src/content/config.ts
+import { defineCollection } from "astro:content";
+import { feedLoader } from "@ascorbic/feed-loader";
+
+const podcasts = defineCollection({
+  loader: feedLoader({
+    url: "https://feeds.99percentinvisible.org/99percentinvisible",
+  }),
+});
+
+export const collections = { podcasts };
+```
+
+### Learn more
+
+To find out more about using the Content Layer API, check out [the Content Layer RFC](https://github.com/withastro/roadmap/blob/content-layer/proposals/0047-content-layer.md) and [share your feedback](https://github.com/withastro/roadmap/pull/982).

benchmark/bench/memory.js

@@ -18,7 +18,7 @@ export async function run(projectDir, outputFile) {
 	const outputFilePath = fileURLToPath(outputFile);
 
 	console.log('Building and benchmarking...');
-	await execaCommand(`node --expose-gc --max_old_space_size=256 ${astroBin} build`, {
+	await execaCommand(`node --expose-gc --max_old_space_size=10000 ${astroBin} build --silent`, {
 		cwd: root,
 		stdio: 'inherit',
 		env: {

benchmark/make-project/markdown-cc1.js

@@ -0,0 +1,63 @@
+import fs from 'node:fs/promises';
+import { loremIpsumMd } from './_util.js';
+
+/**
+ * @param {URL} projectDir
+ */
+export async function run(projectDir) {
+	await fs.rm(projectDir, { recursive: true, force: true });
+	await fs.mkdir(new URL('./src/pages/blog', projectDir), { recursive: true });
+	await fs.mkdir(new URL('./src/content/blog', projectDir), { recursive: true });
+	await fs.copyFile(new URL('./image.jpg', import.meta.url), new URL('./src/image.jpg', projectDir));
+
+	const promises = [];
+
+
+	for (let i = 0; i < 10000; i++) {
+		const content = `\
+# Article ${i}
+
+${loremIpsumMd}
+
+![image ${i}](../../image.jpg)
+
+
+`;
+		promises.push(
+			fs.writeFile(new URL(`./src/content/blog/article-${i}.md`, projectDir), content, 'utf-8')
+		);
+	}
+
+
+	await fs.writeFile(
+		new URL(`./src/pages/blog/[...slug].astro`, projectDir),
+		`\
+---
+import { getCollection } from 'astro:content';
+export async function getStaticPaths() {
+  const blogEntries = await getCollection('blog');
+  return blogEntries.map(entry => ({
+    params: { slug: entry.slug }, props: { entry },
+  }));
+}
+const { entry } = Astro.props;
+const { Content } = await entry.render();
+---
+<h1>{entry.data.title}</h1>
+<Content />
+`,
+		'utf-8'
+	);
+
+	await Promise.all(promises);
+
+	await fs.writeFile(
+		new URL('./astro.config.js', projectDir),
+		`\
+import { defineConfig } from 'astro/config';
+
+export default defineConfig({
+});`,
+		'utf-8'
+	);
+}

benchmark/make-project/markdown-cc2.js

@@ -0,0 +1,80 @@
+import fs from 'node:fs/promises';
+import { loremIpsumMd } from './_util.js';
+
+/**
+ * @param {URL} projectDir
+ */
+export async function run(projectDir) {
+	await fs.rm(projectDir, { recursive: true, force: true });
+	await fs.mkdir(new URL('./src/pages/blog', projectDir), { recursive: true });
+	await fs.mkdir(new URL('./data/blog', projectDir), { recursive: true });
+	await fs.mkdir(new URL('./src/content', projectDir), { recursive: true });
+	await fs.copyFile(new URL('./image.jpg', import.meta.url), new URL('./image.jpg', projectDir));
+
+	const promises = [];
+
+	for (let i = 0; i < 10000; i++) {
+		const content = `\
+# Article ${i}
+
+${loremIpsumMd}
+
+![image ${i}](../../image.jpg)
+
+`;
+		promises.push(
+			fs.writeFile(new URL(`./data/blog/article-${i}.md`, projectDir), content, 'utf-8')
+		);
+	}
+
+	await fs.writeFile(
+		new URL(`./src/content/config.ts`, projectDir),
+		/*ts */ `
+		import { defineCollection, z } from 'astro:content';
+		import { glob } from 'astro/loaders';
+
+		const blog = defineCollection({
+			loader: glob({ pattern: '*', base: './data/blog' }),
+		});
+
+		export const collections = { blog }
+
+		`
+	);
+
+	await fs.writeFile(
+		new URL(`./src/pages/blog/[...slug].astro`, projectDir),
+		`\
+---
+import { getCollection, render } from 'astro:content';
+export async function getStaticPaths() {
+  const blogEntries = await getCollection('blog');
+  return blogEntries.map(entry => ({
+    params: { slug: entry.id }, props: { entry },
+  }));
+}
+const { entry } = Astro.props;
+const { Content } = await render(entry);
+
+---
+<h1>{entry.data.title}</h1>
+<Content />
+`,
+		'utf-8'
+	);
+
+	await Promise.all(promises);
+
+	await fs.writeFile(
+		new URL('./astro.config.js', projectDir),
+		`\
+import { defineConfig } from 'astro/config';
+
+export default defineConfig({
+		experimental: {
+			contentLayer: true
+		}
+});`,
+		'utf-8'
+	);
+}