Typing this in unified plugin (specifically compiler)

[senicko]

Hello! I am writing a custom compiler for markdown AST. I can't figure out how to type this properly (to enforce that CompileTree is Root and CompileResult is my custom type. This is code which I have tried.

type TTS = { input: Array<string>; frontmatter: Record<string, string> };

declare module "unified" {
  interface CompileResultMap {
    TTS: TTS;
  }
}

function ttsCompiler(
  this: Processor<undefined, undefined, undefined, Root, TTS>
) {
  this.compiler = (ast) => ({ input: [], frontmatter: {} });
}

const ast = await unified()
  .use(remarkParse)
  .use(remarkMDX)
  .use(remarkFrontmatter)
  //  ^^^^^^^^^^^^^^^^^^^ this gives the following error
  .use(ttsCompiler)
  .process(rawMDX);

No overload matches this call.
  Overload 1 of 3, '(preset?: Preset | null | undefined): Processor<Root, undefined, undefined, undefined, undefined>', gave the following error.
    Type '(this: Processor<undefined, undefined, undefined, Root, TTS>) => void' has no properties in common with type 'Preset'.
  Overload 2 of 3, '(list: PluggableList): Processor<Root, undefined, undefined, undefined, undefined>', gave the following error.
    Argument of type '(this: Processor<undefined, undefined, undefined, Root, TTS>) => void' is not assignable to parameter of type 'PluggableList'.
  Overload 3 of 3, '(plugin: Plugin<[], undefined, undefined>, ...parameters: [] | [boolean]): Processor<Root, undefined, undefined, undefined, undefined>', gave the following error.
    Argument of type '(this: Processor<undefined, undefined, undefined, Root, TTS>) => void' is not assignable to parameter of type 'Plugin<[], undefined, undefined>'.
      The 'this' types of each signature are incompatible.
        Type 'Processor<undefined, undefined, undefined, undefined, undefined>' is not assignable to type 'Processor<undefined, undefined, undefined, Root, TTS>'.
          Type 'Root' is not assignable to type 'undefined'.ts(2769)

I assume that's because I can't type this like this in my ttsCompile function. But how am I supposed to do that? I looked into source of remarkStringify and they do something like this.

export default function remarkStringify(options) {
  /** @type {Processor} */
  // @ts-expect-error: TS in JSDoc generates wrong types if `this` is typed regularly.
  const self = this

When I assign this to self in my code like that it "works" fine too (I don't get any errors). Is this the proper solution? I really appreciate any tips!

Also, can I somehow make the output to be of the TTS type? The outputted vfile's result is still unknown and I can't see any generic type in which I could pass my custom TTS type. Is this even possible or I just need to rely on casts?

[wooorm]

Hey!

Use Plugin<Parameters, string, TheTypeYouWant> to type your plugin that adds a particular compiler.
That should do the trick.

This is a bit weird because:

a) the this you want in the plugin could actually be any current unified processor; it could have some compiler already; your plugin is overwriting it. So we can’t “get” which output you intend to compile to from a particular this

b) TS for JS is buggy with the this parameter: IIRC it was that it “works” in the file but results in different .d.ts files.

Also, can I somehow make the output to be of the TTS type? The outputted vfile's result is still unknown and I can't see any generic type in which I could pass my custom TTS type. Is this even possible or I just need to rely on casts?

IIRC what you want should indeed happen.

[senicko]

Hmm, I still have some issues with this approach. This is my current code

export type CompiledArticle = {
  sections: Array<string>;
  frontmatter: ArticleFrontmatter;
};

const ttsCompiler: Plugin<[], Root, CompiledArticle> = function () {
  this.compiler = (ast, file) => {
    const frontmatter = file.data.frontmatter as ArticleFrontmatter;
    const sections = [frontmatter.title + "\n\n"];

    for (const child of ast.children) {
      //                    ^^^^^^^^ <------------------------ this fails as ast has type unist.Node and not mdast.Root
      const text = toString(child) + "\n\n";

      if (sections[sections.length - 1].length + text.length <= 4096) {
        sections[sections.length - 1] += text;
      } else {
        sections.push(text);
      }
    }

    return { sections, frontmatter };
  };
};

Property 'children' does not exist on type 'Node'.

Is there any way to say "I want a compiler that receives mdast.Root and returns CompiledArticle"?

[ChristianMurphy]

Heya @senicko! 👋
A few pointers

---

const frontmatter = file.data.frontmatter as ArticleFrontmatter;

I'd highly discourage using a force cast on a type, authors may not perfectly capture the json/yml in their front matter.
Consider using a type validator like https://zod.dev/ to ensure the shape of the data is what you expect, and give clear error messaging if it is not.

---

for (const child of ast.children) {
      //                    ^^^^^^^^ <------------------------ this fails as ast has type unist.Node and not mdast.Root

This is accurate, a compiler is likely, but not guaranteed to receive a root node.
You'll want assert here using the ideas from https://unifiedjs.com/learn/recipe/narrow-node-typescript/
Most likely something like

import type {Root} from 'mdast'
import {is} from 'unist-util-is'

// ...

if (is<Root>(ast, 'root')) {
  // ast is known to be root in this context
  for (const child of ast.children) {
    // do your logic
  }
} else {
  // do some error handling
}

[senicko]

1. Fully agreed, I will use zod

2. Actually, it is not possible to do this. I have opened separate discussion thread in syntax-tree https://github.com/orgs/syntax-tree/discussions/128#discussioncomment-9618585 where you advised me to write plugins for my logic. When you pass mdast.Root as a generic argument to is you can no longer use "root" as a Test. I am probably not understanding something here 😟

I hope the snippet has everything that's relevant to this problem ;d

import type { Root } from "mdast";
import { is } from "unist-util-is";

declare module "unified" {
  interface CompileResultMap {
    // Register a new result (value is used, key should match it).
    CompiledArticle: CompiledArticle;
  }
}

const ttsCompiler: Plugin<[], Root, CompiledArticle> = function () {
  this.compiler = (ast, file) => {
    if (is<Root>(ast, "root")) {
      //             ^^^^^^^^ (1)
      for (const child of ast.children) {
        //                ^^^^^^^^^^^^ (2)
      }

      return { sections: [], frontmatter: { title: "", slug: "" } };
    } else {
      throw new Error("ttsCompiler can be run only on mdast.Root");
    }
  };
};

/** compileArticle extract frontmatter and ttsInput from an article. */
export async function compileArticle(rawMDX: string): Promise<CompiledArticle> {
  const vfile = await unified()
    .use(remarkParse)
    .use(remarkMDX)
    .use(remarkFrontmatter)
    .use(ttsTransformer)
    .use(remarkUnlink)
    .use(ttsCompiler)
    .process(rawMDX);

  return vfile.result;
}

(1):

Argument of type 'string' is not assignable to parameter of type 'Root'

(2):

Property 'children' does not exist on type 'Node'.

I really appreciate your help. I would like to get this working so much 😆

[wooorm]

Why are you even trying to create a custom compiler that returns a non-string result?
In your compiler, you access and later return file.data.frontmatter.
Anyone outside of your customization can already access file.data.frontmatter.

Is there any way to say "I want a compiler that receives mdast.Root and returns CompiledArticle"?

Yes, by typing your compiler function too.
Type both the plugin and compiler

[senicko]

This is the reproduction which still has the error.

(edit): I've removed the compileArticle function from previous snippets as it does not change anything in terms of the produced type error.

import type { Root } from "mdast";
import remarkMDX from "remark-mdx";
import remarkParse from "remark-parse";
import { unified, type Plugin } from "unified";
import type { VFile } from "vfile";

export type CompiledArticle = {
  sections: Array<string>;
};

declare module "unified" {
  interface CompileResultMap {
    // Register a new result (value is used, key should match it).
    CompiledArticle: CompiledArticle;
  }
}

const ttsCompiler: Plugin<[], Root, CompiledArticle> = function () {
  this.compiler = compiler;

  function compiler(ast: Root, file: VFile): CompiledArticle {
    return { sections: [] };
  }
};

const rawMDX = `
import Caption from "../../components/Caption.astro";
import CodeSnippet from "../../components/CodeSnippet.astro";

## Introduction
`;

const vfile = await unified()
  .use(remarkParse)
  .use(remarkMDX)
  .use(ttsCompiler)
  .process(rawMDX);

console.log(vfile.result);

Type '(ast: Root, file: VFile) => CompiledArticle' is not assignable to type 'Compiler<Node, CompileResults>'.
  Types of parameters 'ast' and 'tree' are incompatible.
    Property 'children' is missing in type 'Node' but required in type 'Root'.

[wooorm]

repro needs package.json + tsconfig + filenames + commands for me to be able to repro

[senicko]

@wooorm here you go
https://github.com/senicko/unified-compiler-plugin-reproduction

[wooorm]

Using index.ts and latest packages, VS code does show that type error.
I recommend adding at @ts-expect-error:

const ttsCompiler: Plugin<[], Root, CompiledArticle> = function () {
+ // @ts-expect-error
  this.compiler = compiler;

Or cast:

- function compiler(ast: Root, file: VFile): CompiledArticle {
+ function compiler(ast: Node, file: VFile): CompiledArticle {
+   const tree = ast as Root
    return { sections: [] };
  }

The unified types seem to tie a) what the current compiler could be, with b) the new compiler intended to be set.

[senicko]

Okay, thanks a lot. @ChristianMurphy's approach seems to be the most "valid" one, but I feel like is type is broken (? maybe it works like it's supposed to, but certainly doesn't allow to narrow ts type in the way it's show in recipe, or I am missing something). Whatever, I really appreciate your help @wooorm @ChristianMurphy

Have a nice day :)