Possible to provide a few illustrative examples?

[jehillert]

First, let me say this ecosystem looks awesome and it's going to solve my use case perfectly. So thank you for creating it.

But... The documentation is brutal. I have only been able to find a few examples that show how the most popular plugins and utilities can work together. Most of them just show one example of a plugin or utility used in isolation, and often without enough context to reproduce without a little trial and error.

If anybody could point me to some detailed examples online, that would be greatly appreciated. Or if you have something nice and lengthy from your own code, say a bunch of different plugins and utilities chained together, or composed according to a recommended pattern - pasting it here would be really helpful. With or without comments, either way is fine. It doesn't even have to be an efficient way of doing things, sometimes the long way around is a good way to learn. As long as the example works and I see 2-3 of them I can probably infer most of what I need to know.

If it helps to narrow down, I am most interested in how to assign specific classnames to a set of search results for an HTML document that is fed to rehype-react. I want to decorate the results in batches but only cause a single rerender.

[wooorm]

Heya!

Every repo has an example. Every repo also has tons of tests. They all work. There are more than 500 repos, a bunch of sites, we use our own tools, so you should be able to find what you want. But spend some more time researching and reading and tryhing.

These projects relate to complex computer science topics. Parsers and ASTs. Folks study those topics for years. It’s not something you can get by copy/pasting an example.

There are also many reasons people come to these projects: to build new things that can’t be made in other ways. There are many different ways to combine 700+ packages. We can’t make an example for most of those cases.

Your practical question is extremely niche. Nobody has ever asked that to my knowledge before. Perhaps you can spend time framing your question. It’s much easier to help you if you spend more time on your question.

[ChristianMurphy]

Adding on there is some potentially related discussion in https://github.com/orgs/unifiedjs/discussions/109
This does feel pretty niche.

There are some foundational examples in https://unifiedjs.com/learn/
and many example of different rehype plugins in https://github.com/rehypejs/rehype/blob/main/doc/plugins.md

I'd also second the recommendation

Perhaps you can spend time framing your question. It’s much easier to help you if you spend more time on your question.

https://github.com/unifiedjs/.github/blob/main/support.md has some good advice.
I'd also consider reading https://whatwg.org/faq#adding-new-features especially #1-4 as another guide for framing.

[jehillert]

Hey,
Thanks for the response, but I am started figuring out everything I needed today. I will close this out.

[jehillert]

For the record, this is what I was trying to do. Without punching holes through the virtual DOM.

Screen-2024-02-28-164645.mp4

Seems to be exactly what I was looking for.

[ChristianMurphy]

For the record, this is what I was trying to do. Without punching holes through the virtual DOM.
I am most interested in how to assign specific classnames to a set of search results for an HTML document that is fed to rehype-react. I want to decorate the results in batches but only cause a single rerender.

It sounds a bit like you are looking for https://github.com/bvaughn/react-highlight-words#readme ?
I'd avoid wrapping a virtual dom (hast) around an virtual dom (react) for perceived performance reasons, adding more abstraction is more likely to hurt than help.

But there may be other considerations at play which haven't been articulated, and I'm glad that unified and rehype are meeting those needs.

[jehillert]

The component react-highlight-words provides accepts a body of text in a prop to highlight text in. I don't think you can wrap it around a full HTML markup. But what I ended up doing (yesterday) is not so much wrapping one virtual dom around another as it is using the AST to manage a static markup that is injected into the react app and that will be the target of multiple find and replace operations, without creating blind spots in the virtual dom. I want to be able to make find and replace operations on parts of the injected document but also keep the injected document fully tracked by react. React has or had an API that worked like queryselector, but it's deprecated. And it wasn't helpful because you find the element and use React.createRoot to make it's replacement, but the main react tree is not aware of that new root. createRoot is intended to make siblings, not children.

Below are the relevant bits of the code used for the video. Unless I misunderstand what I'm doing, I am using the AST asynchronously to calculate what the DOM should look like after all find and replaces have been made. Then I inject that whole markup into react, react carries out a single rerender and a single painting to the screen. So the user is never blocked. I guess technically it hast around react, but it only comes into play when I'm making dom substitutions on foreign markup.

I have pasted the relevant bit of code I wrote (for the app in the video). The performance seemed great when I tried it yesterday, and that was even though I animated each highlighted string of text a fadein with a random duration using css keyframes. If anybody has any improvements or a better way of doing things, I'd love to hear it. As for my original question, it took me a really long time to understand where and how to use the hast and unist utilities and when to use the pipeline. I had the impression that these utilities might be usable inline in different .use() statements, maybe by making the tree available through a callback in each statement. Basically the middleware approach that redux and express js takes. That would have been awesome, at least for the initial importing of the documents I want to inject.. But I couldn't find any examples of that, so I guess not.

css part

import styled, { keyframes } from 'styled-components';

const fadeIn = keyframes`
  from {
    opacity: 0;
  }
  to {
    opacity: 1;
  }
`;

const GlobalScreenStyle = styled('div')<Partial<GlobalScreenStyleProps>>`
  .localHighlight0 {
    animation: ${fadeIn} ${Math.random()}s ease-in-out;
    background-color: #cea1dbff;
    color: #000000;
  }
  .localHighlight1 {
    animation: ${fadeIn} ${Math.random()}s ease-in-out;
    background-color: #9ab9eeff;
    color: #000000;
  }
  /* and so on... */
`;

export { GlobalScreenStyle };

App.tsx

import { useState } from 'react';
import { ThemeProvider } from 'styled-components';
import { alice } from './assets';
import { useActiveTheme } from './theme';
import { useProcessor } from './useProcessor';
import { useClick, useSelection } from './hooks';
import { GlobalStyle, GlobalScreenStyle } from './GlobalStyle';

const App = () => {
  useClick();
  useSelection();
  const { DocAsReact, decorate, removeCssClasses, highlight1, highlight2 } =
    useProcessor(alice.screen.clean);
  const [hideMarkup, setHideMarkup] = useState(false);

  const toggleHideMarkup = () => setHideMarkup(prev => !prev);
  const theme = useActiveTheme();

  return (
    <ThemeProvider theme={theme}>
      <GlobalStyle />
      <GlobalScreenStyle $hideMarkup={hideMarkup}>
        <div className="button-group">
          <button onClick={highlight1}>highlight1</button>
          <button onClick={highlight2}>highlight2</button>
          <button onClick={removeCssClasses}>clean</button>
          <button onClick={decorate}>decorate</button>
          <button onClick={toggleHideMarkup}>show/hide</button>
        </div>
        {DocAsReact}
      </GlobalScreenStyle>
    </ThemeProvider>
  );
};

export default App;

useProcessor.ts

import { Fragment, useState, useEffect, createElement, useRef } from 'react';
import { Root } from 'hast';
import { unified } from 'unified';
import rehypeParse from 'rehype-parse';
import rehypeReact, { Options } from 'rehype-react';
import { visit } from 'unist-util-visit';
import rehypeSanitize, { defaultSchema } from 'rehype-sanitize';
import * as prod from 'react/jsx-runtime';
import * as dev from 'react/jsx-dev-runtime';
import { __DEV__ } from './config/config';
import { classnames } from 'hast-util-classnames';
import { findAndReplace } from 'hast-util-find-and-replace';
import { useSelText } from './store';
import { h } from 'hastscript';

// prettier-ignore
// @ts-ignore: the react types are missing.
const production: Options = { Fragment: prod.Fragment, jsx: prod.jsx, jsxs: prod.jsxs, };
// prettier-ignore
// @ts-ignore: the react types are missing.
const development: Options = { Fragment: dev.Fragment, development: true, jsxDEV: dev.jsxDEV, };
const options = __DEV__ ? development : production;

function getRandomIntInclusive(min: number, max: number) {
  const minCeiled = Math.ceil(min);
  const maxFloored = Math.floor(max);
  return Math.floor(Math.random() * (maxFloored - minCeiled + 1) + minCeiled);
}

const escapeRegExp = (text: string, flags = 'gi') => {
  const escText = text.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
  return new RegExp(escText, flags);
};

const useProcessor = (docAsHtmlStr: string) => {
  const selectedText = useSelText();
  const [DocAsReact, setContent] = useState(createElement(Fragment));
  const tree = useRef<Root | undefined>();

  useEffect(() => {
    tree.current = unified()
      .use(rehypeParse, { fragment: true })
      .parse(docAsHtmlStr);
    updateInjectedDoc();
  }, []);

  const highlight1 = () => {
    if (!tree.current) return;
    const regExpQuery = new RegExp(escapeRegExp(selectedText));
    findAndReplace(tree.current, [
      [
        regExpQuery,
        function ($0) {
          return h(
            'mark',
            { classname: [`localHighlight${getRandomIntInclusive(0, 5)}`] },
            $0,
          );
        },
      ],
    ]);
    updateInjectedDoc();
  };
  const highlight2 = () => {
    if (!tree.current) return;
    const regExpQuery = new RegExp(escapeRegExp(selectedText));
    findAndReplace(tree.current, [
      [
        regExpQuery,
        function ($0) {
          return h(
            'mark',
            { classname: [`globalHighlight${getRandomIntInclusive(0, 5)}`] },
            $0,
          );
        },
      ],
    ]);
    updateInjectedDoc();
  };

  const removeCssClasses = async () => {
    tree.current &&
      visit(tree.current, ['element'], node => {
        if ('tagName' in node && node?.tagName === 'mark') {
          node.properties.className = [];
        }
      });
    await updateInjectedDoc();
  };

  /**
   * Adds css indexed css classes to headers and body text.
   * Used for making document map and scrolling to specific elements.
   * e.g.,
   *   <h1>title1</h1>
   *   <h1>title2</h1>
   *     ...
   *   <h1 className="h1-0">title</h1>
   *   <h1 className="h1-1">title</h1>
   */
  const decorate = async () => {
    const levels = ['h1', 'h2', 'h3', 'h4', 'h5', 'h6', 'p'];

    tree.current &&
      visit(tree.current, ['element'], (node, index) => {
        if ('tagName' in node && levels.includes(node?.tagName)) {
          const levelClass = `${node.tagName}-${index}`;
          node = classnames(node, levelClass);
        }
      });
    await updateInjectedDoc();
  };

  const updateInjectedDoc = async () => {
    const file = tree.current
      ? await unified()
          .use(rehypeSanitize, defaultSchema)
          .use(rehypeReact, options)
          .stringify(tree.current)
      : DocAsReact;

    setContent(file);
  };

  return { decorate, removeCssClasses, highlight1, highlight2, DocAsReact };
};

export { useProcessor };