Add pagefind search to abridge

[Hysterelius]

I have added a new pagefind.index.js, which can successfully build the index.

What needs to be fixed:

• Multilingual support
• Minify js files
• Properly edit package_abridge.js
• Put index files in proper location
• Actually use search results in search
• Remove console.logs for testing

[netlify]

✅ Deploy Preview for abridge ready!

Name	Link
🔨 Latest commit	a3c4521
🔍 Latest deploy log	https://app.netlify.com/sites/abridge/deploys/669ba36e0044a90008e0a063
😎 Deploy Preview	https://deploy-preview-179--abridge.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[Hysterelius]

@Jieiku The pagefind has to write to static to make sure that all the files get included in public/, do you want all these files to be added to git?

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
	modified:   static/js/abridge.min.js
	modified:   static/js/abridge_nopwa.min.js
	modified:   static/sw.js
	modified:   static/sw.min.js

Untracked files:
  (use "git add <file>..." to include in what will be committed)
	index.html
	static/js/fragment/
	static/js/index/
	static/js/pagefind-entry.json
	static/js/pagefind-highlight.js
	static/js/pagefind-modular-ui.css
	static/js/pagefind-modular-ui.js
	static/js/pagefind-ui.css
	static/js/pagefind-ui.js
	static/js/pagefind.en_ccab42ec59.pf_meta
	static/js/pagefind.en_d5bbd35c84.pf_meta
	static/js/pagefind.es_62f452c0d4.pf_meta
	static/js/pagefind.es_7556d6166a.pf_meta
	static/js/pagefind.fr_78454663b6.pf_meta
	static/js/pagefind.fr_b5d517652d.pf_meta
	static/js/pagefind.js
	static/js/wasm.en.pagefind
	static/js/wasm.es.pagefind
	static/js/wasm.fr.pagefind
	static/js/wasm.unknown.pagefind

[Jieiku]

Initially it is fine to add whatever you need to add to static, it can always be changed later if needed.

I have been busy with work and will be a couple more days but I can't wait to check this out once I get a little down time.

[Hysterelius]

I just wondering how you do testing with abridge?
Like the base url attribute means that it tries to fetch from netlify and when I tried to change it to localhost I got some weird errors...

[Jieiku]

For most things I test using zola serve.

When I want to test a live build I push it to one of my development containers in my proxmox server. I have haproxy handling SSL termination and just hardcode a host override in the unbound dns resolver (but you could just as easily do it in your hosts file.)

I use the following bash script to build a site and push it to my container:

    # change directory to my site that uses abridge on my local workstation
    cd ~/.dev/mytestdomain.com
    # update abridge
    git submodule update --init --recursive
    git submodule update --remote --merge
    # update the abridge theme node backage in my root site that is using abridge as a submodule
    rsync themes/abridge/package_abridge.js package_abridge.js
    rsync themes/abridge/package.json package.json
    # build the site
    npm run abridge
    cd public
    # The following line is optional it is just to pre gzip/brotli the content to save some cpu cycles when serving content.
    find ~/.dev/mytestdomain.com/public -type f -regextype posix-extended -regex '.*\.(htm|html|css|js|json|xml|xsl|txt|svg|otf|eot|ttf|webmanifest)' -exec gzip --best -k -f {} \+ -exec brotli --best -f {} \;
    # rsync transfers the data to the development container
    rsync -zvrhog --chown=www-data:www-data --delete ~/.dev/mytestdomain.com/public/ webdev:/var/www/mytestdomain.com

Your error indicates that the integrity value of a script does not match what is listed in the head, it has to do with lines like this:

<script src="https://abridge.pages.dev/js/theme.min.js" integrity="sha384-pb++s6uBRKaQv+iAXpgA/H3IlpLZdO14tTwuCI7uXmz4aaZdByoCcM+6BhynMq/1"></script>

<script defer src="https://abridge.pages.dev/js/abridge.min.js?h=4dbe2f6c7673e0a145f0" integrity="sha384-en9uTP2jQMLjO9fwbuGUKlCZ+kGMvN97ASddcA7mljWRGw70rs3zSMPNIozwGYFU"></script>

Zola generates these hashes when we issue a zola build, and because abridge minifies and bundles js files, it is the reason the node script issues zola build twice, once to generate the index, then after minification and bundling it does a second zola build to generate the correct integrity hashes for the newly minified/bundled js files.

[Hysterelius]

I was able to add pagefind to abridge :)

It probably needs a look over and a bit more testing.

I did put all the pagefind files in the .gitignore, as on every build they would regenerate leading to a large amount of pagefind files in static directory, so it is probably good to delete all those files when the package_abridge.js script runs.

EDIT: (I would also probably recommend squashing this PR, as there a lot of testing commits in here)

[Jieiku]

Awesome, thanks for your work on this! I tried it out a moment ago and got the following error:

git clone https://github.com/Hysterelius/abridge
cd abridge
npm run abridge

> [email protected] abridge
> node -e "if ( require('fs').existsSync('./themes/abridge/package_abridge.js')) {require('fs').copyFileSync('./themes/abridge/package_abridge.js', './package_abridge.js')}" && node package_abridge.js

Zola Build to generate files for minification:
Creating index...
node:fs:1525
  handleErrorFromBinding(ctx);
  ^

Error: ENOENT: no such file or directory, scandir '/home/jieiku/.dev/abridge-pagefind/abridge/public'
    at Object.readdirSync (node:fs:1525:3)
    at createIndex (/home/jieiku/.dev/abridge-pagefind/abridge/static/js/pagefind.index.js:13:22) {
  errno: -2,
  syscall: 'scandir',
  code: 'ENOENT',
  path: '/home/jieiku/.dev/abridge-pagefind/abridge/public'
}

Node.js v18.20.3

It appears the error happens when zola build is not ran first, so there is no public directory. I would like to resolve that issue. I am still very busy (13 hour days), but should have some free time by Monday I hope.

I did issue a manual zola build so that I could go ahead and test it out, and it seems to work excellent, and the site integration looks great too!

[Hysterelius]

Basically I was missing an await which led to a few errors - but it should be fixed now :)

I am also going to edit the pagefind files so they can properly adding into the git tracking - so that you can actually deploy the site properly.

---

Todo:

• Put config.toml back to normal
• Double check the pagefind.search.js file
• Add npm script to automatically change search library

[Hysterelius]

I have fixed all the issues and have done some testing :)

The pagefind files are still in the .gitignore to stop the git directory ballooning and they are built everytime anyway. I also added a function to remove all the old indexes to make sure that the number of files don't get out of hand.

I have created a little npm script to automatically swap from on search library to another:

npm run search:elasticlunr

npm run search:pagefind

(This script is currently in the static/js folder, as I didn't know where else to put it)

[Jieiku]

All my testing shows no issues, looks good!

Your fork is missing a few of my latest commits, but it looks like the only affected file is config.toml, your pull request does not modify any of the other files I worked on.

Let me know when you are ready for me to merge this :)

[Hysterelius]

I just remarked it as draft before, as I realised I hadn't tested pagefind with a single language, which worked great.
Everything should be good now!