[infra] GitHub actions no longer completes a build or test

[AndrewJakubowicz]

Context

As of a couple days ago, all GitHub actions now fail on any new PR on this repo.

The site still deploys, and preview URLs still work, so this does not block content changes.
However, it increases the risk of infrastructure changes.

Augustine did some investigating as a possible starting point (rephrased from chat):

There is a `BlockingRender` that exists (which makes some async work sync). It uses `Atomic.wait`, and `Atomics.notify`. At some point it may be that there are so many things happening, that there are not enough cycles for the worker thread to pick up the render request. They hang until one times out.

[AndrewJakubowicz]

Theory is that the very slow build time of lit.dev is due to the special syntax highlighting that is done.

I changed the dev:build command to 0x ../../node_modules/@11ty/eleventy/cmd.js --quiet which runs eleventy and generates a flamegraph using 0x.

Here is the full flamegraph of building the site:

If we color all the work that uses playwright to emulate a browser, we can see a huge amount of the work is playwright related:

And if we filter for renderer.ts or blocking-renderer.ts, this also reveals the length of time spent syntax highlighting:

Some rough numbers (on my M2 silicon mac)

Note the table should be read from top to bottom, and I am removing features using if (!DEV) { ... } flag.

Features	Time
Current dev:build time	17.24 seconds (46.2ms each)
Without blocking renderer	11.28 seconds (30.2ms each)
Without search index creation	7.97 seconds (21.4ms each)

It is possible to remove search index creation in dev mode because we don't upload dev mode search indexes to algolia, so it's wasting time.

Proposal

1. Easy win: do not create search index in dev mode.
2. Not so easy win: do not use the blocking renderer in dev mode.

[AndrewJakubowicz]

Found a third slow path once I added caching to the blocking-renderer.

See flame graph of our lit-eleventy-plugin highlighted in blue: