Fix: ci screenshots

[chrismclarke]

PR Checklist

• PR title descriptive (can be used in release notes)

Description

The visual screenshots test has been failing since last December, due to incompatibilities introduced with automated updates to github actions runner and outdated code dependencies

This PR aims to fix both the code to generate screenshots (applied on merge to master) and compare function (triggered by pr's labelled with test - visual)

Potential Follow-ups

• Create issue to automate generate to trigger ahead of test-visual compare action

• Consider adding better support to define generation parameters for deployments (e.g. setting specific indexeddb fields, which are currently hardcoded for debug deployment)

• Further improvements to outputs (e.g. include diff size in filenames and order from most to least changed)

Author Notes

Once merged authors should start being able to use the test - visual action again to compare against master branc

Previously base screenshots would be generated on every merge to master branch. This isn't ideal as it ends up generating lots of screenshots that are never used for comparison, and also fails to take into account authoring changes which may be the cause of differences (author changes wouldn't trigger new screenshot generation)

As such I've disabled the automated screenshot generator, in favour of manually triggering as required. So that means running visual tests is a semi-manual process involving:

1. Generate the base screenshots by triggering Run Workflow from the Test-Visual Generate action page

2. Wait for action run to complete

3. Assign the test - visual label on the PR to generate and compare pr branch screenshots

Note that this is a temporary workaround which should hopefully be replaced in the future with automated generation as part of the test - visual tagging process

It might also be worth reviewing the the base screenshots generated (323 screenshots for the debug templates), as many are blank/limited (might be worth either tidying some up or including better filters for what to include or not). This isn't critical by any means, more to speed up the overall processes a bit (probably takes 1-2s per screenshot) and make it easier to compare final outputs (fewer false-positives)

Review Notes

See passing Test-Visual Generate action run (triggered manually).

Example passing Test-Visual Compare action run (triggered by PR)

Dev Notes

I decided to try and update all dependencies to latest versions, however that immediately caused some issues with commonJS/esm module interop (the codebase was set as commonJS but a number of the dependencies have been updated to use ESM). As such the following additional steps were required:

• Replace ts-node as the main code runner for TSX which has broadly better interoperability between esm and commonJS
• Update all code to be ESM by default
• Drop axios dependency and replace with native fetch method for HTTP requests

Additionally minor updates were made to both bypass the new language select screen (which otherwise blocks underlying templates) and to bypass Leave site beforeunload alert when trying to close each tab

Git Issues

Closes #

Screenshots/Videos

If useful, provide screenshot or capture to highlight main changes

[github-actions]

Visual Test Summary
new : 23
different : 19
same : 275

Largest Differences
1 | 18.9% | example_parent_point_box_info
2 | 18.7% | example_lang_template_2
3 | 11.3% | debug_video_1
4 | 5.8 % | debug_kids
5 | 3.9 % | example_lang_template_pop_2
6 | 1.6 % | comp_data_items_dev
7 | 1.4 % | debug_data_items
8 | 1.4 % | example_calc_date
9 | 1.3 % | example_calc
10 | 0.7 % | debug_theme_language_assets

Download Link
https://nightly.link/IDEMSInternational/open-app-builder/actions/runs/13142568130

Run Details
https://github.com/IDEMSInternational/open-app-builder/actions/runs/13142568130

[esmeetewinkel]

Thanks for fixing this. I was able to download the assets from the action summary and works as expected.

However, the direct download link that appears as a comment on the PR doesn't seem to work

It might also be worth reviewing the the base screenshots generated (323 screenshots for the debug templates), as many are blank/limited (might be worth either tidying some up or including better filters for what to include or not).

I can look into this at some point (but don't think it should hold up this PR). Does this currently use all templates from the debug deployment or a subset? If the latter - where is it specified?

[chrismclarke]

Thanks for fixing this. I was able to download the assets from the action summary and works as expected.

However, the direct download link that appears as a comment on the PR doesn't seem to work

Thanks for checking, they action was still using the old name for the repo (parenting-app-ui), just pushed a commit so should now be fixed

It might also be worth reviewing the the base screenshots generated (323 screenshots for the debug templates), as many are blank/limited (might be worth either tidying some up or including better filters for what to include or not).

I can look into this at some point (but don't think it should hold up this PR). Does this currently use all templates from the debug deployment or a subset? If the latter - where is it specified?

It's currently all hardcoded and only really setup to work with the debug repo (should try make available to content repos in the future but would require follow-up issue prioritisation). The current hardcoded config supports filtering by flow subtype with additional options to provide specific templates to include/skip. Although in the future I think we would have a more generic function that can be configured on the deployment level

/** List of template flow subtypes to test */
const TEST_FLOW_SUBTYPES = ["debug", "component_demo"];

/** List of additional template names to test */
const ADDITIONAL_TEMPLATE_NAMES = ["home_screen", "weekly_workshops"];

/** List of template names to skip */
const SKIPPED_TEMPLATE_NAMES: string[] = [];

[chrismclarke]

@jfmcquade
I'm going to go ahead and merge here to test the generated screenshots on main. None of the code changes should have any knock-on on main app codebase, but feel free to add any comments post-merge and I can address in a follow-up

[esmeetewinkel]

Follow-up issue: #2778