Code

Composite Actions Tests#93

Code

Closed

dgrebb wants to merge 132 commits intodevelopdgrebb/BackstopJS:developfrom downstream/feature-1533-test-validationsdgrebb/BackstopJS:downstream/feature-1533-test-validationsCopy head branch name to clipboard

+7,719-169

Commits

Commits on Jan 1, 2024

• ci(tests): add test fixtures and validations (garris#1533)

  Show description for d99ffb8
  dgrebb

  committed
  d99ffb8

  Copy full SHA for d99ffb8

• fix: remove errant newlines from fixtures

  dgrebb

  committed
  2d1d41c

  Copy full SHA for 2d1d41c

• style: log reports.json diffs in GitHub Actions summary

  dgrebb

  committed
  b1426b0

  Copy full SHA for b1426b0

• wip: fixing smoke tests

  dgrebb

  committed
  9b0ad1e

  Copy full SHA for 9b0ad1e

• wip: show full diff

  dgrebb

  committed
  32044cc

  Copy full SHA for 32044cc

• wip: set results to var and echo in both pass and fail summary

  dgrebb

  committed
  ea70a0c

  Copy full SHA for ea70a0c

• wip: remove extra playwright install shenanigans

  dgrebb

  committed
  f5d468e

  Copy full SHA for f5d468e

• wip: remove errant quote

  dgrebb

  committed
  f2d18e1

  Copy full SHA for f2d18e1

Commits on Jan 2, 2024

• wip: try set +e to avoid step failure but fail nonetheless

  dgrebb

  committed
  5fec75f

  Copy full SHA for 5fec75f

• wip: do not silence any diff output

  dgrebb

  committed
  6e864d6

  Copy full SHA for 6e864d6

• wip: ignore all of the diff object and add better formatting

  dgrebb

  committed
  96fd517

  Copy full SHA for 96fd517

• wip: fail in a separate step to allow summary before exit

  dgrebb

  committed
  697ff15

  Copy full SHA for 697ff15

Commits on Jan 3, 2024

• wip: remove github_env output

  dgrebb

  committed
  05ec530

  Copy full SHA for 05ec530

• wip: use steps.validate.outcome instead of failure()

  dgrebb

  committed
  c56ff34

  Copy full SHA for c56ff34

• wip: use new lines

  dgrebb

  committed
  b5492b4

  Copy full SHA for b5492b4

• wip: remofve code block

  dgrebb

  committed
  3a5d78c

  Copy full SHA for 3a5d78c

• wip: set diff to var to prevent exit 1

  dgrebb

  committed
  1504917

  Copy full SHA for 1504917

• wip: set +x

  dgrebb

  committed
  31a923f

  Copy full SHA for 31a923f

• wip: set -e after +e

  dgrebb

  committed
  f525cb3

  Copy full SHA for f525cb3

• wip: use single quotes for backticks

  dgrebb

  committed
  0aeb93f

  Copy full SHA for 0aeb93f

• wip: wrong line

  dgrebb

  committed
  affd9d1

  Copy full SHA for affd9d1

• wip: try removing set +e

  dgrebb

  committed
  0b78d50

  Copy full SHA for 0b78d50

• wip: separate summary into its own step

  dgrebb

  committed
  51eb6f1

  Copy full SHA for 51eb6f1

• wip: RESULT already set

  dgrebb

  committed
  db3d516

  Copy full SHA for db3d516

• wip: use output as var immediateily

  dgrebb

  committed
  440f3d3

  Copy full SHA for 440f3d3

• wip: again

  dgrebb

  committed
  90b4e3e

  Copy full SHA for 90b4e3e

• again

  dgrebb

  committed
  dc55e14

  Copy full SHA for dc55e14

• yep

  dgrebb

  committed
  db11270

  Copy full SHA for db11270

• try this

  dgrebb

  committed
  78ff8f2

  Copy full SHA for 78ff8f2

• wip: add both integration test files in fixture

  dgrebb

  committed
  2acb843

  Copy full SHA for 2acb843

• wip: separate integration tests and use wildcards

  dgrebb

  committed
  dc8899c

  Copy full SHA for dc8899c

• use two stars?

  dgrebb

  committed
  4556b70

  Copy full SHA for 4556b70

• move tests

  dgrebb

  committed
  7da28ea

  Copy full SHA for 7da28ea

• wip: refactor command

  dgrebb

  committed
  4877066

  Copy full SHA for 4877066

• if result is nothing, because two files

  dgrebb

  committed
  b38495f

  Copy full SHA for b38495f

• add smoke test

  dgrebb

  committed
  806f0d3

  Copy full SHA for 806f0d3

• check for empty in integration test

  dgrebb

  committed
  ed2bf8f

  Copy full SHA for ed2bf8f

• gotsta set continue-on-error

  dgrebb

  committed
  87bfa67

  Copy full SHA for 87bfa67

• wip: only compare keys

  dgrebb

  committed
  4e27d8f

  Copy full SHA for 4e27d8f

Commits on Jan 4, 2024

• wip: set all properties to empty string

  dgrebb

  committed
  bd9b929

  Copy full SHA for bd9b929

Commits on Jan 5, 2024

• wip: remove diff object to avoid property differences

  dgrebb

  committed
  d04532f

  Copy full SHA for d04532f

• wip: update sanity-test with new jq comparison pattern

  dgrebb

  committed
  545702f

  Copy full SHA for 545702f

• wip: updates remaining tests with new jq pattern

  dgrebb

  committed
  e8b2337

  Copy full SHA for e8b2337

• wip: add property sort and echo real diff to summary on fail

  dgrebb

  committed
  eb29f77

  Copy full SHA for eb29f77

• wip: use quotes to keep formatting

  dgrebb

  committed
  c148892

  Copy full SHA for c148892

Commits on Jan 14, 2024

• wip: extend integration test to display full diff

  dgrebb

  committed
  5e9a5ca

  Copy full SHA for 5e9a5ca

• wip: add summary/details expand/collapse for full diff

  dgrebb

  committed
  4e49911

  Copy full SHA for 4e49911

Commits on Jan 15, 2024

• wip: update integration test fixtures

  dgrebb

  committed
  11c7577

  Copy full SHA for 11c7577

• wip: add var to echo rows

  dgrebb

  committed
  e2450ef

  Copy full SHA for e2450ef

• wip: add linebreaks between markdown and markup

  dgrebb

  committed
  aae54a5

  Copy full SHA for aae54a5

• wip: specifically test reference and test report based on date directory createdf

  dgrebb

  committed
  98f45fc

  Copy full SHA for 98f45fc

• wip: only test test results - not reference

  dgrebb

  committed
  b1e9049

  Copy full SHA for b1e9049

• wip: remove linebreaks

  dgrebb

  committed
  84d6022

  Copy full SHA for 84d6022

• only test integration test result and skip reference report

  dgrebb

  committed
  dfe4f30

  Copy full SHA for dfe4f30

• wip: again

  dgrebb

  committed
  48465ba

  Copy full SHA for 48465ba

• wip: quote

  dgrebb

  committed
  0e8d202

  Copy full SHA for 0e8d202

• wip: closing if

  dgrebb

  committed
  c43ca2e

  Copy full SHA for c43ca2e

• wip: add paragraph

  dgrebb

  committed
  ff2e1dd

  Copy full SHA for ff2e1dd

• wip: final layout for integration workflow

  dgrebb

  committed
  859d88b

  Copy full SHA for 859d88b

• wip: add full diff to sanity test

  dgrebb

  committed
  b6a4cd9

  Copy full SHA for b6a4cd9

• wip: updates smoke and sanity

  dgrebb

  committed
  1a273f8

  Copy full SHA for 1a273f8

• wip: sanity broken

  dgrebb

  committed
  d721183

  Copy full SHA for d721183

• wip: update sanity check

  dgrebb

  committed
  ad128b0

  Copy full SHA for ad128b0

• wip: integration test was not correct

  dgrebb

  committed
  99453aa

  Copy full SHA for 99453aa

• wip: break test

  dgrebb

  committed
  7e9c97b

  Copy full SHA for 7e9c97b

• wip: set diff to var before github env

  dgrebb

  committed
  eafa30e

  Copy full SHA for eafa30e

• add the diff proeprty back to report object

  dgrebb

  committed
  ef714ff

  Copy full SHA for ef714ff

• apply the same to sanity

  dgrebb

  committed
  0603ca4

  Copy full SHA for 0603ca4

• wip: updates sanity with var/echo to env pattern

  dgrebb

  committed
  3a375a0

  Copy full SHA for 3a375a0

• fix syntax

  dgrebb

  committed
  1faa5e7

  Copy full SHA for 1faa5e7

• simplify conditional only test if diff is not empty

  dgrebb

  committed
  bea3364

  Copy full SHA for bea3364

• intentionally break test command

  dgrebb

  committed
  9cee6f3

  Copy full SHA for 9cee6f3

• wrap var in quotes

  dgrebb

  committed
  b511931

  Copy full SHA for b511931

• fixes intentionally broken test command

  dgrebb

  committed
  5d381b4

  Copy full SHA for 5d381b4

• change check workflow names

  dgrebb

  committed
  22962d4

  Copy full SHA for 22962d4

• fix test

  dgrebb

  committed
  cf514c3

  Copy full SHA for cf514c3

• reformat smoke test

  dgrebb

  committed
  4a78ef5

  Copy full SHA for 4a78ef5

• fix jq filter for smoke

  dgrebb

  committed
  23081e5

  Copy full SHA for 23081e5

• echo the var not a new diff

  dgrebb

  committed
  5048247

  Copy full SHA for 5048247

• update smoek test

  dgrebb

  committed
  3ddeb42

  Copy full SHA for 3ddeb42

• remove diffImage from smoketests

  dgrebb

  committed
  bf62dfd

  Copy full SHA for bf62dfd

• try sorting smoke tests

  dgrebb

  committed
  abf2f9a

  Copy full SHA for abf2f9a

• try removing diffimage removal

  dgrebb

  committed
  1ae314a

  Copy full SHA for 1ae314a

• remove diffImage

  dgrebb

  committed
  328d9e3

  Copy full SHA for 328d9e3

• apply logic to docker smoke tests

  dgrebb

  committed
  a26ae31

  Copy full SHA for a26ae31

• remove diff removal

  dgrebb

  committed
  dcb2311

  Copy full SHA for dcb2311

• set +e as well

  dgrebb

  committed
  eb5a523

  Copy full SHA for eb5a523

• continue on error

  dgrebb

  committed
  2757230

  Copy full SHA for 2757230

• set +e for docker as well

  dgrebb

  committed
  564635a

  Copy full SHA for 564635a

• wip: use checkout-cache-install across workflows

  dgrebb

  committed
  071f762

  Copy full SHA for 071f762

• wip: adjust to checkout before using composite action

  dgrebb

  committed
  5947a19

  Copy full SHA for 5947a19

• wip: repush with change

  dgrebb

  committed
  4f7dad7

  Copy full SHA for 4f7dad7

• wip: double checkout party

  dgrebb

  committed
  3a01c55

  Copy full SHA for 3a01c55

• wip: move actions into directories

  dgrebb

  committed
  c520c06

  Copy full SHA for c520c06

• wip: adds base setup action

  dgrebb

  committed
  8fac186

  Copy full SHA for 8fac186

• wip: add shell and wd

  dgrebb

  committed
  da80974

  Copy full SHA for da80974

Commits on Jan 16, 2024

• wip: inputs

  dgrebb

  committed
  bd5e80a

  Copy full SHA for bd5e80a

• wip: removes checkout from base setup

  dgrebb

  committed
  e9b1032

  Copy full SHA for e9b1032

• wip: correct actions path

  dgrebb

  committed
  239ade1

  Copy full SHA for 239ade1

• wip: change path

  dgrebb

  committed
  1046c6b

  Copy full SHA for 1046c6b

• wip: correct outputs var

  dgrebb

  committed
  e308b7e

  Copy full SHA for e308b7e

• wip: remove unused tmate config

  dgrebb

  committed
  b93484d

  Copy full SHA for b93484d

• wip: move setup node

  dgrebb

  committed
  5fa116d

  Copy full SHA for 5fa116d

• wip: more stuff

  dgrebb

  committed
  4b9d579

  Copy full SHA for 4b9d579

• wip: change path after checkout

  dgrebb

  committed
  02d4707

  Copy full SHA for 02d4707

• wip: checkout in composite action

  dgrebb

  committed
  216522f

  Copy full SHA for 216522f

• wip: try to set directory

  dgrebb

  committed
  a220279

  Copy full SHA for a220279

• wip: use repo dir for clone

  dgrebb

  committed
  0ca85d1

  Copy full SHA for 0ca85d1

• wip: change order of workflow root steps

  dgrebb

  committed
  bf931b9

  Copy full SHA for bf931b9

• wip: use base actions dir

  dgrebb

  committed
  a89ba4c

  Copy full SHA for a89ba4c

• wip: update setup-node to use workspace root

  dgrebb

  committed
  e83335b

  Copy full SHA for e83335b

• wip: use in path

  dgrebb

  committed
  ab15391

  Copy full SHA for ab15391

• wip: correct path

  dgrebb

  committed
  fa292a1

  Copy full SHA for fa292a1

• wip: remove tmate

  dgrebb

  committed
  bc15be0

  Copy full SHA for bc15be0

• wip: pass in workspace root

  dgrebb

  committed
  72ac2d0

  Copy full SHA for 72ac2d0

• wip: add description

  dgrebb

  committed
  b9c21e3

  Copy full SHA for b9c21e3

• wip: use composites in smoke

  dgrebb

  committed
  bd0c6b1

  Copy full SHA for bd0c6b1

• wip: add playwright cache

  dgrebb

  committed
  eafac73

  Copy full SHA for eafac73

• wip: pass workspace dir to playwright cacher

  dgrebb

  committed
  114ed00

  Copy full SHA for 114ed00

• wip: get playwright version from correct dep

  dgrebb

  committed
  dbc826e

  Copy full SHA for dbc826e

• wip: use latest if no version found for playwright

  dgrebb

  committed
  247816e

  Copy full SHA for 247816e

• wip: get version with jq

  dgrebb

  committed
  c6c9e1a

  Copy full SHA for c6c9e1a

• wip: add sanity-test composite

  dgrebb

  committed
  6151dc3

  Copy full SHA for 6151dc3

• wip: add shell and cwd

  dgrebb

  committed
  a161e9a

  Copy full SHA for a161e9a

• wip: improve readability

  dgrebb

  committed
  90fcc2f

  Copy full SHA for 90fcc2f

• wip: set up sanity docker check with composites

  dgrebb

  committed
  bf49750

  Copy full SHA for bf49750

• wip: use correct base

  dgrebb

  committed
  954eac3

  Copy full SHA for 954eac3

• wip: paths

  dgrebb

  committed
  f94894e

  Copy full SHA for f94894e

• wip: pass in GH token

  dgrebb

  committed
  470357d

  Copy full SHA for 470357d

• wip: pass in ghcr credentials

  dgrebb

  committed
  7e12f74

  Copy full SHA for 7e12f74

• wip: add smoke test composite

  dgrebb

  committed
  424d314

  Copy full SHA for 424d314

• wip: use inputs and shorten fixture vars

  dgrebb

  committed
  df64663

  Copy full SHA for df64663