diff --git a/README.md b/README.md index 15e9db67..5955dc37 100644 --- a/README.md +++ b/README.md @@ -1,16 +1,10 @@ -# NxVerdaccioE2eSetup +# Buildable Test Environment Plugin - +This plugin provides a way zeros configuration setup to run e2e tests in a package manager environment. -✨ **This workspace maintains enterprise grade E2E steup for vitest and verdaccio** ✨ +## Getting started -This workspace maintains a scalable and maintainable E2E setup for Vite tests and Verdaccio. - -## Getting started - -### Register Plugin - -1. Configure the plugins in `nx.json`: +1. Register and configure the plugins in `nx.json`: ```json { @@ -25,43 +19,61 @@ This workspace maintains a scalable and maintainable E2E setup for Vite tests an } ``` -### Configure the project you want to e2e test as published package +Now you can configure the project you want to e2e test as published package. -1. Add a `publishable` tag to the package under test +2. Add a `publishable` tag to the package under test + +```jsonc +// projects/my-lib/project.json +{ + "name": "my-lib", + "tags": ["publishable"] + // ... +} +``` -### Configure e2e project which tests the published package +Next you need to configure the e2e project to use the package under test. -1. Add the package under test as `implicitDependency` to your e2e project -2. Configure the `setup-env` target as dependent target by using `dependsOn` +3. Add the package under test as `implicitDependency` to your e2e project. ```jsonc +// projects/my-lib-e2e/project.json { - // ... - "e2e": { - "dependsOn": [ - { - "projects": "self", - "target": "setup-env", - "params": "forward" - } - ], - // ... - } + "name": "my-lib-e2e", + "implicitDependency": ["my-lib"] } ``` -### Run e2e test +4. Configure the `setup-env` target as dependent target in your e2e test project by using `dependsOn` -`nx run :e2e` +```jsonc +{ + "name": "my-lib-e2e", + "targets": { + "e2e": { + "dependsOn": [ + { + "projects": "self", + "target": "setup-env", + "params": "forward" + } + ] + // ... + } + } + // ... +} +``` + +Now you are ready to go. +5. Run your e2e test with `nx run my-lib-e2e:e2e` -## Debuggins e2e tests +Tadaaaa! πŸŽ‰ -Publishable project have a `publishable` tag. -Projects that need an environment have a `npm-env` tag. -Targets that need an environment set up before running depend on `{ "projects": "self", "target": "setup-env", "params": "forward"}`. +## DX while debuggins e2e tests -Production usage: +### Production usage: - `nx run cli-e2e:e2e` - setup environment and then run E2E tests for `cli-e2e` @TODO figure out why we can't set the environmentRoot in the target options in `project.json` @@ -85,140 +97,6 @@ Debug packages: - `nx run utils:npm-publish --environmentProject cli-e2e` - publishes `utils` and `models` to the verdaccio registry configured for `cli-e2e` - `nx run utils:npm-install --environmentProject cli-e2e` - installs `utils` and `models` from the verdaccio registry configured for `cli-e2e` - `nx run cli-e2e:stop-verdaccio` - stops the verdaccio server for `cli-e2e` - - -## Motivation - -### Flaws in the current setup - -One of the biggest flaws of e2e setups you find in the wild is one shared environment for all projects. - -This leads to a lot of problems: - -- flaky tests -- long setup times -- EXTREMELY hard to debug -- hard to maintain -- IMPOSSIBLE to scale - -#### Changes files during e2e - -The changed files during testing, interfere with configurations on your system. - -```sh -User/ - └── / - β”œβ”€β”€ .npmrc # πŸ”“ added registry and token entry to OS user specific npm config - └──Root/ # πŸ‘ˆ this is your CWD - β”œβ”€β”€ node_modules/ - β”‚ └── - β”‚ └── /... # πŸ”“ npm install installs into repository folder - β”œβ”€β”€ dist/ - β”‚ └── packages/ - β”‚ └── /... - β”œβ”€β”€ tmp/ - β”‚ └── local-registry/ # πŸ˜“ hard to debug a dynamic port - β”‚ β”œβ”€β”€ storage/... - β”‚ β”‚ └── - β”‚ β”‚ └── /... # nx nx-release-publish saves the package's tarball here - β”‚ └── /... - β”‚ └── /... - β”œβ”€β”€ package-lock.json # πŸ”“ npm install/uninstall installs into workspace root - └── package.json # πŸ”“ npm install/uninstall installs into workspace root -``` - -#### Task Performance - -To elaborate on the performance issues, we show the different cases while writing tests. - -##### Changes in source - -```mermaid -flowchart TB -P[project-e2e:e2e]:::e2e-.implicit.->E[project:build]:::build; -classDef e2e stroke:#f00 -classDef build stroke:#f00 -``` - -##### Changes in the test environments - -```mermaid -flowchart TB -P[project-e2e:e2e]:::e2e-.implicit.->E[project:build]:::build; -classDef e2e stroke:#f00 -classDef build stroke:#f00 -``` - -##### Changes in tests - -```mermaid -flowchart TB -P[project-e2e:e2e]:::e2e-.implicit.->E[project:build]:::build; -classDef e2e stroke:#f00 -``` - -### Solution - -This workspace provides a scalable and maintainable E2E setup for Vite tests and Verdaccio. -It isolates all involved files into an isolated environment for each e2e project. - -#### Changes files during e2e - -The changed files during testing, are all in one isolated folder and don't interfere with your local setup. - -```sh -Root/ # πŸ‘ˆ this is your CWD -β”œβ”€β”€ dist/ -β”‚ └── packages/ -β”‚ └── /... -└── tmp/ - └── e2e/ - └── / # e2e setup - β”œβ”€β”€ storage/... # npm publish/unpublish - β”œβ”€β”€ node_modules/ - β”‚ └── - β”‚ └── /... # npm install/uninstall - β”œβ”€β”€ __test__/... - β”‚ └── /... # e2e beforeEach - β”‚ └── /... - β”œβ”€β”€ .npmrc # local npm config configured for project specific verdaccio registry - β”œβ”€β”€ package-lock.json # npm install/uninstall - └── package.json # npm install/uninstall -``` - -#### Task Performance - -To elaborate on the performance improvements, we show the different cases while writing tests. - -##### Changes in source - -```mermaid -flowchart TB -P[project-e2e:e2e]:::e2e-.implicit.->S[project-e2e:setup-env]:::build; -S-.implicit.->E[project:build]:::build; -classDef e2e stroke:#f00 -classDef setup-env stroke:#f00 -classDef build stroke:#f00 -``` - -##### Changes in the test environments - -```mermaid -flowchart TB -P[project-e2e:e2e]:::e2e-.implicit.->S[project-e2e:setup-env]:::setup-env; -S-.implicit.->E[project:build]:::build; -classDef e2e stroke:#f00 -classDef setup-env stroke:#f00 -``` - -##### Changes in tests - -```mermaid -flowchart TB -P[project-e2e:e2e]:::e2e-.implicit.->S[project-e2e:setup-env]:::build; -S-.implicit.->E[project:build]:::build; -classDef e2e stroke:#f00 -``` ## Connect with us! diff --git a/docs/motivation.md b/docs/motivation.md new file mode 100644 index 00000000..e57b0255 --- /dev/null +++ b/docs/motivation.md @@ -0,0 +1,188 @@ +# Motivation + +The following document explains the motivation behind this library and the problems it solves. +We will discuss common e2e setups in the wild, what problems they have and why they are pretty limited in their scalability and performance. + +What is not covered in this document is basic knowledge about Verdaccio as wel as Nx. + +Let's start of by explaining the common e2e setups. + +## Common e2e setup for publishable packages + +To get e2e tests setup with Verdaccio we typically need the following building blocks: + +- A package we want to test +- A global setup script used by your testing lib to sets up the environment before running all tests +- A test executing the package + +The interesting part here is the global setup script. So let's take a closer look at it this first. + +The following is a simplified version of a global setup script used by your testing lib. + +```ts +// global-setup.ts +import { rm } from 'node:fs/promises'; +import { executeProcess, objectToCliArgs } from '@org/test-utils'; +import { configureRegistry, RegistryResult, startVerdaccioServer, unconfigureRegistry } from '@org/tools-utils'; + +export async function setup() { + const { verdaccioPort } = await startVerdaccioServer({ + storage: 'local-registry/storage', + }); + + // Configure npm with form the created Verdaccio registry + // `npm config set registry http://localhost:${verdaccioPort}` + // `npm config set //localhost:${verdaccioPort}:_authToken "my-auth-token"` + configureRegistry(verdaccioPort); + + // Publish the package to test to the Verdacio storage (local-registry/storage) + // `npm publish @my-org/my-lib@0.0.1 --registry=http://localhost:${verdaccioPort}` + await publishProject('my-lib'); + + // Install the package locally + // `npm install my-lib --registry=http://localhost:${verdaccioPort}` + await installProject('my-lib'); +} + +export async function teardown() { + // Uninstall the package from the Verdacio storage (local-registry/storage) + // `npm uninstall my-lib` + await uninstallProject('my-lib'); + + // Revert configure npm with form the created Verdaccio registry + // `npm config delete registry` + // `npm config delete //localhost:${verdaccioPort}:_authToken` + unconfigureRegistry(registry, isVerbose); + + stopVerdaccioServer(); + + // Delete the Verdaccio storage + await rm('local-registry/storage', { recursive: true, force: true }); +} +``` + +Now you could run `nx run my-lib-e2e:e2e` which would start the server publish and install, executes the tests and runs the cleanup logic. +Viola, you have a working e2e setup for your package. πŸŽ‰ + +But wait! There is are MANY caveats with this setup. Let's discuss them one by one. + +## File system changes while running the e2e test + +The following file tree is a result of running our e2e setup. +It is particular bad as it interfere with your local package manager configuration. + +```sh +User/ + └── / + β”œβ”€β”€ .npmrc # πŸ”“ added registry and auth token entry to OS user specific npm config + └──Root/ # πŸ‘ˆ this is your CWD + β”œβ”€β”€ node_modules/ + β”‚ └── @my-org + β”‚ └── my-lib/... # πŸ”“ npm install installs into repository folder + β”œβ”€β”€ dist/ + β”‚ └── packages/ + β”‚ └── my-lib/... + β”œβ”€β”€ e2e/ + β”‚ └── my-lib-e2e/ + β”‚ └── some.test.ts + β”œβ”€β”€ tmp/ + β”‚ └── local-registry/ + β”‚ └── storage/ + β”‚ └── @my-org + β”‚ └── my-lib/... # npm publish saves the package's tarball here + β”œβ”€β”€ package-lock.json # πŸ”“ npm install/uninstall installs into workspace root + └── package.json # πŸ”“ npm install/uninstall installs into workspace root +``` + +### Task Performance + +To elaborate on the performance issues, we show the different cases while writing tests. + +#### Changes in source + +```mermaid +flowchart TB +P[project-e2e:e2e]:::e2e-.implicit.->E[project:build]:::build; +classDef e2e stroke:#f00 +classDef build stroke:#f00 +``` + +#### Changes in the test environments + +```mermaid +flowchart TB +P[project-e2e:e2e]:::e2e-.implicit.->E[project:build]:::build; +classDef e2e stroke:#f00 +classDef build stroke:#f00 +``` + +#### Changes in tests + +```mermaid +flowchart TB +P[project-e2e:e2e]:::e2e-.implicit.->E[project:build]:::build; +classDef e2e stroke:#f00 +``` + +## Solution + +This workspace provides a scalable and maintainable E2E setup for Vite tests and Verdaccio. +It isolates all involved files into an isolated environment for each e2e project. + +### Changes files during e2e + +The changed files during testing, are all in one isolated folder and don't interfere with your local setup. + +```sh +Root/ # πŸ‘ˆ this is your CWD +β”œβ”€β”€ dist/ +β”‚ └── packages/ +β”‚ └── /... +└── tmp/ + └── e2e/ + └── / # e2e setup + β”œβ”€β”€ storage/... # npm publish/unpublish + β”œβ”€β”€ node_modules/ + β”‚ └── + β”‚ └── /... # npm install/uninstall + β”œβ”€β”€ __test__/... + β”‚ └── /... # e2e beforeEach + β”‚ └── /... + β”œβ”€β”€ .npmrc # local npm config configured for project specific verdaccio registry + β”œβ”€β”€ package-lock.json # npm install/uninstall + └── package.json # npm install/uninstall +``` + +### Task Performance + +To elaborate on the performance improvements, we show the different cases while writing tests. + +#### Changes in source + +```mermaid +flowchart TB +P[project-e2e:e2e]:::e2e-.implicit.->S[project-e2e:setup-env]:::build; +S-.implicit.->E[project:build]:::build; +classDef e2e stroke:#f00 +classDef setup-env stroke:#f00 +classDef build stroke:#f00 +``` + +#### Changes in the test environments + +```mermaid +flowchart TB +P[project-e2e:e2e]:::e2e-.implicit.->S[project-e2e:setup-env]:::setup-env; +S-.implicit.->E[project:build]:::build; +classDef e2e stroke:#f00 +classDef setup-env stroke:#f00 +``` + +#### Changes in tests + +```mermaid +flowchart TB +P[project-e2e:e2e]:::e2e-.implicit.->S[project-e2e:setup-env]:::build; +S-.implicit.->E[project:build]:::build; +classDef e2e stroke:#f00 +```