Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs(e2e): examples for e2e tests #1982

Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 24 additions & 9 deletions docs/e2e-tests/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
| `e2e-tests/playwright-report/index.html` | HTML report of the test execution |
| `e2e-tests/test-results` | Contains video recordings of the executed test cases |

### Navigate to the E2E Tests Directory and Install Dependencies
## Navigate to the E2E Tests Directory and Install Dependencies

From the root of the project directory, navigate to the `e2e-tests` directory:

Expand All @@ -24,31 +24,31 @@ cd e2e-tests
yarn install
```

### Install Playwright Browsers
## Install Playwright Browsers

The Playwright browsers should be installed automatically via the `postinstall` script in `package.json`. If not, you can manually install them:

```bash
yarn playwright install chromium
```

### Adding a Test
## Adding a Test

To incorporate a new test case, create a file with a `.spec.ts` extension in the `e2e-tests/playwright/e2e` directory.
The tests within a spec file can run in parallel (by default) or sequentially if using the .serial like in [this example](../../e2e-tests/playwright/e2e/github-happy-path.spec.ts). Note that sequential execution is considered a bad practice and is strongly discouraged.
Note that, in order to add or edit a test, you should adhere to the [contribution guidelines](./CONTRIBUTING.MD).
The tests within a spec file can run in parallel (by default) or sequentially if using the `.serial` modifier like in [this example](../../e2e-tests/playwright/e2e/github-happy-path.spec.ts). Note that sequential execution is considered a bad practice and is strongly discouraged.
To add or edit a test, you should adhere to the [contribution guidelines](./CONTRIBUTING.MD).

### Running the Tests
## Running the Tests

#### Prerequisites
### Prerequisites

To run the tests, ensure you have:

- **Node.js** (minimum version 18)
- An instance of the application to run the tests against
- [Playwright browsers installed](#install-playwright-browsers)

#### Environment Variables
### Environment Variables

Certain environment variables need to be set up, depending on what you intend to run. The most convenient way is to export them from the CLI or add them in your `.bash_profile` or `.zshrc`. Alternatively, they can be passed to Playwright via the `--env` flag:

Expand All @@ -71,7 +71,7 @@ The currently supported environment variables are:
| `KEYCLOAK_CLIENT_ID` | Keycloak client ID | Tests involving Keycloak authentication |
| `KEYCLOAK_CLIENT_SECRET` | Keycloak client secret | Tests involving Keycloak authentication |

#### Running the Tests
### Running the Tests

The Playwright command line supports many options; see them [here](https://playwright.dev/docs/test-cli). Flags like `--ui` or `--headed` are very useful when debugging. You can also specify a specific test to run:

Expand All @@ -88,3 +88,18 @@ yarn showcase-1-2-x # Runs the showcase 1.2.x test suite
yarn showcase-rbac-1-2-x # Runs the showcase RBAC 1.2.x test suite
```

## Setting Up Backstage Configuration During the Pipeline

[app-config-rhdh.yaml](../../.ibm/pipelines/resources/config_map/app-config-rhdh.yaml) is the configuration file used to add plugins or any other kind of configuration into Backstage during pipeline execution.

### Environment Variables in `configmap-app-config-rhdh.yaml`

To use environment variables in [`configmap-app-config.yaml`](../../.ibm/pipelines/resources/config_map/app-config-rhdh.yaml), you need to set the variables encoded as Base64 in the [`secrets-rhdh-secrets.yaml`](../../.ibm/pipelines/auth/secrets-rhdh-secrets.yaml). You can use temporary values for the secrets because they can be replaced by the pipeline. Add the required environment variables as Base64-encoded values using secure properties.

To replace the values in `secrets-rhdh-secrets.yaml`, you need to create a replace function using the [`openshift-ci-tests.sh`](../../.ibm/pipelines/openshift-ci-tests.sh) script. For example:

```bash
sed -i "s|KEYCLOAK_BASE_URL:.*|KEYCLOAK_BASE_URL: $KEYCLOAK_BASE_URL|g" $DIR/auth/secrets-rhdh-secrets.yaml
```

This command replaces the `KEYCLOAK_BASE_URL` value in the secrets file with the one provided in your environment variables.
23 changes: 23 additions & 0 deletions docs/e2e-tests/examples.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# Examples for e2e tests

## Basics

At this point in time, the framework mixes a few tests with [Fixtures](https://playwright.dev/docs/test-fixtures) and some others with [`beforeEach/All` and `afterEach/All`](https://playwright.dev/docs/api/class-test).
Some contraints in the current architecture do not facilitate the use of Fixture, but it is prefered as it is considered a good practice.

You can see an examples of fixture usage at [github discovery test](../../e2e-tests/playwright/e2e/github-discovery.spec.ts).

An example on the before/after approach is the [github happy path](../../e2e-tests/playwright/e2e/github-happy-path.spec.ts), but almoust any test that you will find at e2e will follow this approach.

Also, note that some test (like [github happy path](../../e2e-tests/playwright/e2e/github-happy-path.spec.ts)) uses a global page variable `let page: Page;`. This is used to avoid reauthentications on the test due to the current rate limits on github.

## Types of tests

Almost all test are related to testing the frontend functionalities of the app and plugins like all the tests related to catalog, learning page or the ones under `/plugins`.

Some other notable ones are:

- Integration with Github: `github-discovery.spec, github-integration-org-fetch.spec, github-happy-path.spec.ts`
- Checks on OC cluster elements like `verify-redis-cache.spec`

Besides, there are many other tests realted to different authentication providers, like `google-signin-happy-path.spec` or `guest-signin-happy-path.spec`, and all under `/authProviders`
25 changes: 4 additions & 21 deletions e2e-tests/README.md
Original file line number Diff line number Diff line change
@@ -1,22 +1,5 @@
# Showcase end-to-end tests
# Readme for e2e tests

### How to setup backstage configuration during the pipeline

[app-config-rhdh.yaml](../.ibm/pipelines/resources/config_map/app-config-rhdh.yaml) is the
configuration file
to add plugins or any other kind of configuration into Backstage

### Environment Variables into app-config-rhdh.yaml

To use environment variables
on [app-config-rhdh.yaml](../.ibm/pipelines/resources/config_map/app-config-rhdh.yaml)
we need to set the envs encoded as Base64 in the
[secrets-rhdh-secrets.yaml](.ibm/pipelines/auth/secrets-rhdh-secrets.yaml) .
You can use temporary values for the secrets because it can be replaced by the pipeline.
Add the environments needed in our as Base64 encoded values and using secure property.

To replace the values into secrets-rhdh-secrets.yaml we need to create the replace function using
[openshift-tests.sh](.ibm/pipelines/openshift-tests.sh) script. For example:

`sed -i "s|KEYCLOAK_BASE_URL:.*|KEYCLOAK_BASE_URL: $KEYCLOAK_BASE_URL|g" $DIR/auth/secrets-rhdh-secrets.yaml
`
The readme for the e2e framework is located [here](../docs/e2e-tests/README.md)
The contribution guidelines are [here](../docs/e2e-tests/CONTRIBUTING.MD)
The example and bootstraps to create tests are [here](../docs/e2e-tests/examples.md)
Loading