Skip to content

Commit

Permalink
Remove hard line wraps
Browse files Browse the repository at this point in the history
  • Loading branch information
twometresteve committed Dec 13, 2022
1 parent 4f5a7d6 commit 19641e7
Show file tree
Hide file tree
Showing 6 changed files with 30 additions and 70 deletions.
2 changes: 1 addition & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
## Enhancements

- Use `docker compose` instead of `docker-compose` [429](https://github.com/bugsnag/maze-runner/pull/429)
- Bump Bitbar browsers to those currently available [430](https://github.com/bugsnag/maze-runner/pull/430)
- Bump BitBar browsers to those currently available [430](https://github.com/bugsnag/maze-runner/pull/430)
- Firefox 102 to 107
- Chrome 103 to 108
- Edge 101 to 106
Expand Down
25 changes: 7 additions & 18 deletions DOCS.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,17 +4,13 @@ A test runner for validating requests.

## How it works

The test harness launches a mock API which awaits requests from sample applications. Using the runner, each scenario is
executed and the requests are validated to have to correct fields and values. Uses Gherkin and Cucumber under the hood
to draft semantic tests.
The test harness launches a mock API which awaits requests from sample applications. Using the runner, each scenario is executed and the requests are validated to have to correct fields and values. Uses Gherkin and Cucumber under the hood to draft semantic tests.

[Getting Started](./docs/Getting_Started.md)

## Mock server

In all modes of operation, Maze Runner automatically starts an HTTP server that mocks Bugsnag's server endpoints.
This mock server provides a URL for each endpoint that Bugsnag does, allowing it to receive errors, session, builds,
source map uploads and traces.
In all modes of operation, Maze Runner automatically starts an HTTP server that mocks Bugsnag's server endpoints. This mock server provides a URL for each endpoint that Bugsnag does, allowing it to receive errors, session, builds, source map uploads and traces.

See [Mock Server](./docs/Mock_Server.md) for more details of the options available for controlling the behaviour of the mock
server.
Expand All @@ -23,19 +19,14 @@ server.

Internally, Maze Runner has the following modes of operation

* [Appium/Devices](./docs/Device_Mode.md) - For running tests with Appium on local Android/iOS device or using a device
farm such as Bitbar or BrowserStack.
* [Selenium/Browsers](./docs/Browser_Mode.md) - For running tests with Selenium locally using a browser farm such as
Bitbar or BrowserStack.
* [Appium/Devices](./docs/Device_Mode.md) - For running tests with Appium on local Android/iOS device or using a device farm such as BitBar or BrowserStack.
* [Selenium/Browsers](./docs/Browser_Mode.md) - For running tests with Selenium locally using a browser farm such as BitBar or BrowserStack.

## Troubleshooting

### Logging

Maze Runner contains a Ruby logger connected to `STDOUT` that will attempt to log several events that occur during the
testing life-cycle. By default, the logger is set to report `INFO` level events or higher, but will log `DEBUG` level
events if the `VERBOSE` or `DEBUG` flags are set. If the `QUIET` flag is set it will instead log at the `ERROR` level
and above.
Maze Runner contains a Ruby logger connected to `STDOUT` that will attempt to log several events that occur during the testing life-cycle. By default, the logger is set to report `INFO` level events or higher, but will log `DEBUG` level events if the `VERBOSE` or `DEBUG` flags are set. If the `QUIET` flag is set it will instead log at the `ERROR` level and above.

| Log Level | Event | Information |
|-----------|-------|-------------|
Expand Down Expand Up @@ -86,10 +77,8 @@ Note: Maze Runner does not set the `progname`, so it will always be `nil` in a f

### Known issues

* Payload field matching for raw string values can be ambiguous when there is a possible regex match (e.g. when using
"." as a part of an expected value without escaping it).
* Payload field matching for raw string values can be ambiguous when there is a possible regex match (e.g. when using "." as a part of an expected value without escaping it).

### Running the tests

maze-runner uses test-unit and minunit to bootstrap itself and run the sample app suites in the test fixtures.
Run `bundle exec rake` to run the suite.
maze-runner uses test-unit and minunit to bootstrap itself and run the sample app suites in the test fixtures. Run `bundle exec rake` to run the suite.
7 changes: 2 additions & 5 deletions docs/Browser_Mode.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,7 @@

## Overview

Browser mode allows Selenium-based tests to be run on either local or remote browsers using a service such as BitBar or
BrowserStack. Browser mode is invoked by using the `--browser` option, which for remote browsers expects a symbolic
name that maps to an entry in the appropriate device farm's yaml file, for example:
Browser mode allows Selenium-based tests to be run on either local or remote browsers using a service such as BitBar or BrowserStack. Browser mode is invoked by using the `--browser` option, which for remote browsers expects a symbolic name that maps to an entry in the appropriate device farm's yaml file, for example:

* [BitBar](../lib/maze/client/selenium/bb_browsers.yml)
* [BrowserStack](../lib/maze/client/selenium/bs_browsers.yml)
Expand All @@ -14,8 +12,7 @@ For local browsers, the value given for `--browser` is converted to a symbol and

## Environment variables

To run tests on a service such as BitBar or BrowserStack you will need an account and set one of the following pairs
of environment variables:
To run tests on a service such as BitBar or BrowserStack you will need an account and set one of the following pairs of environment variables:

* `BROWSER_STACK_USERNAME`/`BROWSER_STACK_ACCESS_KEY`
* `BITBAR_USERNAME`/`BITBAR_ACCESS_KEY`
Expand Down
24 changes: 6 additions & 18 deletions docs/Device_Mode.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,28 +2,22 @@

## Overview

Device mode allows Appium-based tests to be run on either local or remote Android and iOS devices using a service such
as BitBar or BrowserStack. Device mode is generally invoked by using the `--app` option, which is the path to an APK
or IPA application file. For some device farms, an id or URI can be used to refer to a previously uploaded app.
Device mode allows Appium-based tests to be run on either local or remote Android and iOS devices using a service such as BitBar or BrowserStack. Device mode is generally invoked by using the `--app` option, which is the path to an APK or IPA application file. For some device farms, an id or URI can be used to refer to a previously uploaded app.

## Environment variables

To run tests on BitBar or BrowserStack you will need an account and set one of the following pairs of environment
variables:
To run tests on BitBar or BrowserStack you will need an account and set one of the following pairs of environment variables:

* `BROWSER_STACK_USERNAME`/`BROWSER_STACK_ACCESS_KEY`
* `BITBAR_USERNAME`/`BITBAR_ACCESS_KEY`

Device farms tend to offer a tunnelling facility, allowing HTTP requests sent by the device to be received by the Maze
Runner server. Both BitBar and BrowserStack use a local binary to support this mechanism, and one of the following
environment variables should be set to its location in order that Maze Runner can automatically start the tunnel.
Device farms tend to offer a tunnelling facility, allowing HTTP requests sent by the device to be received by the Maze Runner server. Both BitBar and BrowserStack use a local binary to support this mechanism, and one of the following environment variables should be set to its location in order that Maze Runner can automatically start the tunnel.

`MAZE_BS_LOCAL`/`MAZE_SB_LOCAL`

## Example usage with BrowserStack

Run the `bugsnag-android` "smoke tests" the `fixture-debug.apk` app on BrowserStack using an Android 9 device, stopping
immediately if any scenario fails.
Run the `bugsnag-android` "smoke tests" the `fixture-debug.apk` app on BrowserStack using an Android 9 device, stopping immediately if any scenario fails.

```
bundle exec maze-runner features/smoke_tests \
Expand All @@ -33,17 +27,11 @@ bundle exec maze-runner features/smoke_tests \
--fail-fast
```

Note the use of `ANDROID_9_0`, which is mapping to an entry in a hash that's dynamically generated by
[BrowserStack devices](../lib/maze/client/appium/bs_devices.rb). There are many other available options, as shown by
comments in that source file.
Note the use of `ANDROID_9_0`, which is mapping to an entry in a hash that's dynamically generated by [BrowserStack devices](../lib/maze/client/appium/bs_devices.rb). There are many other available options, as shown by comments in that source file.

## Appium client modes

By default, Maze Runner will use the latest version of `appium_lib`, the Ruby Appium library. For versions onwards,
the library enforces the use of W3C mode, which in turn means that Appium server v1.15 or later must be used. If an
earlier version is needed for any reason, this case be achieved by setting the `USE_LEGACY_DRIVER` environment variable
when both `bundle install`ing the Gem and when executing it. `appium_lib` v11 will then be installed and JWP protocol
employed.
By default, Maze Runner will use the latest version of `appium_lib`, the Ruby Appium library. For versions onwards, the library enforces the use of W3C mode, which in turn means that Appium server v1.15 or later must be used. If an earlier version is needed for any reason, this case be achieved by setting the `USE_LEGACY_DRIVER` environment variable when both `bundle install`ing the Gem and when executing it. `appium_lib` v11 will then be installed and JWP protocol employed.

```
rm Gemfile.lock
Expand Down
25 changes: 6 additions & 19 deletions docs/Getting_Started.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,7 @@

3. Run `bundle install` to fetch and install Maze Runner

4. Run `bundle exec maze-runner init` from the root of your project to build the
basic structure to run test scenarios.
4. Run `bundle exec maze-runner init` from the root of your project to build the basic structure to run test scenarios.
* `features/fixtures`: Test fixture files, such as sample JSON payloads
* `features/scripts`: Scripts to be run in scenarios. Any environment
variables set in scenarios are passed to scripts. The `MOCK_API_PORT`
Expand All @@ -42,10 +41,7 @@

## Writing features

Features should be composed as concisely has possible, reusing existing steps as
needed. The harness provides a number of reusable step definitions for
interacting with scripts, setting environment variables, and inspecting output.
Each new feature should go into a `.feature` file in the `features` directory.
Features should be composed as concisely has possible, reusing existing steps as needed. The harness provides a number of reusable step definitions for interacting with scripts, setting environment variables, and inspecting output. Each new feature should go into a `.feature` file in the `features` directory.

```
Feature: Sinatra support
Expand All @@ -60,10 +56,7 @@ Scenario: Sinatra unhandled exception
And the event "unhandled" is true
```

This example includes a mix of the included steps as well as custom ones
specific to the library being tested. `When I set an environment variable` is
provided by default while `When I start a Sinatra app` is defined in a custom
steps file in `features/steps/`, wrapping other included steps:
This example includes a mix of the included steps as well as custom ones specific to the library being tested. `When I set an environment variable` is provided by default while `When I start a Sinatra app` is defined in a custom steps file in `features/steps/`, wrapping other included steps:

```ruby
When("I start a Sinatra app") do
Expand All @@ -82,11 +75,7 @@ When("I navigate to the route {string}") do |route|
end
```

In addition, any helper functions or instance variables defined in
`features/support/env.rb` are available to step files. See the included
`_step.rb` files for examples. The files in `features/support` are evaluated
before scenarios are run, so this is where one-time or per-scenario
configuration should go.
In addition, any helper functions or instance variables defined in `features/support/env.rb` are available to step files. See the included `_step.rb` files for examples. The files in `features/support` are evaluated before scenarios are run, so this is where one-time or per-scenario configuration should go.

```ruby
# A helper function included with the harness to run commands and
Expand Down Expand Up @@ -127,8 +116,7 @@ end

#### Field path notation

Anywhere a `{field}`, `{path}`, `{key_path}`, etc is denoted, it can be replaced with dot-delimited key path to indicate
the path from the root of an object to the intended target.
Anywhere a `{field}`, `{path}`, `{key_path}`, etc is denoted, it can be replaced with dot-delimited key path to indicate the path from the root of an object to the intended target.

For example, to match the name of the second objects in the the key `fruits` below, use `fruits.1.name` as the keypath.

Expand All @@ -150,8 +138,7 @@ Then the payload body matches the JSON fixture in "features/fixtures/template.js
Then the payload field "items.0.subset" matches the JSON fixture in "features/fixtures/template.json"
```

The template file can either be an exact match or specify regex matches for string fields. For example, given the
template:
The template file can either be an exact match or specify regex matches for string fields. For example, given the template:
```json
{ "fruit": { "apple": { "color": "\\w+" } } }
```
Expand Down
17 changes: 8 additions & 9 deletions docs/Mock_Server.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,26 +2,25 @@

## Overview

The mock server is central to every Maze Runner test. It exposes various HTTP endpoints that replicate those available
at bugsnag.com (only conceptually - it doesn't use https and the domains/paths do not match precisely). Requests
received by the mock server are stored in memory and various Cucumber steps are provided for verifying their contents.
The mock server is central to every Maze Runner test. It exposes various HTTP endpoints that replicate those available at bugsnag.com (only conceptually - it doesn't use https and the domains/paths do not match precisely). Requests received by the mock server are stored in memory and various Cucumber steps are provided for verifying their contents.

Maze Runner also provides the ability to forward any POST requests that it receives to the real Bugsnag backend,
allowing errors and other data to be viewed in the dashboard. For more information see the `--repeater` option below.
Maze Runner also provides the ability to forward any POST requests that it receives to the real Bugsnag backend, allowing errors and other data to be viewed in the dashboard. For more information see the `--repeater` option below.

## Command line options

This section highlights some key command line options for controlling the behavior of the mock server. For the full
list of available options run `bundle exec maze-runner --help`.
This section highlights some key command line options for controlling the behavior of the mock server. For the full list of available options run:
```
bundle exec maze-runner --help
```

`--bind-address` - The mock server will bind to the same default address that the Ruby WEBrick server does.

`--port` - The port to listen on.

`--repeater` - Maze Runner can repeat the POST requests it receives on to bugsnag.com. This can be useful for:

- Populating the Bugsnag dashboard with dummy data
- Identifying gaps in test coverage
- Testing new features of Bugsnag during development

If using this option, the `MAZE_REPEATER_API_KEY` environment should be set to the API key of the desired destination
project in the Bugsnag dashboard.
If using this option, the `MAZE_REPEATER_API_KEY` environment should be set to the API key of the desired destination project in the Bugsnag dashboard.

0 comments on commit 19641e7

Please sign in to comment.