Hermione is the utility for integration testing of web pages using WebdriverIO and Mocha.
- Why should you choose hermione?
- Prerequisites
- Skip
- Quick start
- .hermione.conf.js
- CLI
- Environment variables
Hermione
provides several features which WebdriverIO
doesn't and makes testing process easier.
If you know WebdriverIO and Mocha, you can start writting and running tests in 5 minutes! You need to install hermione
via npm and add a tiny config in your project. See details in Quick start section.
When tests are run one by one, it takes a lot of time. Hermione
can run tests in parallel sessions in different browsers out of the box.
WebdriverIO
provides built-in commands for browser and page manipulation. Often projects need to store some common code and reuse it through all tests. So a developer should create some helpers and include them in the tests.
With hermione
it's very simple and straightforward. You can add any number of custom commands in the hermione config and use them as this.browser.myCustomCommand
in tests.
Moreover, hermione
provides plugins which work as a some kind of hooks. They allow a developer to prepare environement for tests and react properly to test execution events.
Integration tests use a dynamic environment with a lot of dependencies where any of them can work unstable from time to time. As a result, integration tests become red randomly and make them undetermined. It spoils all testing process.
To prevent incidental fails hermione
retryies a failed test before marking it as failed. It makes it possible to get rid of a majority of incidental fails. Number of retries can be specified for all browsers or for a separate browser.
Hermione
reruns tests in a new browser session to exclude situations when the browser environment is a cause of this fail.
Sometimes it is needed to run only specific tests but not all set. Hermione
makes it possible.
hermione tests/func/mytest.js
All work with a grid client is incapsulated in hermione. Forget about client.init
and client.end
in your tests ;)
Because of hermione
is based on WebdriverIO
you need to set up Selenium before proceed further.
The simplest way to get started is to use one of the NPM selenium standalone packages like: vvo/selenium-standalone. After installing it (globally) you can run your server by executing:
selenium-standalone start
This feature allows you to ignore the specified suite or test in any browser with additional comment.
You can do it by using global hermione.skip
helper. It supports the following methods:
.in
— adds matchers for browsers with additional comment.notIn
—.in
method with reverted value;
Each method takes following arguments:
- browser {String|RegExp|Array<String|RegExp>} — matcher for browser(s) to skip
- [comment] {String} — comment for skipped test;
Note that matchers will be compared with browserId
specified in a config file, e.g. chrome-desktop
.
For example,
describe('feature', () => {
hermione.skip.in('chrome', "It shouldn't work this way in Chrome");
it('should work this way', () => runTestThisWay());
it('should work that way', () => runTestThatWay());
hermione.skip.in(['chrome', 'firefox', /ie\d*/], 'Unstable test, see ticket TEST-487');
it('should done some tricky things', () => runTrickyTest());
});
in this case behaviour it should work this way
will be skipped only in chrome
browser, but will be run in other browsers. It should work that way
will not be ignored. So skip will be applied only to the nearest test. If you need to skip all tests within a suite you can apply skip
helper to a describe
- all tests within this suite will be skipped with the same comment.
hermione.skip.in('chrome', 'skip comment');
describe('some feature', () => {
it(...);
it(...);
});
Also you can use .notIn
method to invert matching. For example,
// ...
hermione.skip.notIn('chrome', 'some comment');
it('should work this way', () => doSomething());
// ...
in this case test will be skipped in all browsers except chrome
.
Methods in
and notIn
are chainable. So you can skip test in several browsers with different comments. For example,
// ...
hermione.skip
.in('chrome', 'some comment')
.notIn('ie9', 'another comment');
it('test1', () => doSomething());
// ...
If you need to skip test in all browsers without a comment you can use mocha .skip
method instead of hermione.skip.in(/.*/);
. The result will be the same.
First of all, make sure that all prerequisites are satisfied.
Install the package.
npm install -g hermione
Then put .hermione.conf.js
in the project root.
module.exports = {
specs: ['tests/func'],
browsers: {
chrome: {
desiredCapabilities: {
browserName: 'chrome'
}
}
}
};
Write your first test.
var assert = require('chai').assert;
describe('yandex', function() {
it('should find itself', function() {
return this.browser
.url('https://yandex.com')
.setValue('.search2__input input', 'yandex')
.click('.search2__button button')
.getText('.z-entity-card__title')
.then(function(title) {
assert.equal(title, 'Yandex')
});
});
});
Finally, run tests.
hermione
hermione
is tuned using a configuration file. By default .hermione.conf.js
is used but a path to the configuration file can be specified using --conf
option.
There are only two required fields: specs
and browsers
.
module.exports = {
specs: [
'tests/desktop',
'tests/touch'
],
browsers: {
chrome: {
desiredCapabilities: {
browserName: 'chrome'
}
}
}
};
Required. The list of paths where hermione
will look for tests.
For example,
specs: [
{ // run tests associated with this path in all browsers
files: 'tests/desktop' // which are configured in option `browsers`
},
'tests/deskpad', // the alias for the previous case
{
files: 'tests/touch', // run tests associated with this path in a browser with id `browser`
browsers: ['browser'] // which is configured in option `browsers`
}
]
Required. The list of browsers which should be used for running tests.
Browser section has the following format
browsers: {
<browser_id>: {
<option>:<value>
<option>:<value>
}
}
<browser-id>
values is used for browser identification.
Available browser options:
Option name | Description |
---|---|
desiredCapabilities |
Required. Used WebDriver DesiredCapabilites |
sessionsPerBrowser |
Number of sessions which are run simultaneously. Default value is 1 . |
retry |
How many times test should be rerun. Default value is 0 . |
Selenium grid URL. Default value is http://localhost:4444/wd/hub
.
Base service-under-test url. Default value is http://localhost
.
Timeout for text execution. Default value is 60000
ms.
Timeout for web page events. Default value is 1000
ms.
If test execution time is greater than this value, then test is slow. Default value is 10000
.
Turn webdriver debug mode on. Default value is false
.
Number of sessions which are run simultaneously. Global value for all browsers. Default value is 1
.
How many times test should be retried in case of a fail. Global value for all browsers. Default value is 0
.
Hermione
plugins are commonly used to extend built-in possibilities. For example, hermione-allure-reporter and hermione-tunnel.
Plugin is a module which exports a single function. The function has two arguments:
- hermione instance;
- plugin options from configuration file.
Plugins will be loaded before hermione
runs tests.
It's strongly recommended to name hermione
plugins with hermione-
prefix. It makes search for user plugins very simple.
If a plugin name starts with hermione-
, then the prefix can be ommited in the configuration file. If two modules with names hermione-some-module
and some-module
are specified, then module with prefix will have higher priority.
For example.
// .hermione.conf.js
// ...
plugins: {
'my-cool-plugin': {
param: 'value'
}
}
// ...
// hermione-my-cool-plugin/index.js
module.exports = function(hermione, opts) {
hermione.on(hermione.events.RUNNER_START, function() {
return setUp(hermione.config, opts.param); // config can be mutated
});
hermione.on(hermione.events.RUNNER_END, function() {
return tearDown();
});
}
Properties of hermione
object
Property name | Description |
---|---|
config |
Config which is used in test runner. Can be mutated. |
events |
Events list for subscription |
Available events
Event | Description |
---|---|
RUNNER_START |
Will be triggered before tests execution. If handler return a promise, tests will be executed only after promise is resolved. |
RUNNER_END |
Will be triggered after tests execution. If handler return a promise, tests will be executed only after promise is resolved. |
SUITE_BEGIN |
Test suite is about to execute |
SUITE_END |
Test suite execution is finished |
SUITE_FAIL |
Suite failed. For instance, before hook failed or a browser can not be launched (in fact, browsers are launched in before hook implicitly in the core of hermione ) |
TEST_BEGIN |
Test is about to execute |
TEST_END |
Test execution is finished |
TEST_PASS |
Test passed |
TEST_FAIL |
Test failed |
TEST_PENDING |
Test is skipped |
RETRY |
Test failed but went to retry |
ERROR |
Generic (no tests) errors. |
INFO |
Reserved |
WARNING |
Reserved |
EXIT |
Will be triggered when SIGTERM is received (for example, Ctrl + C). Handler can return a promise. |
Extra options for mocha
which are passed to mocha.setup
. See Mocha documentation for the list of options.
mochaOpts: {
ignoreLeaks: true
}
Prepare browser session before tests are run. For example, adding custom user commands.
prepareBrowser: function(browser) {
// do setup here
}
browser
argument is a WebdriverIO
session.
Configuration data can be changed depending on extra conditions in prepareEnvironment
function.
Usage: hermione [options]
Options:
-h, --help Output usage information
-c, --conf <path> Path to configuration file [./.hermione.conf.js]
--baseUrl <url> Base service-under-test url [http://localhost]
--grid <url> Selenium grid URL [http://localhost:4444/wd/hub]
--wait-timeout <ms> Timeout for web page events [10000]
--screenshot-path <path> Path for saving screenshots []
--debug <boolean> Turn webdriver debug mode on [false]
--grep <grep> Filter tests matching string or regexp
-r, --reporter <reporter> Reporter [flat]
-b, --browser <browser> Run test in a specific browser
For example,
hermione --baseUrl http://yandex.ru/search
Note. All CLI options override config values.
Skip browsers specified in a config by passing browser ids. Several browser ids should be split by commas (spaces after commas are allowed).
For example,
HERMIONE_SKIP_BROWSERS=ie10,ie11 hermione