Features
Start a free trial

Hermione Testing

Hermione is a test framework inspired upon WebDriverIO. If you are already familiar with WebDriverIO and Mocha, you will enjoy working with the Hermione Test Framework.

There are several advantages that Hermione has over WebDriverIO:

  • Runs tests in subprocesses: to take advantage of all your CPU cores.
  • Extensible: You can add custom commands in the hermione config and use them as browser.myCustomCommand in tests.
  • Built-in assert library: Hermione provides a global expect variable that gives you access to a number of matchers that let you validate different things on the browser, an element or mock object.
  • Retry failed tests: To prevent flaky tests, hermione retries a failed test before marking it as failed.

Setting up Hermione

To set up Hermione, you can enter the following command in your terminal. First, make sure you've created a directory where you will set up a new Hermione project. 

npm init hermione-app .

You will be asked a couple of questions, which will help Hermione to decide how to configure your configuration file (.hermione.conf.js).

Once the configuration file has been created, you'll need to edit the browsers block and add your TestingBot key and secret. Here's an example of what it should look like:

module.exports = {
    gridUrl: 'http://hub.testingbot.com/wd/hub',
    baseUrl: 'http://localhost',
    pageLoadTimeout: 0,
    httpTimeout: 60000,
    testTimeout: 90000,
    resetCursor: false,
    sets: {
        desktop: {
            files: [
                'tests/**/*.hermione.js'
            ],
            browsers: [
                'chrome'
            ]
        }
    },
    browsers: {
    chrome: {
            automationProtocol: 'webdriver', // default value
            desiredCapabilities: {
                browserName: 'chrome',
                key: process.env.TB_KEY, // or replace with your TestingBot Key
                secret: process.env.TB_SECRET // or replace with your TestingBot Secret
            }
        }
    },
    plugins: {
        'html-reporter/hermione': {
            // https://github.com/gemini-testing/html-reporter
            enabled: true,
            path: 'hermione-report',
            defaultView: 'all',
            diffMode: '3-up-scaled'
        }
    }
}

Specify Browsers & Devices

To let TestingBot know on which browser/platform/device you want to run your test on, you need to specify the browsername, version, OS and other optional options in the capabilities field.

1. Select a Platform
2. Select a browser

Testing Internal Websites

We've built TestingBot Tunnel, to provide you with a secure way to run tests against your staged/internal webapps.
Please see our TestingBot Tunnel documentation for more information about this easy to use tunneling solution.

The example below shows how to easily run a Hermione test with our Tunnel:

1. Download our tunnel and start the tunnel: 

java -jar testingbot-tunnel.jar key secret

2. Adjust your test: instead of pointing to 'hub.testingbot.com/wd/hub' like the example above - change it to point to your tunnel's IP address.
Assuming you run the tunnel on the same machine you run your tests, change to 'localhost:4445/wd/hub'. localhost is the machine running the tunnel, 4445 is the default port of the tunnel.

This way your test will go securely through the tunnel to TestingBot and back:

module.exports = {
    gridUrl: 'http://localhost:4445/wd/hub',
    baseUrl: 'http://localhost',

Run tests in Parallel

Parallel Testing means running the same test, or multiple tests, simultaneously. This greatly reduces your total testing time.

You can run the same tests on all different browser configurations or run different tests all on the same browser configuration.
TestingBot has a large grid of machines and browsers, which means you can use our service to do efficient parallel testing. It is one of the key features we provide to greatly cut down on your total testing time.

By default, Hermione will run all tests simultaneously. We advise to set the parallelLimit option to the same number of parallel tests as your TestingBot plan allows. This will prevent tests from queueing up and eventually timing out. 

module.exports = {
    parallelLimit: 2 // depends on the TestingBot plan you have
}

Queuing

Every plan we provide comes with a limit of parallel tests.
If you exceed the number of parallel tests assigned to your account, TestingBot will queue the additional tests (for up to 6 minutes) and run the tests as soon as slots become available.

Mark tests as passed/failed

As TestingBot has no way to dermine whether your test passed or failed (it is determined by your business logic), we offer a way to send the test status back to TestingBot. This is useful if you want to see if a test succeeded or failed from the TestingBot member area.

In your tests, you can mark the current test as passed or failed, by using the executeScript command:

browser.executeScript("tb:test-result=" + (passed ? "passed" : "failed"));

Other NodeJS Framework examples

  • WebDriverIO

    With WebDriverIO you can run Mocha, Jasmine and Cucumber tests.

  • CodeceptJS

    Run acceptance tests with CodeceptJS on TestingBot.

  • Protractor

    Protractor is an end-to-end test framework for AngularJS applications. Protractor is a NodeJS program built on top of WebDriverJS.

  • Soda

    Selenium Node Adapter. A light-weight Selenium RC client for NodeJS.

  • Nightwatch

    Nightwatch is an automated testing framework written in NodeJS.

  • Intern

    Intern is a nodeJS framework for testing Web sites and applications.

  • WD.js

    WD.js is a NodeJS client for WebDriver/Selenium.