Massah

Making BDD style automated browser testing with node.js very simple. Get running automated browser tests within 5 minutes.

• Introduction
• Build Status
• How to use
  • Install
  • Getting started
  • Running tests
    • Browsers
    • Configuration file
      • Capabilities
    • Browserstack
  • Getting help
• Tests
  • Writing features
  • Writing steps
  • Sharing data between tests
  • Starting / Stopping / Accessing your application from massah
    • Starting your application
    • Stopping your application
    • Accessing your application
• Testing
• Name
• Licence

Thanks

Thanks to Browserstack for a FREE basic account in order to test Massah.

Introduction

Massah is essentially a wrapper around the following projects just making things a few steps easier for developers to run up a BDD-style automated browser testing setup.

• Yadda
• Mocha
• Should
• selenium-webdriver
• webdriverjs-helper

Build status

How to use

Install

npm i --save-dev massah

To access cli tools you can install Massah globally, or access them in ./node_modules/.bin/massah*.

Getting started

massah init

This command initializes your project creating the following folder structure (if not present):

• test
  • features
  • steps
  • screenshots

Features are where you write your BDD tests in plain text, steps is where you define these steps in code, and screenshots are used to store screenshots of any failed test steps.

Additionally an empty helper file is installed at test/helper.js this is not necessary but can be used later.

Running tests

massah test

You can set this to be used for npm test in your package.json.

Browsers

Massah is set up to use multiple browsers. By default Firefox is used. Browsers available to use are:

• Firefox (firefox)
• Phantomjs (phantomjs) requires install of phantomjs
• Opera (opera)
• Safari (safari)
• IE (ie)
• Chrome
  • Remotely (chrome)
  • Using chromedriver (chromedriver)

To switch browser simply set an environment variable of BROWSER set to one of the bold values above. For example,

BROWSER=chromedriver massah test

Options:

• --headless: Run tests headlessly

Configuration file

Placing a file .massah.js in the root of your project will cause Massah to read this file for configuration values. The configuration file should return a function which is passed a copy of arguments (as processed by yargs), this in turn should return a JSON object of values. It is up to the user to define how values are set.

Standard values:

• timeout: Timeout for test steps in milliseconds, defaults to 60000
• runner: Which runner would like to use (e.g. vanilla, browserstack) uses vanilla by default
• headless: Whether to run tests headlessly or not (not supported by all runners), defaults to false false
• applicationPort: Tells Massah which port your application is running on where appropriate. Defaults to 3000
• seleniumServerPort: Which port to use for the selenium server. Defaults to 4444
• capabilities: Object containing browser/os details see below

See the example file for more information.

Capabilities

capabilties is a key within the configuration object with its own sets of properties. Essentially it describes the browser and environment that you wish to run your tests in. Some keys can be specific to the platform they are running on (e.g. browserstack):

• browser: Browser to use for tests, defaults to firefox
• os: Operating system. No default
• os_version: Operating system version. No default
• browser_version: Browser version. No default
• screen_resolution: Screen resolution. No default

Browserstack

Massah can now run tests via browserstack meaning that you can run your automated UI tests again several browsers on several different operating systems.

To achieve this set runner equal to browserstack and set values for your username and key. An example of how to do this is in the .massah.js file.

Note: If running parallel tests running more than the permitted parallel test runs on your browserstack account will result in tests failing.

Getting help

massah help

Tests

Writing features

For an example of a feature file please see: test/features/add-option.feature

After this please read the Yadda guide

Writing steps

For an example of a step definition file please see: test/steps/url.js

After this please read the Yadda guide

All step files are loaded at once before the test suite starts meaning that defined steps can be shared.

An example step file would appear as follows:

var massah = require('massah/helper')

module.exports = (function() {
    var library = massah.getLibrary()
        .given('I visit http://google.co.uk', function() {
            this.driver.get('http://google.co.uk')
        })
        .when('I search for \'(.*)\'', function(searchTerm) {
            this.params.searchTerm = searchTerm
            this.driver.input('input[name=q]').enter(searchTerm)
        })
        .then('I expect to see the results page', function() {
            var params = this.params
            this.driver.currentUrl(function(currentUrl) {
                currentUrl.should.include(params.searchTerm)
            })
        })
    
    return library
})()

Also see the helper functions in webdriverjs-helper for some extra usefulness.

Sharing data between tests

As you can see from the example above data can be shared between test steps using the params object. This object is cleaned with each new test scenario, but can be used for sharing data in between tests.

Starting / Stopping / Accessing your application from Massah

Sometimes it makes sense to contain your application within Massah, e.g. to provide canned responses to API calls or to serve files to the browser.

In order to do this create a helper.js file in the test folder of your application. This test helper will then be provided to each test as part of the context object. From step definition files this is available at this.application.helper. You may export as many or as few helper functions as you require.

Starting your application

If your helper exports a beforeSuite or beforeFeature method then these will be called and passed a callback parameter. The callback should be called when your application has completed starting up.

If using Browserstack then you will need to start your application before your tests run (i.e. in beforeSuite). This is because their browserstack local process requires a listener to be available on your application port whilst running.

Stopping your application

If your helper exports a afterSuite or afterFeature method then these will be called and passed a callback parameter. The callback should be called when your application has completed closing down.

Accessing your application

If your helper exports a beforeScenario method then this will be called at the beginning of a scenario. This allows you to set up spies/mocks/etc as required for each test.

The first argument to this method is the annotations object which contains any annotations added to the feature or scenario. The second argument is the context object allowing you to set data which will be available to each of the test steps. The context object exists as follows:

{
    driver: /* browser */
    params: {}  /* Location to store shared data */
    application: /* The exports from your application test helper */
}

For an example of annotations being used please see the annotation override test in Massah.

@Only / @Pending / grepping tests

You can run only a certain scenario or feature by adding the @Only annotation.

Tests can be skipped by adding the @Pending annotation.

You can grep (or inverse grep for tests using your .massah.js file), see this example. The grep can be in the form of a string or a RegExp object. When using strings annotations are checked using a strict match.

Advanced

Running tests with garbage collection enabled

For long running tests you may want garbage collection to be forced so you don't experience long (potentially timing out pauses). To do this you'll need to run the special garbage collection method. Install massah to your project rather than locally and then run:

node --enable-gc ./node_modules/massah/bin/massah-gc

Splitting tests

If tests are taking too long to run, or you are running into memory/gc issues it is possible to split the tests down by feature file. In order to do this we set a total number of runners and the number of the current runner (starting from 1). These values can be set in your .massah.js file as per this example.

MASSAH_SPLIT_TOTAL=5 MASSAH_SPLIT_RUNNER=2 massah test

Note: @Only annotation won't really work as expected if using split.

Testing

To test Massah, simply run

npm test

From the command line. Massah uses itself to test itself. Please also check out our build job on Travisci where we use Browserstack to test multiple browsers in parallel: https://travis-ci.org/lloydwatkin/massah/builds

Name

The name Massah comes from the fantasy fiction novel called 'The Torah' and its cumulative sequel 'The Bible'. During the chapter titled 'Exodus' the Israelite people are being led out of Egypt. At one point they begin to worry about their lack of water/supplies/etc. Their leader, a character named Moses, gets a little miffed at them for daring to question the wisdom of the "sky man" for sending them on the journey. This place was then named Massah which basically means to test.

Licence

MIT