GUIMate Automated UI Testing Framework

Runs completely in browser to automate GUI testing while in development and in devops environments. The main feature is that it is possible to do reloads or page navigations while testing - and to continue the javascript tests while retaining all test information and flow.

Installation

Installation is as simple as including one file on each page that will run tests:

<script src="dist/guimate.js"></script>

Then you can start the testing process either by starting from javascript:

await window.guiMateRun({}).run("../tests");

or by using an url parameter like:

https://.../test_index.html?guimateRunUrl=../../unittest

For examples, you can refer to our internal test suite located in /unittest where we use guiMate to test the guiMate functionality.

Creating tests

To create tests we would suggest the following directory structure:

root
   |- js
   |   |- test_1.js
   |   |- test_2.js
   |   |- ...
   |   |- global.js
   |
   |- test-configuration.json

test_1.js, test_2.js,... are test suites and can be named anything you want.

global.js is an optional global Javascript file that will be executed every time guiMate loads (after navigations, on initial startup).

test-configuration.json is the main configuration file that defines all the test suites and the run configuration.

A complete working test suite can be found in the /unittest folder.

Main configuration test-configuration.json

We suggest you use the default name of this file to avoid complications. The file is built like this:

{
  "testVersion": 1,
  "id": "testsuite_complete",

  "config": {
    "showReport": true,
    "showProgress": true,
    "onFinish" : "console.log('-- Tests ready ---');",
    "runTiming" : {
      "preFunction": 200,
      "postFunction": 200
    },
    "runSuite": "",
    "runFunction": ""
  },

  "repository": {
    "test": "success"
  },

  "globalJs": [
    "js/global.js"
  ],

  "testSuites": [
    "js/test_suite_index.js",
    "js/test_suite_browser.js",
    "js/test_suite_user.js"
  ]
}

Parameter description:

• testVersion: version of the test suites (anything numeric you want)
• id: unique id for this test suite (used for persistance in localStorage)
• config.showReport: true to show the report in the browser at the end of all tests
• config.showProgress: true to show the status gui in the browser while running tests
• config.onFinish: String containing javascript code that will be executed once the tests finish
• config.runTiming.preFunction: time in ms that will be waited prior to execution of a test
• config.runTiming.postFunction: time in ms that will be waited after execution of a test
• runSuite: while developing it can be useful to just execute one test suite. Can be one of the values from testSuites
• runFunction: while developing it can be useful to just execute one test function. Enter the name of the function here (eg testClick)
• repository: a global repository of any values you like (like css selectors, etc) - available like gm.repository.<key>
• globalJs: a list of URLs for global javascript files that will be loaded and executed on guiMate run
• testSuites: a list of test suite javascript URLs that will be executed while running tests

Writing Unit Tests

Writing tests is as simple as it gets - just write a javascript in the following form:

let testSuite = {

  // A description for this test suite
  description: "General tests",

  // (optional) the url to load in the browser prior to executing this test suite
  suiteUrl: "test_browser.html",

  // (optional) function that will rin prior to each test
  before: async function () {
  },

  // (optional) function that will run after each test
  after: async function () {
  },

  // test functions always start with "test..." and have one parameter (gm) of type "runnerParameter"
  testSelect: function (gm) {
    gm.assert.strictEqual(gm.select("#test_01").dom.innerHtml(), "CONTENT_01");
  },
  ...
}

A description of the runnerParameter can be found here

For convenience you can find detailed docs to available functions here:

• select - Select elements jUnit style
• browser - Perform clicks or keyboard commands
• dom - Manipulate or query the DOM tree
• page - Query page attributes like url parameters
• verify - Quick page verifications

Writing Tests: Assert and Expect

The main purpose of unit testing is to verify the states and contents of various things that are currently running.

The assert and expect function groups are included from the open source Chai framework. You can access them from the runnerParameter like this:

let testSuite= {
  testSelect: function (gm) {
    // Assert examlpe
    gm.assert.strictEqual(gm.select("#test_01").dom.innerHtml(), "CONTENT_01");

    // Expect example
    gm.expect(gm.select("#select_01").dom.selectOptions()).deep.equal(
      [
        {
          "value": "-",
          "text": "Empty",
          "selected": false
        }
      ]
    );
  }
}

Writing Tests: Verify

To quickly verify states in your gui, we created a verify library. A detailed description can be found here.

Here is a quick overview of the most important functions:

let testSuite= {
  testVerify: async function (gm) {
    
    // verify if this element exists in the DOM. Throw assertion otherwise.
    gm.verify.exists("#test_01");

    // verify if this element does not exist in the DOM. Throw assertion if it was found.
    gm.verify.exists("#test_01");

    // verify if this element is visible (see notes for visibility). Throw assertion otherwise.
    gm.verify.visible("#test_01");

    // verify if this element is invisible (see notes for visibility). Throw assertion otherwise.
    gm.verify.visible("#test_01");

    // Find a string inside this element (case sensitive). Throw assertion otherwise.
    gm.verify.find("#test_01", "some string");

    // Find a string inside this element (case insensitive). Throw assertion otherwise.
    gm.verify.find("#test_01", "some string", {caseSensitive:false});

    // Check a text input box (also radio, select or textarea) for value (case sensitive). Throw assertion otherwise.
    gm.verify.inputValue("#input_text", "test");

    // Check a checkbox input box if it is checked. Throw assertion otherwise.
    gm.verify.inputValue("#input_check", true);
    
    // Wait for any condition to become valid before the timeout runs out (see notes for waitFor)
    await gm.verify.waitFor(function() {
      // verify if the content of #waitfor_01 becomes 'Ready' in less than 2 seconds.
      self.verify.find("#waitfor_01", "Ready");
    }, 2000);
    
    // Verify innerHTML source in a consistent way
    // options are: "default", "none", "all" or a combination of "space", "nbsp", "newline", "attr", "readable", "lower", "trim"
    gm.verify.innerHtml("#html_01",
      "<p>test</p>",
      "all");
  }
}

Notes for visibility: The element is considered visible if width and height is > 0, opacity is > 0 (also parents), display != none (also parents) and visibility != none (also parents)

When more than one element is selected by the cssSelector, ALL have to be visible or invisible to pass the test.

Notes for waitFor: The function is considered invalid if throws any assertion or exception or returns false.

To verify you have to return anything but false.

Writing Tests: Page Navigation or Reloads

Normally everything stops when the browser reloads or navigates anywhere else. The guiMate test framework is designed for exactly this case: to continue testing even if the browser navigates somewhere else. All you have to do is include the guiMate script on all the involved pages.

To realize this, we use a flowId parameter to determine the current state in your test function.

Here is an example:

let testSuite= {
  testFlow: function (gm) {
    if (gm.flowId === null) {
      // here we enter on the first execution of the function
      gm.navigate(testSuite.suiteUrl + "?test="+gm.util.generateDummyId(), "flow-1");
    } else if (gm.flowId === "flow-1") {
      // after the browser navigation, the flowId is 'flow-1'
      gm.reload("flow-2");
    } else if (gm.flowId === "flow-2") {
      // after reloading and entering the function a third time.
      gm.done(); // have to call this to signal that the test is successfully completed
    }
  }
}

When using the navigate/reload functions in a test function, you have to call the done() function or the test will fail.

Writing Tests: User Interaction

We included some methods to simulate user interaction by keyboard or mouse.

Examples are:

let testSuite= {
  testUserInteraction: async function (gm) {
    
    // clicks on the element
    await gm.select("#click_02").browser.click();
        
    // sets the cursor to beginning and types "Something"
    await gm.select("#input_01").browser.cursorSelect(0);
    await gm.select("#input_01").browser.keyboardWrite("Something");

    // selects characters 1 to 3 and replaces them with "Replaced"
    await gm.select("#input_01").browser.cursorSelect(1, 3);
    await gm.select("#input_01").browser.keyboardWrite("Replaced");

    // selects the whole content and hits the "delete" button to empty it
    await gm.select("#input_01").browser.selectAll();
    await gm.select("#input_01").browser.keyboardWrite("<<delete");
    
  }
}

The cursorSelect() can also be used on contenteditable=true elements. In this case, the index starts with the first non-whitespace character in the DOM tree beneath. To select all content in such an element, use browser.selectChildren()