Cucumber Service Library -- A Toolbox for API Testing

   Related

Module Documentation

Contributing This is a new library in active development. Bug reports, suggestions for improvement, etc. are welcome and should be submitted through github issues.

What you're looking at is a toolbox of services and steps for testing API's using Cucumber. The intent is for you to write automated tests for your API's as fast or faster than running manual tests (e.g Postman).

Features include:

1. Starting multiple servers with automatic stopping and clean up
2. HTTP server request and response steps
3. SQL database queries
4. Shell script testing steps
5. ... and much more

Several self contained examples applications are included as well.

All of these facilities are decoupled as much as possible. You can take what you like and leave the rest. So let's dive right in.

Simple Echo Server Example

This tests a simple server that echos back it's request body. (Note on exceptionally terse Gherkin.)

Feature: Super Simple Echo Server

  Scenario: A basic GET call just responds
    When GET "/users"
    Then responded with status code 200


  Scenario: A basic PUT call echos its input body
    Given PUT "/losers"
      """
      [
        "Sally Sad",
        "Billy Bad"
      ]
      """
    Then responded with status code 200
    And response matched pattern
      """
      [
        "Sally Sad",
        "Billy Bad"
      ]
      """

Here's all the support code that's needed:

const { childService, requestSteps, responseSteps } = require('cukelib');

module.exports = function () {
  childService.initialize.call(this);

  requestSteps.call(this, {
    host: 'http://localhost:3001'
  });
  responseSteps.call(this);

  this.Before(() => {
    return childService.spawn({
      name: 'echo_server',
      cmd: 'node',
      args: [`${__dirname}/../../index.js`, '--port=3001'],
    });
  });
};

That's simple enough, but there's actually a lot going on under the hood here.

0. The childService.spawn(...) function launches the server, connects its output streams, and handles its errors.
1. Notice there's no need for an After hook to stop the server -- childService.spawn(...) also sets up its own cleanup hooks.
2. Furthermore you can launch the server in BeforeFeatures or BeforeFeature hooks or even in actual steps, and cukelib will stop and cleanup the server at the end of the run, at the end of the feature, or at the end of the scenario as appropriate.
3. The request text is actually parsed as YAML, and...
4. The response text is matched using the lodash-match-pattern library, and...
5. Both the request and support interpret single cell tables as argument strings, so a similar scenario can be concisely expressed:

Feature: Cleaner Request/Response Steps

  Scenario: A basic PUT call echos its input body
    Given PUT "/losers"
      | [ Sally Sad, Billy Bad ] |

    Then responded with status code 200
    And response matched pattern
      | [ _.isString, /\w+\s\w+/ ] |

Interested? Yeah, we're just getting started, but before we get into some details, here's some ...

Conventions

The library is generally organized around services and steps. Although it's mostly non-opinionated there are some organizational conventions. First the library avoids the Cucumber "World" object, and implements its own "Universe" mechanism under the covers. This leaves you free to manage the World as needed or access and use the Universe functionality as well.

Service conventions

All of the services are variations on the theme illustrated by childService in the support code snippet above. They all implement launch and stop functionality and automatically run their stop functions within an appropriate testing life-cycle hook. All of the services and most of the steps also have initialize functions. In most cases it's fine to just call any one of them at the beginning of a step definition file.

And yes, it's possible and in fact typical, to launch multiple concurrent services.

Step definitions conventions

For the purposes of getting you started with testing quickly, I've made some non-dogmatic choices about the included step definitions. (Note on terse Gherkin.)

0. Step definitions are strictly decoupled from their support function code which are in separate respective ..._steps.js and ..._support.js files. This generally a good practice analogous to keeping views separate from business logic, but my main objective is to allow you to customize your own more relevant and readable step definitions.
1. The step definitions are intentionally terse. This is just a choice of simplicity over readability. Terse definitions are easier to write and easier to find, but again, you're free to customize your own.
2. The step definitions follow a strict convention with "Given" and "When" (setup) steps in present tense, and "Then" (assertion) steps in past tense. This is may be a good practice overall, but it's especially necessary for disambiguating terse definitions.
3. Postfix ... Not! steps. The module includes a tool for creating a logical opposite assertion steps from assertion steps. The step definition is the same as the original with a suffix of '... Not!'. This is to be read out loud as if from Wayne's World.

Library Details

1. Universe manages a namespaced object which is copied from the "universe" scope to the feature scope for each new feature, and is copied from the feature scope to the scenario scope for each new scenario. Most of the steps and service make use of the Universe for their internal state.
2. Service Control is the abstract parent of all of the services.
3. Child Service launches shell executable servers as child processes of the cucumber process.
4. Embed Service launches NodeJS servers (e.g. http.Server) embedded within the cucumber process.
5. Knex Service connects to SQL databases via the knex query builder.
6. Create Database Service is a convenience service to create (and drop) feature databases.
7. Selenium Standalone Service launches the selenium standalone server. This is included for completeness. Most likely you are managing your Selenium server separately.
8. WebdriverIO Service launches the WebdriverIO Selenium interface as an embedded service within the cucumber process.
9. GetSet Steps exposes access to the "universe" objects as cucumber steps.
10. Request Steps provides cucumber steps for issuing http requests, usually to servers launched by the Child or Embed services.
11. Request Response Overview
12. Response Steps are for interpreting the results of Request Steps.
13. SQL Overview
14. SQL Steps provides steps for issuing and interpreting results of SQL commands issued through Knex Services.
15. Shell Steps for running and interpreting output of shell scripts.
16. Not! and Throw! Steps wrappers for generating logical "Not!" steps (Throw! steps are useful for testing the development of step libraries such as this one. They aren't useful for general library usage.)
17. Diagnostic Steps for inspecting the state of the universe and debugging potential confusion.

Examples

0. Simple Echo Server -- Child Service, Request Steps, Response Steps.
1. Ad Server -- Child Service, Create Database Service, Knex Service, SQL Steps, Request Steps, Response Steps

Diagnostic Tips

1. Run with environment variable CUKELIB_VERBOSITY set to 1, 2, or 3

Contributing

Thank You in Advance. By all means submit issues, bug reports, suggestions, etc. to the github issues.