Newman Async Runner

newman async runner lets you run your local and online (API) postman collections asynchronously as all-against-all matrix for:

-> collections or their specific folders/items

-> environments

-> iteration data files

It uses htmlfull reporter - however this can be easily override by passing custom newmanOptions, see API

Installation

npm i newman-async-runner

Usage

You need to simply instantiate NewmanRunner with runnerOptions as parameter and call runTests()

const runner = require('newman-async-runner');

const runnerOptions = {
		folders: {
			collections:'./collections/',
			environments: './environments/',
			reports: './reports/', 
			data: './data/',
		},
		newmanOptions: {
			timeoutRequest: 10000
		}
	};

new runner.NewmanRunner(runnerOptions).runTests();

Integration with other tools

Runner can be easily paired with popular test frameworks like Mocha or Jest. runTests() method returns array of default newman results array [error, summary] for each run in matrix. Simple Mocha example with chai:

console.log = function(){};
const expect = require('chai').expect;
const runner = require('newman-async-runner').NewmanRunner;

const UAT = {
	folders:
		{collections:'https://api.getpostman.com/collections/?apikey=YOUR_POSTMAN_API_KEY'},
	specific_collection_items_to_run: ['test folder A', 'test folder B'],
	newmanOptions:
		{reporters: 'htmlfull'}
	};

const SIT = {
	folders:
		{collections:'https://api.getpostman.com/collections/?apikey=YOUR_POSTMAN_API_KEY'},
	specific_collection_items_to_run: ['test folder C', 'test folder D', 'test folder E'],
	newmanOptions:
		{reporters: 'htmlfull'} 
	};

describe('My Application API tests', function(){
	this.timeout(10000)
	
	it('passes all UAT tests', async function(){
		for (const eachResult of await new runner(UAT).runTests()){
			expect(eachResult.summary.run.failures).to.be.empty;
		}
	})

	it('passes all SIT tests', async function(){
		for (const eachResult of await new runner(SIT).runTests()){
			expect(eachResult.summary.run.failures).to.be.empty;
		}
	})
})

...and the result should be something like:

  My Application API tests
    √ passes all UAT tests (2989ms)
    √ passes all SIT tests (2137ms)

  2 passing (5s)

API

runnerOptions:

example of available options:

const runnerOptions = {
	folders: {
		collections:'./collections/',
		environments: './environments/',
		reports: './reports/', 
		data: './data/',
		templates: './templates/'},

	reporter_template: 'htmlreqres.hbs',
	anonymizeFilter: /(?<=\<password:\>)(.*?)(?=\<\\password\>)/g,
	specific_collection_items_to_run: ['folder 1', 'folder 2'],
	parallelFolderRuns: false,
	
	newmanOptions: {
		color: 'off',
		timeoutRequest: 10000
	}
};

folders : MANDATORY object

-> collections : MANDATORY string local path or online (postman API) uri to collections folder or single file.

-> environments : optional string local path or online (postman API) uri to environments folder or single file.

-> reports : optional string local path to reports output folder.

-> data : optional string local path to test run iteration data folder or single file.

-> templates : optional string local path to htmlfull templates folder.

reporter_template : optional string - htmlfull reporter specific template filename located in local options.templates folder. If not used runner will use default htmlfull template.

anonymizeFilter : optional js regex expression - report file anonymize regex filter. Runner will put *** in place of matching groups. If not used runner will not anonymize reports.

specific_collection_items_to_run : optional string array - specific collection(s) folder/item names to run from all collections or single collection located under options.collections path. If not used runner will run all folders and items in collection(s). In case of nested collection sub-folders it will trigger run for last matching folder/item, this is by newman design.

parallelFolderRuns : optional boolean will atomize each collection into separate folder and item runs. USE ONLY WITH options.specific_collection_items_to_run SET. Otherwise it will trigger separate run for each item in collection ;) When setup correctly this may speed-up whole collection execution time but may also clog it if there are too many runs. It will also generate much more report files since these produces separate run for each item. Also beware of default Node heap allocation limit (1.7GB for V8).

newmanOptions:optional object this will pass-trough any standard nemwan.run() options. In case of conflict it overwrites options used by newman-async-runner.

runnerMethods:

runTests() - asynchronously (all runs at once) fires newman test runs matrix for each combination of environment, collection and iteration data files. It will pass already pre-fetched data to newman, so in case of giving postman api uri paths in options.folder runner will minimize api calls needed - currently there are 60 calls/minute limit for postman api. Method returns standard newman [error, summary] results array for each run in matrix, so it can be processed same way as newman callback arguments.

Roadmap

• fetch collection, environment and data files trough any http raw data source

License

MIT