Japa

A test runner to create test runners

Japa is a tiny Node.js test runner that you can use to test your apps or even create your test runner.

Japa is simple, fast and has minimal core. Japa doesn't ship with any CLI. You run your test files as standard Node.js scripts.

node test/list-users.spec.js

Table of contents

• Features
• Why Japa?
  • Faster boot time ⏰
  • Simpler Syntax 💅
• Test your apps
  • Installation
  • Writing your first test
  • async/await
  • Test timeouts
  • Test groups
  • Skipping tests
  • Skipping/Running tests in CI
  • Run selected tests
  • Retry flaky tests
  • Regression tests
  • Assertion Planning
  • Cleaner Error Stack
• Runner hooks
• Japa flow
• Running multiple test files
  • Filtering files
• Configure options
• Running typescript tests
• ES modules support
• Coverage

Features

• Supports ES6 async/await style tests.
• Support for ES modules.
• Doesn't pollute the global namespace.
• First class support for regression tests.
• Conditionally skip when running tests in CI like travis.
• Retry flaky tests.
• Define test timeout.
• Cleaner error stack.
• Test groups with lifecycle hooks.
• Inbuilt assertion library with assertion planning.

Why Japa?

The primary reason to use Japa is that you can create your test runner using it. Which is impossible or cumbersome with other test runners like Ava or Mocha.

However, Japa also shines as a standalone test runner.

Faster boot time ⏰

Since Japa core is minimal doesn't have any CLI to run tests, it boots faster than Mocha and Ava.

The following benchmark is not a comparison to look down at Mocha or Ava, they are great test runners and offers a lot more than Japa.

1. First is mocha ( 0.20 seconds )
2. Seconds is Ava ( 0.73 seconds )
3. Third is Japa ( 0.12 seconds )

Simpler Syntax 💅

The Japa syntax is very similar to Ava. Additionally, it allows grouping tests using the group method.

const test = require('japa')

test('list users', () => {
})

Group tests

const test = require('japa')

test.group('User', (group) => {
  group.beforeEach(() => {
  })
  
  test('list users', () => {
  
  })
})

Test your apps

This section covers the topics to write tests for your apps using Japa. If you are looking to create a test runner, then read the custom test runner guides.

Installation

The installation is like any other Node.js package.

npm i --save-dev japa

# yarn
yarn add --dev japa

Writing your first test

Let's start by writing the first test for a method that returns the user object.

src/getUser.js

module.exports = function getUser () {
  return {
    username: 'virk',
    age: 28
  }
}

test/get-user.spec.js

const test = require('japa')
const getUser = require('../src/getUser')

test('get user with username and age', (assert) => {
  assert.deepEqual(getUser(), {
    username: 'virk',
    age: 28
  })
})

Now run the test file as follows.

node test/get-user.spec.js

That's all there to learn! See you tomorrow 😝. Okay wait, let's explore all the features of Japa.

async/await

Async/await one of the best ways to write async code, without spending your entire day in opening and closing curly braces.

Japa has out of the box support for them.

test('get user', async () => {
  const user = await Db.getUser()
})

Also, you can return Promises from your tests and Japa will resolve them for you. If the promise rejects, the test will be marked as failed.

test('get user', () => {
  return Db.getUser()
})

Test timeouts

Your tests must always timeout, otherwise you may end up with a never-ending process if one of your tests gets stuck.

Japa adds a timeout of 2000 milliseconds by default to all of your tests, which is a reasonable time. However, tests which interact with a Browser or 3rd party API may need a larger timeout.

Passing 0 as the timeout will disable the timeout

test('query contributors from github', async () => {
  await gh.getContributors()
}).timeout(6000)

You can also configure the timeout for group using the group object.

test.group('Test group', (group) => {
  group.timeout(6000)

  test('slow test', () => {
    // timeout is set 6000
  })
  
  test('fast test', () => {
    // timeout is set 2000
  }).timeout(2000)
})

Test groups

Grouping tests are helpful when you want to perform actions before, after the group or beforeEach or afterEach test.

const test = require('japa')

test.group('Group name', (group) => {

  group.before(async () => {
  })

  group.beforeEach(async () => {
  })  

  group.after(async () => {
  })  

  group.afterEach(async () => {
  })  
  
})

• The before hook is executed only once before all the tests.
• The after hook is executed only once after all the tests.
• The beforeEach hook is executed before every test.
• The afterEach hook is executed after every test.

Skipping tests

When refactoring code bases, you may break a bunch of existing functionality causing existing tests to fail.

Instead of removing those tests, it's better to skip them and once your codebase gets stable, re-run them to make sure everything is working fine.

test.skip('I exists, but will be skipped', () => {

})

The default list reporter will show skipped tests in yellow.

Skipping/Running tests in CI

Some projects are highly dependent on the execution environment. For example, A test of yours depends on an API_KEY which cannot be shared with everyone in the company. In this case, you can save the key with a CI like Travis and only run dependent tests in CI and not on local machine.

Japa supports this behavior as a first class citizen using runInCI method.

test.runInCI('I will execute in CI only', () => {
})

The opposite of same is also available.

test.skipInCI('I will be skipped in CI', () => {
})

Run selected tests

Just like skipping tests, you can also run a specific test using test.only method.

• If multiple tests uses test.only, then only the last one will be entertained.

test.only('all other tests will be skipped, except me', () => {
})

Retry flaky tests

Flaky tests are those, which needs a couple of retries before they can pass. Japa exposes a helpful method to retry the same test (until it passes) before marking it as failed.

Following will be executed four times in total. 3 retries + 1 actual.

test('I am flaky attimes', () => {

}).retry(3)

Regression tests

The best way to accept code contributions is to ask people to write failing tests for the bugs. Later, you fix that bug and keep the test in its place.

When creating a regression that, you are telling japa, I want this test to fail and when it fails, Japa marks it passed to keep the whole tests suite green.

test.failing('user must be verified when logging', async (assert) => {
  const user = await Db.getUser()
  user.verified = false
  
  const loggedIn = await auth.login(user)
  assert.isFalse(loggedIn, 'Expected logged in to be false for unverified user')
})

Now if login returns true, Japa will mark this test as passed and will print the assertion note.

When you fix this behavior, remove .failing and everything should be green.

Assertion Planning

Have you ever wrote one of those try/catch tests, in which you want the test to throw an exception?

test('raise error when user is null', async (assert) => {
  try {
    await auth.login(null)
  } catch ({ message }) {
    assert.equal(message, 'Please provide a user')
  }
})

Now, what if auth.login never throws an exception? The test will still be green since the catch block was never executed.

This is one of those things, which even bites seasoned programmers. To prevent this behavior, Japa asks you to plan assertions.

test('raise error when user is null', async (assert) => {
  assert.plan(1)

  try {
    await auth.login(null)
  } catch ({ message }) {
    assert.equal(message, 'Please provide a user')
  }
})

Now, if catch block is never executed, Japa will complain that you planned for 1 assertion, but never ran any.

Cleaner Error Stack

The default list reporter, will clean the stack traces by removing Japa core from it. It helps you in tracing the errors quickly.

Runner hooks

Runner hooks are executed before and after running the entire test suite. These hooks can be used to perform global actions, which are required for all test groups.

The before and after hooks can be defined inside the configure object.

const { configure } = require('japa')

configure({
  before: [
    async (runner) => {
      // setup db
    }
  ],
  after: [
    async (runner) => {
      // cleanup db
    }  
  ]
})

Japa flow

Japa will attempt to run as many tests as possible when tests or group hooks start failing.

• If one of the group fails, it will not impact the other groups.
• If one of the test fails, it will not affect other tests.

However, if lifecycle hooks will fail, they will exit early and will mark the entire group as failed. This behavior is done intentionally since if beforeEach hook is failing, then tests will get unstable, and there is no point in running them.

Check out the following flowchart to understand it better.

Running multiple test files

Sooner or later, you will have multiple tests files, that you would like to run together, instead of running one file at a time. Doing same is very simple and is achieved using a master test file.

Let's create a master test file called japaFile.js in the project root and write following code inside it.

const { configure } = require('japa')
configure({
  files: ['test/*.spec.js']
})

Now execute the file using the node command node japaFile.js.

Filtering files

Not only you can define the glob, you can also filter the test files.

const { configure } = require('japa')

configure({
  filter (filePath) {
    // return true for file(s) that you want to load
  },
  files: ['test/*.spec.js']
})

Also, you can define files to ignore with the files property. The files property is 100% compatible with fast-glob.

const { configure } = require('japa')
configure({
  files: ['test/*.spec.js', '!test/users.spec.js']
})

Configure options

Here's the list of the options the configure method accepts.

Running typescript tests

Running test files written in Typescript is a piece of cake for Japa. Since everything is done inside the Javascript files, we can ask japaFile.js to load ts-node.

// Load ts-node
require('ts-node').register()

const { configure } = require('japa')
configure({
  files: ['test/*.ts']
})

ES modules support

Japa automatically imports the ES modules with the .mjs extension. However, for files with .js extension, you will have to explicitly tell japa to import test files as ES modules and not CJS.

const { configure } = require('japa')
configure({
  files: ['test/*.js'],
  experimentalEsmSupport: true,
})

Coverage

You can use nyc to check the coverage of your code. It works for Typescript files too.

npm install -D nyc

Following is a screenshot from one of my projects.

Then, in your package.json file, just add:

{
  "scripts": {
    "test": "node japaFile.js",
    "coverage": "nyc npm run test"
  }
}