npm package discovery and stats viewer.

Discover Tips

  • General search

    [free text search, go nuts!]

  • Package details

    pkg:[package-name]

  • User packages

    @[username]

Sponsor

Optimize Toolset

I’ve always been into building performant and accessible sites, but lately I’ve been taking it extremely seriously. So much so that I’ve been building a tool to help me optimize and monitor the sites that I build to make sure that I’m making an attempt to offer the best experience to those who visit them. If you’re into performant, accessible and SEO friendly sites, you might like it too! You can check it out at Optimize Toolset.

About

Hi, 👋, I’m Ryan Hefner  and I built this site for me, and you! The goal of this site was to provide an easy way for me to check the stats on my npm packages, both for prioritizing issues and updates, and to give me a little kick in the pants to keep up on stuff.

As I was building it, I realized that I was actually using the tool to build the tool, and figured I might as well put this out there and hopefully others will find it to be a fast and useful way to search and browse npm packages as I have.

If you’re interested in other things I’m working on, follow me on Twitter or check out the open source projects I’ve been publishing on GitHub.

I am also working on a Twitter bot for this site to tweet the most popular, newest, random packages from npm. Please follow that account now and it will start sending out packages soon–ish.

Open Software & Tools

This site wouldn’t be possible without the immense generosity and tireless efforts from the people who make contributions to the world and share their work via open source initiatives. Thank you 🙏

© 2024 – Pkg Stats / Ryan Hefner

@bahmutov/cypress-code-coverage

v2.6.1

Published

My version of Cypress code coverage plugin

Downloads

34,930

Readme

@bahmutov/cypress-code-coverage cypress version CircleCI ci

My version of Cypress code coverage plugin

🎓 Covered in my course Testing The Swag Store

Install

npm install -D @bahmutov/cypress-code-coverage

Note: this plugin assumes cypress is a peer dependency already installed in your project.

Versions

| Plugin version | Cypress version | | -------------- | --------------- | | v1 | < v10 | | v2 | >= v10 |

Add to your cypress/support/e2e.js file

// https://github.com/bahmutov/cypress-code-coverage
import '@bahmutov/cypress-code-coverage/support'

Cypress v10+

Register this plugin in your Cypress config file

// cypress.config.js
module.exports = defineConfig({
  e2e: {
    setupNodeEvents(on, config) {
      // https://github.com/bahmutov/cypress-code-coverage
      require('@bahmutov/cypress-code-coverage/plugin')(on, config)

      // IMPORTANT to return the config object
      // with the any changed environment variables
      return config
    },
  },
})

Cypress v9

Add to your cypress/support/index.js file

// https://github.com/bahmutov/cypress-code-coverage
import '@bahmutov/cypress-code-coverage/support'

Register the plugin from your cypress/plugins/index.js file

// cypress/plugins/index.js
module.exports = (on, config) => {
  // https://github.com/bahmutov/cypress-code-coverage
  require('@bahmutov/cypress-code-coverage/plugin')(on, config)

  // add other tasks to be registered here

  // IMPORTANT to return the config object
  // with the any changed environment variables
  return config
}

Instrument your application

This plugin DOES NOT instrument your code. You have to instrument it yourself using Istanbul.js tool. Luckily it is not difficult. For example, if you are already using Babel to transpile you can add babel-plugin-istanbul to your .babelrc and instrument on the fly.

{
  "plugins": ["istanbul"]
}

Please see the Examples section down below, you can probably find a linked project matching your situation to see how to instrument your application's source code before running end-to-end tests to get the code coverage.

If your application has been instrumented correctly, then you should see additional counters and instructions in the application's JavaScript resources, like the image down below shows.

Instrumented code

You should see the window.__coverage__ object in the "Application under test iframe"

Window coverage object

If you have instrumented your application's code and see the window.__coverage__ object, then this plugin will save the coverage into .nyc_output folder and will generate reports after the tests finish (even in the interactive mode). Find the LCOV and HTML report in the coverage/lcov-report folder.

Coverage report

That should be it! You should see messages from this plugin in the Cypress Command Log

Plugin messages

More information

App vs unit tests

You need to instrument your web application. This means that when the test does cy.visit('localhost:3000') any code the index.html requests should be instrumented by YOU. See Examples section for advice, usually you need to stick babel-plugin-istanbul into your pipeline somewhere.

If you are testing individual functions from your application code by importing them directly into Cypress spec files, this is called "unit tests" and Cypress can instrument this scenario for you. See Instrument unit tests section.

Reports

The coverage folder has results in several formats, and the coverage raw data is stored in .nyc_output folder. You can see the coverage numbers yourself. This plugin has nyc as a dependency, so it should be available right away. Here are common examples:

# see just the coverage summary
$ npx nyc report --reporter=text-summary
# see just the coverage file by file
$ npx nyc report --reporter=text
# save the HTML report again
$ npx nyc report --reporter=lcov

It is useful to enforce minimum coverage numbers. For example:

$ npx nyc report --check-coverage --lines 80
----------|---------|----------|---------|---------|-------------------
File      | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s
----------|---------|----------|---------|---------|-------------------
All files |     100 |      100 |     100 |     100 |
 main.js  |     100 |      100 |     100 |     100 |
----------|---------|----------|---------|---------|-------------------

$ npx nyc report --check-coverage --lines 101
----------|---------|----------|---------|---------|-------------------
File      | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s
----------|---------|----------|---------|---------|-------------------
All files |     100 |      100 |     100 |     100 |
 main.js  |     100 |      100 |     100 |     100 |
----------|---------|----------|---------|---------|-------------------
ERROR: Coverage for lines (100%) does not meet global threshold (101%)

Watch video How to read code coverage report to see how to read the HTML coverage report.

Default summary after each spec file

By default, this plugin shows the coverage summary after each spec file. You can disable this summary from the cypress.json file

{
  "env": {
    "coverage": {
      "reportAfterEachSpec": false
    }
  }
}

You can also specify the reporter to use after each spec:

{
  "env": {
    "coverage": {
      "reportAfterEachSpec": "text"
    }
  }
}

The default reporter is "text-summary"

Instrument unit tests

If you test your application code directly from specs you might want to instrument them and combine unit test code coverage with any end-to-end code coverage (from iframe). You can easily instrument spec files using babel-plugin-istanbul for example.

Install the plugin

npm i -D babel-plugin-istanbul

Set your .babelrc file

{
  "plugins": ["istanbul"]
}

Put the following in cypress/plugins/index.js file to use .babelrc file

module.exports = (on, config) => {
  require('@bahmutov/cypress-code-coverage/task')(on, config)
  on(
    'file:preprocessor',
    require('@bahmutov/cypress-code-coverage/use-babelrc'),
  )
  return config
}

Now the code coverage from spec files will be combined with end-to-end coverage.

Find example of a just the unit tests and JavaScript source files with collected code coverage in examples/unit-tests-js.

Alternative for unit tests

If you cannot use .babelrc for some reason (maybe it is used by other tools?), try using the Browserify transformer included with this module in use-browserify-istanbul file.

module.exports = (on, config) => {
  require('@bahmutov/cypress-code-coverage/task')(on, config)
  on(
    'file:preprocessor',
    require('@bahmutov/cypress-code-coverage/use-browserify-istanbul'),
  )
  return config
}

Instrument backend code

Example in examples/backend folder.

You can also instrument your server-side code and produce combined coverage report that covers both the backend and frontend code

  1. Run the server code with instrumentation. The simplest way is to use nyc. If normally you run node src/server then to run instrumented version you can do nyc --silent node src/server.
  2. Add an endpoint that returns collected coverage. If you are using Express, you can simply do
const express = require('express')
const app = express()
require('@bahmutov/cypress-code-coverage/middleware/express')(app)

Tip: you can register the endpoint only if there is global code coverage object, and you can exclude the middleware code from the coverage numbers

// https://github.com/gotwarlost/istanbul/blob/master/ignoring-code-for-coverage.md
/* istanbul ignore next */
if (global.__coverage__) {
  require('@bahmutov/cypress-code-coverage/middleware/express')(app)
}

If you use Hapi server, define the endpoint yourself and return the object

if (global.__coverage__) {
  require('@bahmutov/cypress-code-coverage/middleware/hapi')(server)
}

For any other server, define the endpoint yourself and return the coverage object:

if (global.__coverage__) {
  // add method "GET /__coverage__" and response with JSON
  onRequest = (response) => response.sendJSON({ coverage: global.__coverage__ })
}
  1. Save the API coverage endpoint in cypress.json file to let the plugin know where to call to receive the code coverage data from the server. Place it in env.codeCoverage object:
{
  "env": {
    "codeCoverage": {
      "url": "http://localhost:3000/__coverage__"
    }
  }
}

That should be enough - the code coverage from the server will be requested at the end of the test run and merged with the client-side code coverage, producing a combined report.

expectBackendCoverageOnly

If there is NO frontend code coverage, and you want to only collect the backend code coverage using Cypress tests, set expectBackendCoverageOnly: true in cypress.json file. Otherwise Cypress complains that it cannot find the frontend code coverage.

Default:

No frontend code coverage warning

After:

{
  "env": {
    "codeCoverage": {
      "url": "http://localhost:3003/__coverage__",
      "expectBackendCoverageOnly": true
    }
  }
}

Cypress knows to expect the backend code coverage only

Custom report folder

You can specify custom report folder by adding nyc object to the package.json file. For example to save reports to cypress-coverage folder, use:

{
  "nyc": {
    "report-dir": "cypress-coverage"
  }
}

Custom reporters

You can specify custom coverage reporter(s) to use. For example to output text summary and save JSON report in cypress-coverage folder set in your package.json folder:

{
  "nyc": {
    "report-dir": "cypress-coverage",
    "reporter": ["text", "json"]
  }
}

Tip: find list of reporters here

Custom NYC command

Sometimes NYC tool might be installed in a different folder not in the current or parent folder, or you might want to customize the report command. In that case, put the custom command into package.json in the current folder and this plugin will automatically use it.

{
  "scripts": {
    "coverage:report": "call NYC report ..."
  }
}

TypeScript users

TypeScript source files should be automatically included in the report, if they are instrumented.

See examples/ts-example, bahmutov/cra-ts-code-coverage-example or bahmutov/cypress-angular-coverage-example.

Include code

By default, the code coverage report includes only the instrumented files loaded by the application during the tests. If some modules are loaded dynamically, or are loaded by the pages NOT visited during any tests, these files are not going to be in the report - because the plugin does not know about them. You can include all expected source files in the report by using include list in the package.json file. The files without counters will have 0 percent code coverage.

For example, if you want to make sure the final report includes all JS files from the "src/pages" folder, set the "nyc" object in your package.json file.

{
  "nyc": {
    "all": true,
    "include": "src/pages/*.js"
  }
}

See example examples/all-files

Exclude code

You can exclude parts of the code or entire files from the code coverage report. See Istanbul guide. Common cases:

Filter files

Specify the list of files to exclude from the saved coverage report using E2E env object in your cypress.config.js file.

// cypress.config.js
module.exports = defineConfig({
  e2e: {
    coverage: {
      // exclude the service worker files
      exclude: ['**/src/service*.js'],
    },
  },
})

You can try excluding the default specs and support files only using

// cypress.config.js
module.exports = defineConfig({
  e2e: {
    coverage: {
      exclude: true,
    },
  },
})

Exclude "else" branch

When running code only during Cypress tests, the "else" branch will never be hit. Thus we should exclude it from the branch coverage computation:

// expose "store" reference during tests
/* istanbul ignore else */
if (window.Cypress) {
  window.store = store
}

Exclude next logical statement

Often needed to skip a statement

/* istanbul ignore next */
if (global.__coverage__) {
  require('@bahmutov/cypress-code-coverage/middleware/express')(app)
}

Or a particular switch case

switch (foo) {
  case 1 /* some code */:
    break
  /* istanbul ignore next */
  case 2: // really difficult to hit from tests
    someCode()
}

Exclude files and folders

See nyc configuration and include and exclude options. You can include and exclude files using minimatch patterns in .nycrc file or using "nyc" object inside your package.json file.

For example, if you want to only include files in the app folder, but exclude app/util.js file, you can set in your package.json

{
  "nyc": {
    "include": ["app/**/*.js"],
    "exclude": ["app/util.js"]
  }
}

Note: if you have all: true NYC option set, this plugin will check the produced .nyc_output/out.json before generating the final report. If the out.json file does not have information for some files that should be there according to include list, then an empty placeholder will be included, see PR 208.

Another important option is excludeAfterRemap. By default it is false, which might let excluded files through. If you are excluding the files, and the instrumenter does not respect the nyc.exclude setting, then add excludeAfterRemap: true to tell nyc report to exclude files. See examples/exclude-files.

Quiet mode

You can hide plugin's Command Log messages by using the quiet: true option

// cypress.config.js objects
{
  e2e: {
    env: {
      coverage: {
        // false by default
        quiet: true
      }
    }
  }
}

Disable plugin

You can skip the client-side code coverage hooks by setting the environment variable coverage to false.

# tell Cypress to set environment variable "coverage" to false
cypress run --env coverage=false
# or pass the environment variable
CYPRESS_coverage=false cypress run

or set it to false in the cypress.json file

{
  "env": {
    "coverage": false
  }
}

If you have an object with coverage settings, you can still disable it

{
  "env": {
    "coverage": {
      "disable": true
    }
  }
}

Tip: you can use disable: true or disabled: true

See Cypress environment variables and support.js. You can try running without code coverage in this project yourself

# run with code coverage
npm run dev
# disable code coverage
npm run dev:no:coverage

Merge coverage reports

When running tests in parallel, you can download all coverage folders into one top folder and then merge the coverage and regenerate new combined report using cc-merge command included in this module.

For example, if this is the top folder with downloaded coverage subfolders:

project/
  reports/
    one/
      coverage-final.json
    two/
      coverage-final.json
    three/
      coverage-final.json

Use this command to combine all coverage-final.json files

npx cc-merge reports

The coverage will be written into .nyc_output folder (which is automatically cleared) and the reports will be written into coverage folder.

Links

GitHub Actions

When using this plugin with GitHub Actions, if reporting is enabled, then each job summary will include the current final percentages in the Actions summary report.

Summary table

The table headers emoji depend on the percentage coverage

| Coverage % | Emoji | | ---------- | ----- | | >= 95 | ✅ | | >= 90 | 🏆 | | >= 80 | 🥇 | | >= 70 | 🥈 | | >= 60 | 🥉 | | >= 50 | 📈 | | >= 40 | ⚠️ | | < 40 | 🪫 |

Examples

Internal examples

Full examples we use for testing in this repository:

External examples

Look up the list of examples under GitHub topic cypress-code-coverage-example

Migrations

v2 to v3

Change the plugins file cypress/plugins/index.js

// BEFORE
module.exports = (on, config) => {
  on('task', require('@bahmutov/cypress-code-coverage/task'))
}
// AFTER
module.exports = (on, config) => {
  require('@bahmutov/cypress-code-coverage/task')(on, config)
  // IMPORTANT to return the config object
  // with the any changed environment variables
  return config
}

Tip: we include plugins.js file you can point at from your code in simple cases. From your cypress.json file:

{
  "pluginsFile": "node_modules/@bahmutov/cypress-code-coverage/plugins",
  "supportFile": "node_modules/@bahmutov/cypress-code-coverage/support"
}

See examples/use-plugins-and-support

Debugging

This plugin uses debug module to output additional logging messages from its task.js file. This can help with debugging errors while saving code coverage or reporting. In order to see these messages, run Cypress from the terminal with environment variable DEBUG=code-coverage. Example using Unix syntax to set the variable:

$ DEBUG=code-coverage npm run dev
...
  code-coverage reset code coverage in interactive mode +0ms
  code-coverage wrote coverage file /code-coverage/.nyc_output/out.json +28ms
  code-coverage saving coverage report using command: "nyc report --report-dir ./coverage --reporter=lcov --reporter=clover --reporter=json" +3ms

Deeply nested object will sometimes have [object Object] values printed. You can print these nested objects by specifying a deeper depth by adding DEBUG_DEPTH= setting

$ DEBUG_DEPTH=10 DEBUG=code-coverage npm run dev

Common issues

Common issue: not instrumenting your application when running Cypress.

If the plugin worked before in version X, but stopped after upgrading to version Y, please try the released versions between X and Y to see where the breaking change was.

If you decide to open an issue in this repository, please fill all information the issue template asks. The issues most likely to be resolved have debug logs, screenshots and hopefully public repository links so we can try running the tests ourselves.

Contributing

You can test changes locally by running tests and confirming the code coverage has been calculated and saved.

npm run test:ci
# now check generated coverage numbers
npx nyc report --check-coverage true --lines 80
npx nyc report --check-coverage true --lines 100 --include cypress/about.js
npx nyc report --check-coverage true --lines 100 --include cypress/unit.js

Tip: use check-code-coverage for stricter code coverage checks than nyc report --check-coverage allows.

Markdown

You can validate links in Markdown files in this directory by executing (Linux + Mac only) script

npm run check:markdown

License

This project is licensed under the terms of the MIT license. It was originally forked from cypress-io/code-coverage.