npm package discovery and stats viewer.

Discover Tips

  • General search

    [free text search, go nuts!]

  • Package details

    pkg:[package-name]

  • User packages

    @[username]

Sponsor

Optimize Toolset

I’ve always been into building performant and accessible sites, but lately I’ve been taking it extremely seriously. So much so that I’ve been building a tool to help me optimize and monitor the sites that I build to make sure that I’m making an attempt to offer the best experience to those who visit them. If you’re into performant, accessible and SEO friendly sites, you might like it too! You can check it out at Optimize Toolset.

About

Hi, 👋, I’m Ryan Hefner  and I built this site for me, and you! The goal of this site was to provide an easy way for me to check the stats on my npm packages, both for prioritizing issues and updates, and to give me a little kick in the pants to keep up on stuff.

As I was building it, I realized that I was actually using the tool to build the tool, and figured I might as well put this out there and hopefully others will find it to be a fast and useful way to search and browse npm packages as I have.

If you’re interested in other things I’m working on, follow me on Twitter or check out the open source projects I’ve been publishing on GitHub.

I am also working on a Twitter bot for this site to tweet the most popular, newest, random packages from npm. Please follow that account now and it will start sending out packages soon–ish.

Open Software & Tools

This site wouldn’t be possible without the immense generosity and tireless efforts from the people who make contributions to the world and share their work via open source initiatives. Thank you 🙏

© 2024 – Pkg Stats / Ryan Hefner

@postlight/ci-failed-test-reporter

v1.0.26

Published

đź“ť A tool for outputting JSON test reports as PR comments

Downloads

311,356

Readme

CI Failed Test Reporter

A PR comment posted with the CI Failed Test Reporter

A familiar scene: you open up a PR only to see that your CI build failed because of some tests that didn't pass. But which tests? GitHub won't tell you. Your best options for finding out include opening a console and running your test suite locally or manually sifting through the CI logs, and neither of these is as efficient as you'd like.

This tool was built to facilitate this process—when your CI build breaks due to failing tests, it reads the JSON test report generated by your testing framework, formats it into markdown, and posts it as a comment directly on your PR. With this tool, you can see which tests broke the build in the same place you find out it's broken, no context-switching necessary.

Installation

yarn add @postlight/ci-failed-test-reporter

# or

npm install @postlight/ci-failed-test-reporter

Note on testing frameworks

This package is currently only compatible with JSON test results generated by Jest and Mocha, but we're hoping to add additional support in the future.

Usage

The basic usage is simple: First, you'll run your tests in a way that outputs the results to a JSON file:

Jest:

jest --json --outputFile test-output.json

Mocha:

mocha --recursive './src/tests/mocha/*.js' --reporter json > test-output.json 

You'll likely want to add this as a script to your package.json, like so:

Jest:

  "scripts": {
    "test-with-output": "jest --json --outputFile test-output.json",
  }

Mocha:

    "scripts": {
    "test-with-output": "mocha --recursive './src/tests/mocha/*.js' --reporter json > test-output.json",
  }

Next, you run ciftr (ci-failed-test-reporter) to parse the test results and report failed tests:

yarn ciftr test-output.json

Before that will work in your CI environment, you'll need to do two things:

  1. Set up your CI config
  2. Create a GitHub API key and add it to your CI environment variables

Currently, this tool currently only works with CircleCI, but we're looking into opening it up to other CI solutions.

CI Setup

This tool has only been tested using CircleCI and Travis, but should be able to work with any CI solution that allows you to set the proper environment variables. Below, we've outlined the process for setting up a CircleCI config for use with the tool. The process should be somewhat similar across CI solutions—make sure to look at the CI tool's docs to determine what needs to be done differently.

Setting up a CircleCI config

To set things up with CircleCI, there's just one run block you'll need to add to your .circleci/config.yml file.

After the step where you run your tests, you'll run ciftr:

- run: yarn test
- run:
    name: Upload Test Report
    command: yarn ciftr test-report.json # this should mirror whereever you've saved your test results
    when: on_fail # only run this when tests have failed

You can check out this CircleCI config for a full working example. The only thing you will definitely want to change is the working_directory value, which should be changed to the name of your repo. Note that this config assumes you're saving your test reports as test-report.json in the root directory.

Setting up a Travis config

To set things up with Travis, you only need to add the following step to your .travis.yml file:

after_failure:
  - yarn ciftr /test-report.json

You'll be all set after that! Again, this assumes that the testing framework is writing test results in a file called test-report.json in the root directory of your project. Check out our example Travis config for a working example that you can use as a guide.

Using this project with another CI solution

While we've only tested this package with Travis and CircleCI, it should work swimmingly with any CI solution that allows you to access a few key pieces of info about the current build. The following environment variables must be defined—you may be able to export them as part of a step in your build process.

CIFTR_GITHUB_API_KEY="your github API key"
CIFTR_PR_USERNAME="the repo owner's username"
CIFTR_PR_NUMBER="the number of the pull request"
CIFTR_PR_REPONAME="the repo name"

Here's an example of how you might manually export the appropriate environment variables, using CircleCI as an example (remember that this package supports CircleCI, so need to use this code in your actual config). The process for another CI tool may or may not look similar.

  - run:
      name: Define Environment Variables at Runtime
      command: |
        echo 'export CIFTR_PR_REPONAME=${CIRCLE_PROJECT_REPONAME}' >> $BASH_ENV
        echo 'export CIFTR_PR_USERNAME=${CIRCLE_PROJECT_USERNAME}' >> $BASH_ENV
        # grep just the pr number from the PR URL
        echo 'export CIFTR_PR_NUMBER=$(echo $CIRCLE_PULL_REQUEST | grep -Eo "\/pull\/([0-9]+)" | grep -Eo "[0-9]+")' >> $BASH_ENV
        source $BASH_ENV

Setting Up Your GitHub API Key Environment Variable

The only environment variable you need to define for use through the CircleCI webapp is CIFTR_GITHUB_API_KEY, which must be populated with your GitHub API key. This can be the API key of any user with access to the repo—at Postlight, we've created a postlight-bot user and recommend you do similarly. In order to create a GitHub API key, start here. The rest of the necessary environment variables are built into CircleCI and are exported in your CircleCI config file, as detailed [above](#Set up your CircleCI config).

TODO

  • Add support for other JS testing frameworks, or possibly add bring-your-own-framework functionality
  • Allow configuration of comment template
  • Add example config setup for Jenkins

Pull requests are more than welcome!


🔬 A project from your friends at Postlight Labs