npm package discovery and stats viewer.

Discover Tips

  • General search

    [free text search, go nuts!]

  • Package details

    pkg:[package-name]

  • User packages

    @[username]

Sponsor

Optimize Toolset

I’ve always been into building performant and accessible sites, but lately I’ve been taking it extremely seriously. So much so that I’ve been building a tool to help me optimize and monitor the sites that I build to make sure that I’m making an attempt to offer the best experience to those who visit them. If you’re into performant, accessible and SEO friendly sites, you might like it too! You can check it out at Optimize Toolset.

About

Hi, 👋, I’m Ryan Hefner  and I built this site for me, and you! The goal of this site was to provide an easy way for me to check the stats on my npm packages, both for prioritizing issues and updates, and to give me a little kick in the pants to keep up on stuff.

As I was building it, I realized that I was actually using the tool to build the tool, and figured I might as well put this out there and hopefully others will find it to be a fast and useful way to search and browse npm packages as I have.

If you’re interested in other things I’m working on, follow me on Twitter or check out the open source projects I’ve been publishing on GitHub.

I am also working on a Twitter bot for this site to tweet the most popular, newest, random packages from npm. Please follow that account now and it will start sending out packages soon–ish.

Open Software & Tools

This site wouldn’t be possible without the immense generosity and tireless efforts from the people who make contributions to the world and share their work via open source initiatives. Thank you 🙏

© 2024 – Pkg Stats / Ryan Hefner

cymetrics

v1.0.3

Published

Cymetrics is a Cypress post process that looks at durations on an individual test case level and at a file level, to determine the fastest completion time for the amount of test runners available. The average duration of multiple runs will be used, so the

Downloads

3

Readme

Table of Contents

INTRODUCTION

Cymetrics is a Cypress post process that looks at durations on an individual test case level and at a file level, to determine the fastest completion time for the amount of test runners available. The average duration of multiple runs will be used, so the more cymetrics is used, the more accurate it becomes.

Check out the scenario section for a better understanding of cymetrics, or the sample project, cypress-playground demonstrating cymetrics' use.

Example

Scenario

The Cypress test suite contains 4 files and each file contains 2 test cases. We normally run this suite nightly on a single Jenkins runner. We recently got a second Jenkins test runner and would like to as evenly as possible distribute the tests between our 2 test runners in order to get the shortest overall test duration.

After one run of the entire test suite we get the following information:

  • file_1.js: 10 minutes
    • tc_1.1:......2 minutes
    • tc_1.2:......8 minutes
  • file_2.js: 2 minutes
    • tc_2.1.......1 minute
    • tc_2.2.......1 minute
  • file_3.js: 15 minutes
    • tc_3.1:......3 minutes
    • tc_3.2:......12 minutes
  • file_4.js: 9 minutes
    • tc_4.1:......4 minutes
    • tc_4.2:......5 minutes

Total Runtime WITHOUT cymetrics: 36 minutes

Cymetrics will now take this information and find the shortest overall duration possible given 2 test runners.

Cymetric Test Case Level Output

Scenario output at the test case level: (Total Runtime WITH cymetrics: 18 minutes)

  • Test Runner 1
    • tc_3.2.......12 minutes
    • tc_4.1.......4 minutes
    • tc_1.1.......2 minutes
  • Test Runner 2
    • tc_1.2.......8 minutes
    • tc_4.2.......5 minutes
    • tc_3.1.......3 minutes
    • tc_2.1.......1 minute
    • tc_2.2.......1 minute

Real output would look something like this:

{
  "g1": {
    "files": [
      "tc_3.2",
      "tc_4.1",
      "tc_1.1"
    ],
    "estTotalDuration": 1080000
  },
  "g2": {
    "files": [
      "tc_1.2",
      "tc_4.2",
      "tc_3.1",
      "tc_2.1",
      "tc_2.2"
    ],
    "estTotalDuration": 1080000
  }
}

Cymetric File Level Output

Scenario output at the file level: (Total Runtime WITH cymetrics: 19 minutes)

  • Test Runner 1
    • file_3.js...... 15 minutes
    • file_2.js.... 2 minutes
  • Test Runner 2
    • file_1.js.....10 minutes
    • file_4.js.......9 minutes

Real output would look something like this:

{
  "g1": {
    "title": [
      "cypress/integration/file_3.js",
      "cypress/integration/file_2.js"
    ],
    "estTotalDuration": 1080000,
    "command": "npm run spec 'cypress/integration/file_3.js','cypress/integration/file_2.js',"
  },
  "g2": {
    "title": [
      "cypress/integration/file_1.js",
      "cypress/integration/file_4.js"
    ],
    "estTotalDuration": 1080000,
    "command": "npm run spec 'cypress/integration/file_1.js','cypress/integration/file_4.js',"
  }
}

REQUIREMENTS

Cymetrics requires Mochawesome reporting as input to produce the final metrics.

In order to use mochawesome correctly, we will add async and child_process to the /plugins/index.js file. These are optional and feel free to stray from these packages.

INSTALLATION

The following installations assumes you have met the requirements within the requirements section.

Async

  • npm install async --save-dev

Mochawesome

  • npm install mocha --save-dev
  • npm install cypress-multi-reporters --save-dev
  • npm install mochawesome --save-dev
  • npm install mochawesome-merge --save-dev
  • npm install mochawesome-report-generator --save-dev

Set the mochawesome reporter settings within the cypress.json file

{
  "reporter": "mochawesome",
  "reporterOptions": {
    "reportDir": "cypress/reports/testCaseResults",
    "overwrite": false,
    "html": false,
    "json": true
  }
}

Add these lines to the package.json in the script section.

{
  "scripts": {
    "clean-reports": "npx rimraf cypress/reports/\"**\"",
    "posttest": "npm run mocha-merge && npm run mocha-marge",
    "mocha-merge": "npx mochawesome-merge 'cypress/reports/testCaseResults/*json' > cypress/reports/mochawesomeMerged.json",
    "mocha-marge": "npx marge cypress/reports/mochawesomeMerged.json -f report -o cypress/reports/html"
  }
}

Within the plugins/index.js call the mochawesome scripts

on('before:run', (config) => {
  series([() => exec('npm run clean-reports')]);
})

on('after:run', async (config) => {
  series([() => exec('npm run posttest')]);
})

Cymetrics

npm install cymetrics

Set the mochawesome reporter settings within the cypress.json file

{
  "load_balancer": {
    "outputDirectory": "output",
    "testCaseOutputFileName": "tcDataOutput",
    "fileOutputFileName": "fileDataOutput",
    "balancedFilesOutputFileName": "balancedFiles",
    "balancedTcOutputFileName": "balancedTestCases",
    "testRunnerCount": "2",
    "mochawesomeReport": "cypress/reports/mochawesomeMerged.json"
  }
}

Within the plugins/index.js call cymetrics in an after:run block

on('after:run', async (config) => {
        await cymetrics.balance(config)
    })

Note: the final plugins/index.js file should look like this:

const cymetrics = require('cymetrics')
const series = require('async').series
const {exec} = require('child_process');

module.exports = (on, config) => {
  on('before:run', (config) => {
    series([() => exec('npm run clean-reports')]);
  })
  on('after:run', async (config) => {
    series([() => exec('npm run posttest')]);
    await cymetrics.balance(config)
  })
}

Options

outputDirectory - Folder location where cymetrics output will placed
testCaseOutputFileName - File name for unbalanced data on all the test cases 
fileOutputFileName - File name for unbalanced data on all files with test cases
balancedFilesOutputFileName - File name for balanced data for all files with test cases
balancedTcOutputFileName - File name for balanced data on all the test cases
testRunnerCount - The amount of 'baskets' to split up test cases and test files
mochawesomeReport - Location of mochawesome file with combined results

MAINTAINERS

Jacob Bles