npm package discovery and stats viewer.

Discover Tips

  • General search

    [free text search, go nuts!]

  • Package details

    pkg:[package-name]

  • User packages

    @[username]

Sponsor

Optimize Toolset

I’ve always been into building performant and accessible sites, but lately I’ve been taking it extremely seriously. So much so that I’ve been building a tool to help me optimize and monitor the sites that I build to make sure that I’m making an attempt to offer the best experience to those who visit them. If you’re into performant, accessible and SEO friendly sites, you might like it too! You can check it out at Optimize Toolset.

About

Hi, 👋, I’m Ryan Hefner  and I built this site for me, and you! The goal of this site was to provide an easy way for me to check the stats on my npm packages, both for prioritizing issues and updates, and to give me a little kick in the pants to keep up on stuff.

As I was building it, I realized that I was actually using the tool to build the tool, and figured I might as well put this out there and hopefully others will find it to be a fast and useful way to search and browse npm packages as I have.

If you’re interested in other things I’m working on, follow me on Twitter or check out the open source projects I’ve been publishing on GitHub.

I am also working on a Twitter bot for this site to tweet the most popular, newest, random packages from npm. Please follow that account now and it will start sending out packages soon–ish.

Open Software & Tools

This site wouldn’t be possible without the immense generosity and tireless efforts from the people who make contributions to the world and share their work via open source initiatives. Thank you 🙏

© 2024 – Pkg Stats / Ryan Hefner

@mastermindzh/cypress-testrail-reporter

v1.4.2-beta1

Published

A Testrail reporter for cypress including TestRail API basic library

Downloads

2,332

Readme

TestRail Reporter for Cypress

version downloads MIT License

Publishes Cypress runs on TestRail.

Core features:

  • Test results are aggregated under the same test run if you are executing more spec(test) files and they are belongs to the same suite
  • Results are reported immediately after single test execution (real-time reporting)
  • Test run would be closed after last spec(test) file has been finished
  • Possibility to upload screenshots for failed and retried test cases - optional (allowFailedScreenshotUpload: true)
  • Multi suite project support (set suiteId=1 in cypress.json or set it as a part of runtime environment variables as testRailSuiteId=1)
  • Reporting retest status of a test cases - handy in terms of marking tests as flaky (test is reported with retest status for the first try and after second try it passes) Note: cypress retry logic must be enabled for this feature.

Important note: Keep your tests under the one of the following folder structure:

  • cypress/e2e - default path for Cypress v10 and above
  • cypress/integration - default path for Cypress v9 and bellow
  • cypress/tests

Install

$ npm install cypress-testrail-reporter --save-dev

Usage

Add reporter to your cypress.json:

...
"reporter": "cypress-testrail-reporter",
"reporterOptions": {
  "host": "https://yourdomain.testrail.com",
  "username": "username",
  "password": "password",
  "projectId": 1,
  "suiteId": 1,
}

Your Cypress tests should include the ID of your TestRail test case. Make sure your test case IDs are distinct from your test titles:

// Good:
it("C123 C124 Can authenticate a valid user", ...
it("Can authenticate a valid user C321", ...

// Bad:
it("C123Can authenticate a valid user", ...
it("Can authenticate a valid userC123", ...

Reporter Options

host: string host of your TestRail instance (e.g. for a hosted instance https://instance.testrail.com).

username: string email of the user under which the test run will be created. When you set CYPRESS_TESTRAIL_REPORTER_USERNAME in environment variables, this option would be overwritten with it.

password: string password or the API key for the aforementioned user. When you set CYPRESS_TESTRAIL_REPORTER_PASSWORD in runtime environment variables, this option would be overwritten with it.

projectId: number project with which the tests are associated.

suiteId: number suite with which the tests are associated. Optional under cypress.json file in case that you define suiteId under gitlab-ci.yml file or set this value in runtime environment varables.

runName: string (optional) name of the Testrail run. When you set CYPRESS_TESTRAIL_REPORTER_RUNNAME in runtime environment variables, this option would be overwritten with it.

singleRun: string (optional) Causes the reporter to add all results into a single test run in Testrail

disableDescription: bool (optional: default is false) possibility to disable description for test run in case that someone don’t have cypress dashboard feature (disableDescription: true)

allowFailedScreenshotUpload: bool (optional: default is false) will upload failed screenshot to corresponding test result comment for easier debugging of failure.

includeAllInTestRun: bool (optional: default is true) will return all test cases in test run. set to false to return test runs based on filter or section/group.

groupId: string (optional: needs "includeAllInTestRun": false ) The ID of the section/group. When you set CYPRESS_TESTRAIL_REPORTER_GROUPID in runtime environment variables, this option would be overwritten with it.

filter: string (optional: needs "includeAllInTestRun": false) Only return cases with matching filter string in the case title

Multiple suite

This reporter can handle multiple suite project in TestRail. In order to use it, don't define suiteId under cypress.json file and instead you should pass testRailSuiteId variable when you define all other CLI agruments for cypress execution(through command line). If you are using CI integration solution (e.g. GitLab) testRailSuiteId can be set before every pipeline job or predefined for each spec (test) file for which suiteId belongs to.

gitlab-ci.yml file (Here you can pass suiteId as a variable):


e2e_test1:
  script:
    - e2e-setup.sh
  variables:
    CYPRESS_SPEC: "cypress/integration/dashboard/*"
    TESTRAIL_SUITEID: 1

e2e_test2:
  script:
    - e2e-setup.sh
  variables:
    CYPRESS_SPEC: "cypress/integration/login/*"
    TESTRAIL_SUITEID: 2

and use it later during cypress run:

e2e-setup.sh file


CYPRESS_OPTIONS="baseUrl=${url},trashAssetsBeforeRuns=false,video=${video},screenshotOnRunFailure=${screenshotOnRunFailure}"
CYPRESS_ENV="testRailSuiteId=${TESTRAIL_SUITEID}"

npx cypress run --headed --browser chrome --config "${CYPRESS_OPTIONS}" --env="${CYPRESS_ENV}" --spec "${CYPRESS_SPEC}"

Cucumber preprocessor

This reporter can miss spec files if they are suffixed as .feature or if you are not using the default cypress/integration folder. In order to use it with the Cucumber Preprocess, you should pass the location of your spec files cypress/tests/**/*.feature when you define all other CLI agruments for cypress execution(through command line). If you are using CI integration solution (e.g. GitLab) CYPRESS_SPEC can be set before every pipeline job


CYPRESS_SPEC="cypress/tests/**/*.feature"

npx cypress run --headed --browser chrome --spec "${CYPRESS_SPEC}"

TestRail Settings

To increase security, the TestRail team suggests using an API key instead of a password. You can see how to generate an API key here.

If you maintain your own TestRail instance on your own server, it is recommended to enable HTTPS for your TestRail installation.

For TestRail hosted accounts maintained by Gurock, all accounts will automatically use HTTPS.

You can read the whole TestRail documentation here.

Author

Milutin Savovic - github

Core contributors

License

This project is licensed under the MIT license.

Acknowledgments