backstop-retry-failed-scenarios
v1.3.0
Published
A wrapper command to retry failed scenario for BackstopJS
Downloads
2,218
Readme
backstop-retry-failed-scenarios
A wrapper command to retry failed scenario for BackstopJS.
It parses report.json, extract failed tests, and rerun with --filter
option, and rewrite result file.
This is useful when your tests are unstable.
FEATURES
- Invoke specified test command
- Generate filter option from failed tests
- Reorganize each failed reports to latest one status
- Supports HTML report, JSON report, CI report.
EXAMPLE
This endpoint returns random emoji from three emojis(😀, 😇, 😝). Each try may fail, but retrying failed tests will pass all test.
- https://blog.sushi.money/random_face
$ cd examples/retry
$ backstop reference
$ npx backstop-retry-failed-scenarios
HOW TO RUN
Instead of backstop test
, run backstop-retry-failed-scenarios
and set command, retry count, config path.
$ npm install -g backstop backstop-retry-failed-scenarios
$ backstop init
$ backstop reference
$ backstop-retry-failed-scenarios --retry 5 --command 'backstop test' --config backstop.js
Instead of npm install, you can pull hitode909/backstop-retry-failed-scenarios from DockerHub.
Its image is based on backstopjs/backstopjs, so you can execute backstop directly from docker run hitode909/backstop-retry-failed-scenarios
.
$ docker pull hitode909/backstop-retry-failed-scenarios
$ docker run --rm -v $(pwd):/src hitode909/backstop-retry-failed-scenarios backstop init
$ docker run --rm -v $(pwd):/src hitode909/backstop-retry-failed-scenarios backstop reference
$ docker run --rm -v $(pwd):/src hitode909/backstop-retry-failed-scenarios backstop-retry-failed-scenarios
If you want to recreate references every time, Please set --reference-command
option. This will do like this.
- create all references
- run all tests
- create references for failed tests on 2.
- run tests for failed tests on 2.
This is useful when your reference refers to production instead static file, and test refers to staging environment.
$ backstop init
$ backstop-retry-failed-scenarios --reference-command 'backstop reference' --command 'backstop test' --config backstop.js
OPTIONS
--retry number Trial count. default: 2(Run once, retry once). minimum: 2
--config string Path to config file. default: backstop.json
--command string Command to run test. default: backstop test
--reference-command string Command to create reference before testing. Default: null (Do not create reference before test).
--output-profile string Path to profiler output file. If present, measure the execution time and output the report as JSON.