purpleteam

v4.0.0-alpha.3

Published

3 years ago

CLI for driving purpleteam -- security regression testing SaaS

Downloads

If you are planning on running the local environment, once you have installed, configured and are ready to run the PurpleTeam CLI, head back to the local setup documentation and make sure all of the other PurpleTeam components are also set-up and ready to run. After that work through the local workflow documentation.

If you are planning on targeting the cloud environment, the PurpleTeam CLI is all you need to have set-up.

If you have any issues with the set-up, be sure to check the trouble shooting page.

Minimum Supported Versions

NodeJS: v14

Install

There are several options.

Clone the git repository

If you are planning on running/debugging purpleteam stand-alone, cloning is a good option.

From a directory that you would like the CLI cloned to run the following command:

git clone https://github.com/purpleteam-labs/purpleteam.git

Install the dependencies with the following command:

npm install

Another option with cloning if you want the CLI (purpleteam) to be available as a system wide command is to use the following command from the repositories root directory:

npm link

NPM install locally

If you are planning on running/debugging purpleteam from another NodeJS process, for example a CI/nightly build/build pipeline project of your own, installing via NPM is a good option.

For the locally installed via NPM option the purpleteam-labs Team uses the purpleteam-build-test-cli project as an example to test that this option works as expected. The following example package.json and index.js files are from the purpleteam-build-test-cli example project. Feel free to clone it, or use your own project to follow along.

This example exports two environment variables:

NODE_ENV=local: Means that purpleteam will be using the local configuration. If instead you have signed up for a cloud license, you will want to be targeting the cloud environment instead
PURPLETEAM_UI=noUi: As detailed in the Configure sub-section

Using the above mentioned example build project files, and for the sake of this example, let's assume your NodeJS build project has the same following files:

package.json

{
  "name": "purpleteam-build-test-cli",
  "description": "Used to test that the purpleteam CLI runs within a build pipeline successfully",
  "main": "index.js",
  "scripts": {
    "// Don't forget to export any required env vars before running the purpleteam CLI. For example": "NODE_ENV=local and PURPLETEAM_UI=noUi",
    "// Invoke purpleteam binary from NPM script": "npm run purpleteam",
    "purpleteam": "NODE_ENV=local purpleteam",
    "// Start your node app": "npm start",
    "start": "NODE_ENV=local PURPLETEAM_UI=noUi node index.js",
    "// Debug your node app": "npm run debugApp",
    "debugApp": "NODE_ENV=local PURPLETEAM_UI=noUi node --inspect-brk=localhost:9230 index.js",
    "// Debug your node app and the purpleteam CLI": "npm run debugAppAndCli",
    "debugAppAndCli": "NODE_ENV=local PURPLETEAM_UI=noUi DEBUG_PURPLETEAM=true node --inspect-brk=localhost:9230 index.js"
  },
  "dependencies": {
    "purpleteam": "*"
  }
}

index.js

const { spawn } = require('child_process');

// You will need to define two debuggers in what ever tool you're using.
// localhost:9230 and localhost:9231
const execArgvDebugString = '--inspect-brk=localhost';
const childProcessInspectPort = 9231;
// You can run any of the purpleteam commands [about|status|test|testplan], `test` is just one example.
const purpleteamArgs = ['purpleteam', 'test'];

startPurpleteam = () => {
  const purpleteam = spawn('node', [
    ...(process.env.DEBUG_PURPLETEAM
    ? [`${execArgvDebugString}:${childProcessInspectPort}`]
    : []),
    ...purpleteamArgs],
    { cwd: `${process.cwd()}/node_modules/.bin/`, env: process.env, argv0: process.argv[0] }
  );

  purpleteam.stdout.on('data', (data) => {
    process.stdout.write(data);
  });
  purpleteam.stderr.on('data', (data) => {
    process.stdout.write(data);
  });
  purpleteam.on('exit', (code, signal) => {
    console.debug(`Child process "purpleteam" exited with code: "${code}", and signal: "${signal}".`);
  });
  purpleteam.on('close', (code) => {
    console.debug(`"close" event was emitted with code: "${code}" for "purpleteam".`);
  });
  purpleteam.on('error', (err) => {
    process.stdout.write(`Failed to start "purpleteam" sub-process. The error was: ${err}.`);
  });
};

startPurpleteam();

From within your NodeJS build project run the following command to install the PurpleTeam CLI locally into your NodeJS project:

npm install

NPM install globally

For example, you may have a build project/pipeline that is written in some language besides JavaScript. In this case the most suitable install technique may be to install the PurpleTeam CLI globally.

To do so, run the following command:

npm install -g purpleteam

Now the PurpleTeam CLI is installed and on your path to invoke from anywhere on your system.

which purpleteam
# Will print where purpleteam is located.
# You will need this to configure it if you choose to install globally.

As mentioned under the Clone section, another option for a system wide install is to use npm link.

Configure

CLI

No matter which install option you decide on the PurpleTeam CLI will require configuration.

If you are planning on using the cloud environment copy the config/config.example.cloud.json to config/config.cloud.json and make the necessary changes. If you are planning on using the local environment copy the config/config.example.local.json to config/config.local.json and make the necessary changes.

Use the config/config.js for documentation and further examples.

`loggers.testerProgress.dirname`

Configure this property. This is where the CLI logs to. Additional details can be found on the Log and Outcomes files page.

`loggers.testPlan.dirname`

Configure this property. Using the same value as used for loggers.testerProgress.dirname is an option. This is where the CLI logs the test plans to when running in noUi mode. Additional details can be found on the Log and Outcomes files page.

`purpleteamApi`

If you are planning on using the local environment you can stick with the default property values. If you are planning on using the cloud environment you will be given this information when you sign-up for a PurpleTeam account.

`purpleteamAuth`

`job.fileUri`

Configure this property if you do not want to manually pass it as an argument to the CLI. This is the Job file you have configured to specify your System Under Test (SUT) details.

If you installed the PurpleTeam CLI via git clone (You are intending to run PurpleTeam CLI stand-alone), then a relative directory path from the root of the repository ("./testResources/jobs/your_job_file") is acceptable.
If you installed the PurpleTeam CLI via npm install Then it's more likely that you will need this path to be absolute, as the current directory (./) is more than likely not going to be within the PurpleTeam CLI project itself, but rather wherever the purpleteam binary is itself.

This value can be overridden by passing it in as an option to the commands that require it (currently test and testplan).

`outcomes.dir`

Configure this property. This is a directory of your choosing that Outcomes files from the PurpleTeam API (orchestrator if running in local env, AWS API Gateway if running in cloud env) will be persisted to. Additional details can be found on the Log and Outcomes files page.

`uI`

This property is configured by default to use the character user interface (cUi value) (your terminal). This value can be changed in one of the following ways:

Change it directly in config/config.js
Add an override to your: config/config.local.json if running in the local environment, or config/config.cloud.json if running in the cloud environment
Exporting the PURPLETEAM_UI environment variable (PURPLETEAM_UI=noUi) for example

uI options:

cUi: Is well suited to running the PurpleTeam CLI directly in your terminal.
With the uI configured to use cUi the following putpleteam CLI commands have the associated behaviours:
- about: Writes to the console using the purpleteam-logger configured with the SignaleTransport
- status: Writes to the console using the purpleteam-logger configured with the SignaleTransport, via blessed
- test: Writes to file using purpleteam-logger configured with the File transport, writes to the console using blessed. On a successful test run, an outcomes zip file will be written to the directory specified by outcomes.dir
- testplan: Writes to the console using blessed
noUi: Is well suited to running the PurpleTeam CLI from another process (your build/CI/CD process for example). With the uI configured to use noUi the following putpleteam CLI commands have the associated behaviours:
- about: Writes to the console using the purpleteam-logger configured with the SignaleTransport. The about screen is written. Exits with code: "0"
- status: Writes the following messages to the console using the purpleteam-logger configured with the SignaleTransport. These messages and their meanings apply to both uI modes:
  - orchestrator is down, or an incorrect URL has been specified in the CLI config if the orchestrator is unreachable
  - orchestrator is ready to take orders.
  - Test Run is in progress.
  Exits with code: "0"
- test: Writes to file using purpleteam-logger configured with the File transport. These messages and their meanings apply to both uI modes
  - If the orchestrator/API is unreachable: orchestrator is down, or an incorrect URL has been specified in the CLI config is written using the SignaleTransport. Exits with code: "0"
  - If the orchestrator/API is reachable CLI logs will be written to the directory specified by loggers.testerProgress.dirname as the Test Run progresses and an outcomes zip file will be written to the directory specified by outcomes.dir on Test Run completion. The CLI does not terminate
  - Retry sequence documented in this blog post
  - If there is a Tester failure Tester failure:... will be written using the SignaleTransport and to the directory specified by loggers.testerProgress.dirname for the specific Tester that issued the Tester failure:... message, so you may want to keep watch on the logs for all Testers if you are searching for the Tester failure: string. The orchestrator will issue warning messages for the other Testers, but they may not contain the text: Tester failure:.
    This can happen for varius reasons such as:
    - The number of Test Sessions provided in the Job file falls outside of the valid range:
      Tester failure: The only valid number of tlsScanner resource objects is one. Please modify your Job file.
      Tester failure: The only valid number of appScanner resource objects is from 1-12 inclusive. Please modify your Job file.
    - Tester failure: S2 app containers were not ready. app Tester(s) failed initialisation. Test Run aborted - This occurs in the cloud environment if ECS doesn't bring the stage two containers up in time. The App Tester gives ECS 2 minutes to bring the stage two containers up, usually they come up from cold start with 40 seconds to spare, if they don't come up in {s2Containers.serviceDiscoveryServiceInstances.timeoutToBeAvailable (from the app-scanner config)} milliseconds then the App Tester decides it is unable to start a Test Run due to circumstances outside of it's control (ECS is not going to bring the stage two containers up) and the orchestrator aborts the Test Run with this message. The orchestrator then issues the order to bring all stage two containers down (clean-up).
      As the Build User you can rely on the text Tester failure: to mean you will need to initiate a retry. You can do this after some time, or continue to issue the CLI status command, after approximately {coolDown.timeout (from the orchestrator config)} milliseconds the response will change from Test Run is in progress. to orchestrator is ready to take orders., at which point you can initiate a retry (run the test command again)
- testplan: Writes to file using purpleteam-logger configured with the File transport
  - If the orchestrator/API is down orchestrator is down, or an incorrect URL has been specified in the CLI config is written using the SignaleTransport. Exits with code: "0"
  - If the orchestrator/API is up, CLI logs will be written to the directory specified by loggers.testPlan.dirname on completion. Exits with code: "0"

The PurpleTeam CLI uses the convict package for it's configuration.

Sensitive Values (`cloud` environment only)

There are several ways you can handle the sensitive values that need to be read into the PurpleTeam CLI to access your instance of the PurpleTeam cloud service:

Place sensitive values in-line in the config.cloud.json file, providing you are confident that you have sufficiently locked down file, directory permissions and access to the host that will be running the PurpleTeam CLI
Place sensitive values in a similarly structured file but in some other directory that the PurpleTeam CLI has access to and is sufficiently locked down as previously mentioned. The path of which said file can be added to the array as an element that is feed to config.loadFile in the main config.js file
Place sensitive values in environment variables yourself, or pass them as environment variables in the current shell to the PurpleTeam CLI:
```
PURPLETEAM_APP_CLIENT_ID=<app-client-id> PURPLETEAM_APP_CLIENT_SECRET=<app-client-secret> PURPLETEAM_API_KEY=<api-key> purpleteam test
```

The precedence order of where values will be read from is defined by convict.

Job File

The Job file is what purpleteam uses to do the following. Most properties should be self documenting, although the official documentation is here. If you are unsure of any of the properties, start a Github discussion or reach out in the #project-purpleteam channel of OWASP Slack. Examples of Job files that the PurpleTeam-Labs team use can be found here. Once you have defined the location of your SUT, you may want to consider defining some of the following:

Defining your Test Session(s)
Authentication to your System Under Test (SUT)
Which browser you may want to use to test your application in (not applicable to APIs)
Defining alertThresholds
Defining specific routes to test (for browser based applications), or an API definition
Defining fields of each specific route (for browser based applications), other fields "may" also be tested
Work through the Job file documentation, there are many additional knobs and levers you can apply and tweak

Remember to keep it simple to start with.

Run

There are several ways you can run the PurpleTeam CLI. The following options line up with the Install options detailed above. Make sure you have installed and configured purpleteam correctly before attempting to run:

Clone the git repository option

For those that chose to clone the purpleteam:

You can choose to export the NODE_ENV environment variable before running the following commands, or simply do so as part of running the commands. For example: NODE_ENV=local or NODE_ENV=cloud.

Run the bin/purpleteam file via npm script

From the root directory of the purpleteam repository

Run one of the following commands

To start the CLI:

npm start
# Should print out the PurpleTeam top level help

To start the CLI and pass commands (status for example):

npm start status
# Should print the following message if the orchestrator is not running:
# ☰  notice     [cUi] orchestrator is down, or an incorrect URL has been specified in the CLI config.

To start the CLI and pass commands (test for example):

npm start test
# Should print the following message if the orchestrator is not running:
# ✖  critical   [apiDecoratingAdapter] orchestrator is down, or an incorrect URL has been specified in the CLI config.

To start the CLI and pass commands (test for example) with options:

npm start test -- --help
# Should print the available options for the test command:

To debug the CLI (passing the status command for example):
```
npm run debug status
# Amongst other messages, you should see the following message:
# Debugger listening on ws://localhost:9230/...
```
Now open your debugging UI. If you use the chrome developer tools browse to chrome://inspect and click the inspect link and you will be dropped into the purpleteam CLI code-base.

For further details around running and debugging review the documentation.

Run the bin/purpleteam file directly

From the root directory of the purpleteam repository

Run one of the following commands

To start the CLI:

bin/purpleteam
# Should print out the PurpleTeam top level help

To start the CLI and pass commands (status for example):

bin/purpleteam status
# Should print the following message if the orchestrator is not running:
# ☰  notice     [cUi] orchestrator is down, or an incorrect URL has been specified in the CLI config.

To start the CLI and pass commands (test for example):

bin/purpleteam test
# Should print the following message if the orchestrator is not running:
# ✖  critical   [apiDecoratingAdapter] orchestrator is down, or an incorrect URL has been specified in the CLI config.

To start the CLI and pass commands (test for example) with options:

bin/purpleteam test --help
# Should print the available options for the test command:

Or if you chose to clone the PurpleTeam CLI (purpleteam) repository and npm link it, you can run it as a first class citizen:

purpleteam

NPM install locally option

For those that chose to install locally via npm:

The NODE_ENV environment variable needs to be exported so that the PurpleTeam CLI knows whether it's targeting the cloud or local environment and configuration. In the example build project we have used, NODE_ENV is exported as part of the NPM scripts, and it is using the local environment. Feel free to swap the value to cloud if you have signed up for a cloud account.

Providing your package.json and the JavaScript file (index.js in the above example) that is going to run the PurpleTeam CLI is similar to those configured in the above file examples, you should be able to successfully run the following commands from the root directory of your NodeJS CI/nightly build/build pipeline project.

Run the PurpleTeam CLI directly

npm run purpleteam
# Should print out the PurpleTeam top level help

Run the PurpleTeam CLI directly - with `status` command

Run the PurpleTeam CLI directly but pass the status command to purpleteam:

npm run purpleteam status
# Should print the following message if the orchestrator is not yet running. Be patient, PurpleTeam CLI retries:
# ☰  notice     [cUi] orchestrator is down, or an incorrect URL has been specified in the CLI config.

Run the PurpleTeam CLI directly - with `test` command

Run the PurpleTeam CLI directly but pass the test command to purpleteam:

npm run purpleteam test
# Should print the following message if the orchestrator is not yet running:
# ✖  critical   [apiDecoratingAdapter] orchestrator is down, or an incorrect URL has been specified in the CLI config.

Run the PurpleTeam CLI directly - with `test` options

Run the PurpleTeam CLI directly but you want to see the help options for the test command:

npm run purpleteam test -- --help
# Should print the available options for the test command:

Run your app (build pipeline)

Run your NodeJS CI/nightly build/build pipeline project. This will start the NodeJS application we defined above which will spawn the purpleteam test command.

You could change the const purpleteamArgs = ['purpleteam', 'test']; to use any other PurpleTeam CLI commands, options, or neither.

When running the PurpleTeam CLI from another process, you will usually want to export PURPLETEAM_UI=noUi as mentioned in the NPM install locally sub-section and detailed in the Configure uI sub-section.

npm start
# If the orchestrator is not yet running:
# Should print the following via the purpleteam.stdout.on('data'... handler
# ✖  critical   [apiDecoratingAdapter] orchestrator is down, or an incorrect URL has been specified in the CLI config.

If you get a blank screen or purpleteam help text with an error or warning via a ?⃝ warning logged to your terminal, please confirm you have configured purpleteam correctly.

When running the PurpleTeam CLI embedded, you should expect the behaviours specified under the Configure uI sub-section for the associated PurpleTeam CLI commands.

Debug your app (build pipeline)

If you need to debug your NodeJS CI/nightly build/build pipeline project, run the following command:

npm run debugApp

Now open your debugging UI. If you use the chrome developer tools browse to chrome://inspect and click the inspect link and you will be dropped into your app (index.js in this case).

Debug your app and PurpleTeam CLI

If you need to debug your NodeJS CI/nightly build/build pipeline project as well as the PurpleTeam CLI, do the following:

Make sure your debugging UI is configured to listen on the application and CLI debug ports:
- localhost:9230 as defined in the above package.json
- localhost:9231 as defined in the above index.js
run the following command:
```
npm run debugAppAndCli
```
Then open your debugging UI. If you use the chrome developer tools browse to chrome://inspect and click the "inspect" link and you will be dropped into your app (index.js in this case)
Step through the index.js file until the purpleteam process is spawned, at which point you will see the second "inspect" link if using the chrome developer tools
Click the second "inspect" link and you should be dropped into the purpleteam source code. Now you can step through and inspect both your application code and the PurpleTeam CLI code

NPM install globally option

For those that chose to install globally via npm:

You can choose to export the NODE_ENV environment variable before running the following commands, or simply do so as part of running the commands.

NODE_ENV=local purpleteam
# Or export NODE_ENV then just run:
purpleteam
# Should print out the PurpleTeam top level help

Run any of the PurpleTeam CLI commands as you would with the install of any other system wide binary.

If you choose to clone the PurpleTeam CLI repository and run npm link from it's root directory, the same applies. Plus you get to continue to modify the PurpleTeam CLI config without reinstalling.

Usage

If you are running the PurpleTeam CLI in the default character user interface (cUi) mode there are some interactions you can perform in the terminal while the CLI is running. The following commands have the associated interactions available:

test: Once testing is under way, you can:
- [right-arrow], [left-arrow] through the terminal screens to view the testing progress of each of the Testers in real-time courtesy of the PurpleTeam API
- [down-arrow], [up-arrow] to highlight the different Running Statistics of the Testers as they are provided in real-time courtesy of the PurpleTeam API
testplan: Once the test plans have been retrieved, you can [right-arrow], [left-arrow] through the terminal screens to view the test plans of each specific Tester

Trouble-Shooting

If you encounter any problems with the CLI set-up and you have read and applied the directions, check the trouble-shooting page.

Published

Vulnerabilities

Links

Maintainers

Keywords

Readme

Contents