@processity/cli-performance-testing

Overview

The @processity/cli-performance-testing plugin is a Salesforce CLI extension that enables users to run Apex performance test suites, monitor their progress, and generate detailed reports.

Prerequisites

Before using this plugin, ensure the following:

1. You have the Salesforce CLI installed.
2. The latest version of the apex-performance-testing unlocked package is installed in your target Salesforce org.
3. The target Salesforce org is authenticated using the Salesforce CLI and is set as the default org for your project repository.
4. An Apex Performance suite is created and deployed in your org.

Installation

To install the plugin, use the following command:

sf plugins install @processity/cli-performance-testing

Note: if you have installed the namespaced version of apex-performance-testing, set the namespace environment variable to mantra

export namespace=mantra

Usage

Running a Performance Suite

Execute the following command to run a performance test suite:

sf performance test run TestSuiteClassName

This command will

1. start the performance suite execution
2. Display a live progress bar
3. Generate a PDF report with graphs once the execution is complete.

Comparing against a benchmark

you can compare the generated report against a benchmark by creating a ptest.config.json file in root directory

{
  "benchmarks": [
    {
      "suiteName": "{suite label}",
      "svgResponses": {
        "cpuTime": {
          "plots": {
            "{scenario 1 label}": {
              "xMin": 3,
              "xMax": 30,
              "yMin": 283,
              "yMax": 194
            },
            "{scenario 2 label}": {
              "xMin": 3,
              "xMax": 30,
              "yMin": 301,
              "yMax": 222
            }
          }
        }
        // ... Other metrics
      }
    }
    // ...Other suite's benchmark
  ]
}

Quick Tip: The actual report for each suite can be found in .ptest/{suitename}/response.json

If a benchmark JSON is available for the suite, it will be displayed as a dotted line on the graph to facilitate easy comparison. For example, notice how the Iterator loop takes more time than the set benchmark.

Reporting Issues

If you encounter any issues, please report them on the GitHub Issues page.

Building the Plugin Locally

To build the plugin locally, ensure you have Yarn installed and then run the following commands:

# Clone the repository
git clone [email protected]:processity/cli-performance-testing.git

# Install dependencies and compile
yarn && yarn build

You can use your plugin locally by running it with the ./bin/dev or ./bin/dev.cmd file:

# Run using the local development file
./bin/dev performance test run TestSuiteName

There should be no difference when running the plugin via the Salesforce CLI or using the local run file. However, linking the plugin can be helpful for additional testing or to run commands from any location on your machine:

# Link your plugin to the Salesforce CLI
sf plugins link .

# Verify the link
sf plugins

Commands

sf performance test run

Use this command to initiate the execution of performance suite(s).

USAGE
  $ sf performance test run -n [--json]

FLAGS
  -n, --name=<value>  [required] Comma-separated list of test suite names.
  -m, --metrics [optional] Comma-separated list of metrics for which report needs to be generated

GLOBAL FLAGS
  --json  Output the results in JSON format.

DESCRIPTION
  Initiates the execution of one or more Apex performance test suites, monitors progress, and generates a performance report upon completion.

EXAMPLES
  Run TestSuite1, monitor its progress, and generate a performance report:

    $ sf performance test run -n TestSuite1

  Run TestSuite1 and TestSuite2 sequentially, monitor progress, and generate a performance report:

    $ sf performance test run -n TestSuite1,TestSuite2

  Generate report for only CPU Time
    $ sf performance test report -n TestSuite1,TestSuite2 -m CPUTime

  Generate report for CPU Time and Heap Size
    $ sf performance test report -n TestSuite1,TestSuite2 -m CPUTime,HeapSize

sf performance test report

Use this command to generate report for performance suite(s).

USAGE
  $ sf performance test report -n [--json]

FLAGS
  -n, --name=<value>  [required] Comma-separated list of test suite labels.
  -m, --metrics [optional] Comma-separated list of metrics for which report needs to be generated

GLOBAL FLAGS
  --json  Output the results in JSON format.

DESCRIPTION
  This will generate report in PDF with 1 graph for each suite execution

EXAMPLES
  Generate report for TestSuite1 execution:

    $ sf performance test report -n TestSuite1

  Generate report for TestSuite1 and TestSuite2executions:

    $ sf performance test report -n TestSuite1,TestSuite2

  Generate report for only CPU Time
    $ sf performance test report -n TestSuite1,TestSuite2 -m CPUTime

  Generate report for CPU Time and Heap Size
    $ sf performance test report -n TestSuite1,TestSuite2 -m CPUTime,HeapSize