Welcome to Stress Test Toolbox.

Our in-house load testing tool on AWS Fargate enables developers to effortlessly conduct scalable and comprehensive performance and load tests, ensuring our applications withstand real-world pressures.

Combining this tool with our retool front end enables an even better user experience as well as great dashboards to view the results of your tests.

How to set up the tool

The Backend

To use this construct, first install the npm package.

npm i @stress-test-toolbox/stress-test-construct

Import the construct from the package

import { StressTestConstruct } from "@stress-test-toolbox/stress-test-construct";

Then in your cdk stack create an instance of the construct

new StressTestConstruct(this, "stress-test-toolbox");

Example stack:

import { Stack } from "aws-cdk-lib";
import { StressTestConstruct } from "@stress-test-toolbox/stress-test-construct";
import { Construct } from "constructs";

export class TestingStack extends Stack {
  constructor(scope: Construct, id: string) {
    super(scope, id);

    new StressTestConstruct(this, "stress-test-toolbox");
  }
}

The Frontend

We have created a retool front end that you can use to run this tool.

How to deploy your own version coming soon!

CI/CD

Adding the tool via CI/CD, to hit the load testing endpoint from a github workflow file to trigger your tests:

1. Firstly you need to make sure that you have the load testing stack deployed into your project’s AWS account.
2. You need to get an api key that can be used in the CI/CD to invoke the load testing endpoint. This can be done by hitting the createApiKey endpoint.
3. Next you need to add a config file for the artillery test you want to run, this config file needs to be in json and not yaml. This config file can be stored anywhere in your repo, but would recommend storing in your ‘.github’ folder for easy access.
4. Now we can implement a step in a github workflow file as below.

- name: Run load-test
  run: |
    curl -X POST \
    -H "x-api-key: xxx" \
    -H "Content-Type: application/json" \
    -d "@./.github/config.json" \ 
    https://<your-api-gateway-url>/loadTestRequest

Things to note:

• The x-api-key will be the api key’s value you get from hitting the createApiKey endpoint
• We are using the -d flag to reference our config file. The path specified to your config.json which contains your artillery load test config needs to have an @ at the beginning.
• The api url is the load testing stack url which should include the ‘/loadTestRequest’ path

How to use it

Now that you have the tool all set up, you just need to start using it. The whole point of this tool is to make it easy for users to use so this will be relatively simple.

Via Backend Only

1. First create a config file for your test. We are using artillery to run these test, so have a look at their docs on how to write the config. NOTE: You need to write the config in json not in yml as artillery uses.
2. With your config file, you can now hit the loadTestRequest endpoint to start your test. Have a look at the loadTestRequest endpoint details for the structure of the request body.
3. Once you have successfully hit the endpoint your load test will start running.
4. It is recommended to add your email into the body so that you are notified once you test has completed running.
5. Now that your test is finished the results are stored in S3. You can download a json report as well as a html report that will display the data in graphs etc.

Via Frontend

1. Open your retool app and go to the run a test page. There is a code block where you can write your config file. We are using artillery to run our tests so have a look at their docs on how to write the config. NOTE: You need to write the config in json not in yml as artillery uses.
2. On the right there are options that you can fill in, such as testName and email (to receive a notification once your test is complete.) These are optional, but very useful.
3. Once your test has completed running, go to the Results page. Here you will find a table with all your tests in. Click on a test to view the results via metrics and graphs.

Architecture

Flow:

1. Client sends LoadTestRequest to API Gateway with some config.
   1. API Gateway is protected with IAM Auth. Create and Delete key commands also exist to manage keys easily.
   2. This is a async request so the user gets an immediate response which contains the executionId. This is use to uniquely identify the tests. This is done using response mappings.
2. LoadTestRequest is forwarded to StepFunction which invokes a StateMachine.
3. The StateMachine creates a Database entry into DDB using the execId as the Primary Key, and status sets the config user as well as sets the test status to TEST_INITIATED.
4. Then the StepFunction runs the RunTask step which creates a Fargate Task from a Task Definition.
   1. This will update the status of the test in DDB to TEST_RUNNING
   2. This will run the actual load test with the configuration provided by the user.
5. Once the test has run in the Fargate Task the task will also store the result reports in S3 before terminating.
6. Once the test is completed the status will once again be updated
   1. If it was successful the status will be updated to TEST_COMPLETED.
   2. If it was unsuccessful the status will be updated to TEST_FAILED.
7. If a user provided an email, a email notification will be sent to the user to notify them that the test has finished running
8. The results of the tests are pulled by Retool directly from S3 and DDB.

Services

Api Gateway

Postman Collection: Find the asset for the postman collection in the assets folder, under load-testing-postman-collection.json

Endpoints:

• LoadTestRequest(Has request validation)
• CreateAPIKey (For IAM Auth)
• DeleteAPIKey (For IAM Auth)

/loadTestRequest

Headers:

x-api-key: <IAM Auth API Key>

Body:

config: JSON formatted artillery test script (required)
testName: String (optional)
email: String (optional)

Response Example:

{
  "executionArn": "arn:aws:states:eu-west-2:730335287746:execution:liza-stress-test-RunLoadTestTaskStateMachine:25d70ecb-fad8-47b6-9971-27bf796e75a8",
  "startDate": 1.710772458892e9
}

/createApiKey

Authorization: You need to create an IAM user for something like postman and then use the AccessKey and SecretKey to authorise the request.

Body:

keyName: String(required);

Response Example:

{
  "apiKey": {
    "id": "qg5mwymgg1",
    "value": "NxHvgAqVkZafzmfgf2eSY7Upx8XqfBOU7z95ltVu",
    "name": "liza-test"
  },
  "usagePlan": {
    "id": "zoufj0",
    "name": "liza-test-usage-plan"
  }
}

deleteApiKey

Authorization: You need to create an IAM user for something like postman and then use the AccessKey and SecretKey to authorise the request.

Body:

apiKeyId: String(required);

Response Example:

200: ok

Step Function:

The step function has the following steps:

Flow

1. Adds an execution entry into DDB
   • Status = TEST_INITIATED
   • Config = The config provided by the user in JSON format
   • Test Name = Name provided by user, if it has been provided
2. Runs the fargate task with the provided config.
   • Updates the status to TEST_RUNNING.
   • Starts an artillery load test with the config.
   • Once the test is complete the results are stored in S3.
   • If anything in this task fails, it will be caught and the status will be updated to TEST_FAILED.
3. After the fargate task has completed and terminated, we update the status again
   • Status = TEST_COMPLETED
4. Send an email to notify the user if the test is complete
   • If the user has provided an email, an email will be sent
   • Otherwise, this step is skipped