artillery-plugin-tracetest

About

This module provides a way to enhance your existing Artillery Performance tests with trace-based testing. It allows you to create, run, and orchestrate Tracetest Tests based on your existing Artillery Test scripts with minimal configuration.

To find a detailed step-by-step tutorial on how to use this module head out to the main integration page in our docs.

Usage

Define the Plugin as part of your Artillery Test Script

config:
  target: "http://localhost:8081"
  phases:
    - duration: 1
      arrivalRate: 1
  plugins:
    publish-metrics:
      - type: "open-telemetry"
        serviceName: "artillery"
        metrics:
          reporter: otlp-http
          endpoint: "http://localhost:4318/v1/metrics"
          attributes:
            environment: "test"
            tool: "Artillery"
            type: "Load test"

        traces:
          reporter: otlp-http
          endpoint: "http://localhost:4318/v1/traces"
          attributes:
            environment: "test"
            tool: "Artillery"
    tracetest:
      token: <YOUR_TRACETEST_API_TOKEN>
      definition: import-pokemon-trace-id.yaml
      runInfo:
        variables:
          - key: POKEMON_ID
            value: "6"
scenarios:
  - name: "Import Pokemon"
    flow:
      - post:
          url: "/pokemon/import"
          json:
            id: 6

Output

> [email protected] test:plugin
> artillery run plugin-test.yaml

Test run id: t6pjz_zbaqcap7rhab59xn9kygn66bg8cfq_fqjc
Phase started: unnamed (index: 0, duration: 2s) 15:59:46(-0600)

Phase completed: unnamed (index: 0, duration: 2s) 15:59:48(-0600)

--------------------------------------
Metrics for period to: 15:59:50(-0600) (width: 2.509s)
--------------------------------------

http.codes.200: ................................................................ 10
http.downloaded_bytes: ......................................................... 80
http.request_rate: ............................................................. 10/sec
http.requests: ................................................................. 10
http.response_time:
  min: ......................................................................... 1
  max: ......................................................................... 2
  mean: ........................................................................ 1.6
  median: ...................................................................... 2
  p95: ......................................................................... 2
  p99: ......................................................................... 2
http.responses: ................................................................ 10
tracetest.tests_started: ....................................................... 10
vusers.created: ................................................................ 10
vusers.created_by_name.Import Pokemon: ......................................... 10


--------------------------------------
Metrics for period to: 16:00:10(-0600) (width: 5.71s)
--------------------------------------

tracetest.tests_failed: ........................................................ 2
tracetest.tests_succeeded: ..................................................... 1
vusers.completed: .............................................................. 3
vusers.failed: ................................................................. 0
vusers.session_length:
  min: ......................................................................... 16656.5
  max: ......................................................................... 22462.5
  mean: ........................................................................ 18737.5
  median: ...................................................................... 17158.9
  p95: ......................................................................... 17158.9
  p99: ......................................................................... 17158.9


All VUs finished. Total time: 30 seconds

--------------------------------
Summary report @ 16:00:17(-0600)
--------------------------------

http.codes.200: ................................................................ 10
http.downloaded_bytes: ......................................................... 80
http.request_rate: ............................................................. 5/sec
http.requests: ................................................................. 10
http.response_time:
  min: ......................................................................... 1
  max: ......................................................................... 2
  mean: ........................................................................ 1.6
  median: ...................................................................... 2
  p95: ......................................................................... 2
  p99: ......................................................................... 2
http.responses: ................................................................ 10
tracetest.tests_failed: ........................................................ 9
tracetest.tests_started: ....................................................... 10
tracetest.tests_succeeded: ..................................................... 1
vusers.completed: .............................................................. 10
vusers.created: ................................................................ 10
vusers.created_by_name.Import Pokemon: ......................................... 10
vusers.failed: ................................................................. 0
vusers.session_length:
  min: ......................................................................... 16656.5
  max: ......................................................................... 27889.9
  mean: ........................................................................ 23730.5
  median: ...................................................................... 25091.6
  p95: ......................................................................... 27181.5
  p99: ......................................................................... 27181.5
⠸ ✔ Artillery Plugin: Import a Pokemon (https://app.tracetest.io/organizations/ttorg_ced62e34638d965e/environments/ttenv_b42fa137465c6e04/test/artillery-plugin-import-pokemon/run/104) - trace id: 787ff20c554c1e21cc90a7c3eec8d2ee

✘ Artillery Plugin: Import a Pokemon (https://app.tracetest.io/organizations/ttorg_ced62e34638d965e/environments/ttenv_b42fa137465c6e04/test/artillery-plugin-import-pokemon/run/105) - trace id: 44a208bbe579c513c5f79ac8838aeb46
  > All HTTP Spans: Status  code is 200 (span[tracetest.span.type="general" name = "validate request"] span[tracetest.span.type="http"])

    • Expected: attr:http.status_code = 200

      No Spans
      Received: resolution error: there are no matching spans to retrieve the attribute "http.status_code" from. To fix this error, create a selector matching at least one span.
  > span[tracetest.span.type="http" name="GET" http.method="GET"]

    • Expected: attr:http.route = "/api/v2/pokemon/${var:POKEMON_ID}"

      No Spans
      Received: resolution error: there are no matching spans to retrieve the attribute "http.route" from. To fix this error, create a selector matching at least one span.
  > All Database Spans: Processing time is less than 1s (span[tracetest.span.type="database"])
  > Response time should be fast (span[tracetest.span.type="http" name="post" http.method="POST"])

✘ Artillery Plugin: Import a Pokemon (https://app.tracetest.io/organizations/ttorg_ced62e34638d965e/environments/ttenv_b42fa137465c6e04/test/artillery-plugin-import-pokemon/run/106) - trace id: a677b4c7b68f4d692087c860686699fc
  > All HTTP Spans: Status  code is 200 (span[tracetest.span.type="general" name = "validate request"] span[tracetest.span.type="http"])

    • Expected: attr:http.status_code = 200

      No Spans
      Received: resolution error: there are no matching spans to retrieve the attribute "http.status_code" from. To fix this error, create a selector matching at least one span.
  > span[tracetest.span.type="http" name="GET" http.method="GET"]

    • Expected: attr:http.route = "/api/v2/pokemon/${var:POKEMON_ID}"

      No Spans
      Received: resolution error: there are no matching spans to retrieve the attribute "http.route" from. To fix this error, create a selector matching at least one span.
  > All Database Spans: Processing time is less than 1s (span[tracetest.span.type="database"])
  > Response time should be fast (span[tracetest.span.type="http" name="post" http.method="POST"])

✘ Artillery Plugin: Import a Pokemon (https://app.tracetest.io/organizations/ttorg_ced62e34638d965e/environments/ttenv_b42fa137465c6e04/test/artillery-plugin-import-pokemon/run/107) - trace id: e6e3963c0975367d89308acd4e3c379c
  > All HTTP Spans: Status  code is 200 (span[tracetest.span.type="general" name = "validate request"] span[tracetest.span.type="http"])

    • Expected: attr:http.status_code = 200

      No Spans
      Received: resolution error: there are no matching spans to retrieve the attribute "http.status_code" from. To fix this error, create a selector matching at least one span.
  > span[tracetest.span.type="http" name="GET" http.method="GET"]

    • Expected: attr:http.route = "/api/v2/pokemon/${var:POKEMON_ID}"

      No Spans
      Received: resolution error: there are no matching spans to retrieve the attribute "http.route" from. To fix this error, create a selector matching at least one span.
  > All Database Spans: Processing time is less than 1s (span[tracetest.span.type="database"])
  > Response time should be fast (span[tracetest.span.type="http" name="post" http.method="POST"])

✘ Artillery Plugin: Import a Pokemon (https://app.tracetest.io/organizations/ttorg_ced62e34638d965e/environments/ttenv_b42fa137465c6e04/test/artillery-plugin-import-pokemon/run/108) - trace id: 22684fc7a5d43ed9e44e24c7135a6aa5
  > All HTTP Spans: Status  code is 200 (span[tracetest.span.type="general" name = "validate request"] span[tracetest.span.type="http"])

    • Expected: attr:http.status_code = 200

      No Spans
      Received: resolution error: there are no matching spans to retrieve the attribute "http.status_code" from. To fix this error, create a selector matching at least one span.
  > span[tracetest.span.type="http" name="GET" http.method="GET"]

    • Expected: attr:http.route = "/api/v2/pokemon/${var:POKEMON_ID}"

      No Spans
      Received: resolution error: there are no matching spans to retrieve the attribute "http.route" from. To fix this error, create a selector matching at least one span.
  > All Database Spans: Processing time is less than 1s (span[tracetest.span.type="database"])
  > Response time should be fast (span[tracetest.span.type="http" name="post" http.method="POST"])

✘ Artillery Plugin: Import a Pokemon (https://app.tracetest.io/organizations/ttorg_ced62e34638d965e/environments/ttenv_b42fa137465c6e04/test/artillery-plugin-import-pokemon/run/109) - trace id: c73da8f9a009dc365f19f8701924c9cc
  > All HTTP Spans: Status  code is 200 (span[tracetest.span.type="general" name = "validate request"] span[tracetest.span.type="http"])

    • Expected: attr:http.status_code = 200

      No Spans
      Received: resolution error: there are no matching spans to retrieve the attribute "http.status_code" from. To fix this error, create a selector matching at least one span.
  > span[tracetest.span.type="http" name="GET" http.method="GET"]

    • Expected: attr:http.route = "/api/v2/pokemon/${var:POKEMON_ID}"

      No Spans
      Received: resolution error: there are no matching spans to retrieve the attribute "http.route" from. To fix this error, create a selector matching at least one span.
  > All Database Spans: Processing time is less than 1s (span[tracetest.span.type="database"])
  > Response time should be fast (span[tracetest.span.type="http" name="post" http.method="POST"])

✘ Artillery Plugin: Import a Pokemon (https://app.tracetest.io/organizations/ttorg_ced62e34638d965e/environments/ttenv_b42fa137465c6e04/test/artillery-plugin-import-pokemon/run/110) - trace id: 98c74bd6e6501f6721f058be4f5bf311
  > All HTTP Spans: Status  code is 200 (span[tracetest.span.type="general" name = "validate request"] span[tracetest.span.type="http"])

    • Expected: attr:http.status_code = 200

      No Spans
      Received: resolution error: there are no matching spans to retrieve the attribute "http.status_code" from. To fix this error, create a selector matching at least one span.
  > span[tracetest.span.type="http" name="GET" http.method="GET"]

    • Expected: attr:http.route = "/api/v2/pokemon/${var:POKEMON_ID}"

      No Spans
      Received: resolution error: there are no matching spans to retrieve the attribute "http.route" from. To fix this error, create a selector matching at least one span.
  > All Database Spans: Processing time is less than 1s (span[tracetest.span.type="database"])
  > Response time should be fast (span[tracetest.span.type="http" name="post" http.method="POST"])

✘ Artillery Plugin: Import a Pokemon (https://app.tracetest.io/organizations/ttorg_ced62e34638d965e/environments/ttenv_b42fa137465c6e04/test/artillery-plugin-import-pokemon/run/111) - trace id: 5c722a53b527f7334800dd42d98f5a49
  > All HTTP Spans: Status  code is 200 (span[tracetest.span.type="general" name = "validate request"] span[tracetest.span.type="http"])

    • Expected: attr:http.status_code = 200

      No Spans
      Received: resolution error: there are no matching spans to retrieve the attribute "http.status_code" from. To fix this error, create a selector matching at least one span.
  > span[tracetest.span.type="http" name="GET" http.method="GET"]

    • Expected: attr:http.route = "/api/v2/pokemon/${var:POKEMON_ID}"

      No Spans
      Received: resolution error: there are no matching spans to retrieve the attribute "http.route" from. To fix this error, create a selector matching at least one span.
  > All Database Spans: Processing time is less than 1s (span[tracetest.span.type="database"])
  > Response time should be fast (span[tracetest.span.type="http" name="post" http.method="POST"])

✘ Artillery Plugin: Import a Pokemon (https://app.tracetest.io/organizations/ttorg_ced62e34638d965e/environments/ttenv_b42fa137465c6e04/test/artillery-plugin-import-pokemon/run/112) - trace id: 53ea977eb682fc5dc416f0425dc0ac5e
  > All HTTP Spans: Status  code is 200 (span[tracetest.span.type="general" name = "validate request"] span[tracetest.span.type="http"])

    • Expected: attr:http.status_code = 200

      No Spans
      Received: resolution error: there are no matching spans to retrieve the attribute "http.status_code" from. To fix this error, create a selector matching at least one span.
  > span[tracetest.span.type="http" name="GET" http.method="GET"]

    • Expected: attr:http.route = "/api/v2/pokemon/${var:POKEMON_ID}"

      No Spans
      Received: resolution error: there are no matching spans to retrieve the attribute "http.route" from. To fix this error, create a selector matching at least one span.
  > All Database Spans: Processing time is less than 1s (span[tracetest.span.type="database"])
  > Response time should be fast (span[tracetest.span.type="http" name="post" http.method="POST"])

✘ Artillery Plugin: Import a Pokemon (https://app.tracetest.io/organizations/ttorg_ced62e34638d965e/environments/ttenv_b42fa137465c6e04/test/artillery-plugin-import-pokemon/run/113) - trace id: d1de80e71c85119b62c8495f3ec42e81
  > All HTTP Spans: Status  code is 200 (span[tracetest.span.type="general" name = "validate request"] span[tracetest.span.type="http"])

    • Expected: attr:http.status_code = 200

      No Spans
      Received: resolution error: there are no matching spans to retrieve the attribute "http.status_code" from. To fix this error, create a selector matching at least one span.
  > span[tracetest.span.type="http" name="GET" http.method="GET"]

    • Expected: attr:http.route = "/api/v2/pokemon/${var:POKEMON_ID}"

      No Spans
      Received: resolution error: there are no matching spans to retrieve the attribute "http.route" from. To fix this error, create a selector matching at least one span.
  > All Database Spans: Processing time is less than 1s (span[tracetest.span.type="database"])
  > Response time should be fast (span[tracetest.span.type="http" name="post" http.method="POST"])

Useful Links

• For a detailed step-by-step tutorial on how to use this module: visit https://docs.tracetest.io/tools-and-integrations/artillery-plugin
• If you want to get in touch with the Tracetest team: visit https://dub.sh/tracetest-community