artillery-engine-tracetest

About

This module provides a way to enhance your existing Artillery Performance tests with trace-based testing. It allows you to create, run, and orchestrate Tracetest Tests based on your existing Artillery Test scripts with minimal configuration.

To find a detailed step-by-step tutorial on how to use this module head out to the main integration page in our docs.

Usage

Define the Engine as part of your Artillery Test Script

config:
  target: my_target
  tracetest:
    token: <YOUR_TRACETEST_API_TOKEN>
  phases:
    - duration: 1
      arrivalRate: 1
  engines:
    tracetest: {}
scenarios:
  - name: tracetest_engine_test
    engine: tracetest
    flow:
      - test:
          definition: import-pokemon.yaml
          runInfo:
            variables:
              - key: ENDPOINT
                value: http://api:8081
              - key: POKEMON_ID
                value: "6"
      - summary:
          format: "pretty"

Output

> [email protected] test:engine
> artillery run engine-test.yaml

Test run id: t47ez_f4gmj899tr4nnp465h38d4mnker7z_w8yp
Phase started: unnamed (index: 0, duration: 2s) 09:29:33(-0600)

Phase completed: unnamed (index: 0, duration: 2s) 09:29:35(-0600)

--------------------------------------
Metrics for period to: 09:29:40(-0600) (width: 1.203s)
--------------------------------------

vusers.created: ................................................................ 10
vusers.created_by_name.tracetest_engine_test: .................................. 10


Warning: multiple batches of metrics for period 1709825370000 2024-03-07T15:29:30.000Z
⠏ ✘ Artillery Engine: Import Pokemon (https://app.tracetest.io/organizations/ttorg_ced62e34638d965e/environments/ttenv_b42fa137465c6e04/test/artillery-engine-import-pokemon/run/70) - trace id: 4a2e41bf5b497a643056c8a08d122d82
  > All HTTP Spans: Status  code is 200 (span[tracetest.span.type="general" name = "validate request"] span[tracetest.span.type="http"])

    • Expected: attr:http.status_code = 200

      No Spans
      Received: resolution error: there are no matching spans to retrieve the attribute "http.status_code" from. To fix this error, create a selector matching at least one span.
  > span[tracetest.span.type="http" name="GET" http.method="GET"]

    • Expected: attr:http.route = "/api/v2/pokemon/${var:POKEMON_ID}"

      No Spans
      Received: resolution error: there are no matching spans to retrieve the attribute "http.route" from. To fix this error, create a selector matching at least one span.
  > All Database Spans: Processing time is less than 1s (span[tracetest.span.type="database"])
⠙ ✘ Artillery Engine: Import Pokemon (https://app.tracetest.io/organizations/ttorg_ced62e34638d965e/environments/ttenv_b42fa137465c6e04/test/artillery-engine-import-pokemon/run/74) - trace id: a00ff614a06be196ec5cf74ace906b3e
  > All HTTP Spans: Status  code is 200 (span[tracetest.span.type="general" name = "validate request"] span[tracetest.span.type="http"])

    • Expected: attr:http.status_code = 200

      No Spans
      Received: resolution error: there are no matching spans to retrieve the attribute "http.status_code" from. To fix this error, create a selector matching at least one span.
  > span[tracetest.span.type="http" name="GET" http.method="GET"]

    • Expected: attr:http.route = "/api/v2/pokemon/${var:POKEMON_ID}"

      No Spans
      Received: resolution error: there are no matching spans to retrieve the attribute "http.route" from. To fix this error, create a selector matching at least one span.
  > All Database Spans: Processing time is less than 1s (span[tracetest.span.type="database"])
⠸ ✘ Artillery Engine: Import Pokemon (https://app.tracetest.io/organizations/ttorg_ced62e34638d965e/environments/ttenv_b42fa137465c6e04/test/artillery-engine-import-pokemon/run/76) - trace id: 1fb3ded60f582ecfd151a019df2479cd
  > All HTTP Spans: Status  code is 200 (span[tracetest.span.type="general" name = "validate request"] span[tracetest.span.type="http"])

    • Expected: attr:http.status_code = 200

      No Spans
      Received: resolution error: there are no matching spans to retrieve the attribute "http.status_code" from. To fix this error, create a selector matching at least one span.
  > span[tracetest.span.type="http" name="GET" http.method="GET"]

    • Expected: attr:http.route = "/api/v2/pokemon/${var:POKEMON_ID}"

      No Spans
      Received: resolution error: there are no matching spans to retrieve the attribute "http.route" from. To fix this error, create a selector matching at least one span.
  > All Database Spans: Processing time is less than 1s (span[tracetest.span.type="database"])
⠹ ✘ Artillery Engine: Import Pokemon (https://app.tracetest.io/organizations/ttorg_ced62e34638d965e/environments/ttenv_b42fa137465c6e04/test/artillery-engine-import-pokemon/run/78) - trace id: 750df8c1d55ed6a4e18f4b2b8b573a14
  > All HTTP Spans: Status  code is 200 (span[tracetest.span.type="general" name = "validate request"] span[tracetest.span.type="http"])

    • Expected: attr:http.status_code = 200

      No Spans
      Received: resolution error: there are no matching spans to retrieve the attribute "http.status_code" from. To fix this error, create a selector matching at least one span.
  > span[tracetest.span.type="http" name="GET" http.method="GET"]

    • Expected: attr:http.route = "/api/v2/pokemon/${var:POKEMON_ID}"

      No Spans
      Received: resolution error: there are no matching spans to retrieve the attribute "http.route" from. To fix this error, create a selector matching at least one span.
  > All Database Spans: Processing time is less than 1s (span[tracetest.span.type="database"])
⠦ ✔ Artillery Engine: Import Pokemon (https://app.tracetest.io/organizations/ttorg_ced62e34638d965e/environments/ttenv_b42fa137465c6e04/test/artillery-engine-import-pokemon/run/71) - trace id: c3bdcfb86ca77a9455b29882f593848a
⠏ ✘ Artillery Engine: Import Pokemon (https://app.tracetest.io/organizations/ttorg_ced62e34638d965e/environments/ttenv_b42fa137465c6e04/test/artillery-engine-import-pokemon/run/79) - trace id: 76ecf991c6eb37c2a0dee857a699dc76
  > All HTTP Spans: Status  code is 200 (span[tracetest.span.type="general" name = "validate request"] span[tracetest.span.type="http"])

    • Expected: attr:http.status_code = 200

      No Spans
      Received: resolution error: there are no matching spans to retrieve the attribute "http.status_code" from. To fix this error, create a selector matching at least one span.
  > span[tracetest.span.type="http" name="GET" http.method="GET"]

    • Expected: attr:http.route = "/api/v2/pokemon/${var:POKEMON_ID}"

      No Spans
      Received: resolution error: there are no matching spans to retrieve the attribute "http.route" from. To fix this error, create a selector matching at least one span.
  > All Database Spans: Processing time is less than 1s (span[tracetest.span.type="database"])
⠙ ✘ Artillery Engine: Import Pokemon (https://app.tracetest.io/organizations/ttorg_ced62e34638d965e/environments/ttenv_b42fa137465c6e04/test/artillery-engine-import-pokemon/run/72) - trace id: cece9c79434b3313bf23150bc967e82f
  > All HTTP Spans: Status  code is 200 (span[tracetest.span.type="general" name = "validate request"] span[tracetest.span.type="http"])

    • Expected: attr:http.status_code = 200

      No Spans
      Received: resolution error: there are no matching spans to retrieve the attribute "http.status_code" from. To fix this error, create a selector matching at least one span.
  > span[tracetest.span.type="http" name="GET" http.method="GET"]

    • Expected: attr:http.route = "/api/v2/pokemon/${var:POKEMON_ID}"

      No Spans
      Received: resolution error: there are no matching spans to retrieve the attribute "http.route" from. To fix this error, create a selector matching at least one span.
  > All Database Spans: Processing time is less than 1s (span[tracetest.span.type="database"])
⠇ ✘ Artillery Engine: Import Pokemon (https://app.tracetest.io/organizations/ttorg_ced62e34638d965e/environments/ttenv_b42fa137465c6e04/test/artillery-engine-import-pokemon/run/77) - trace id: b3f3682eb328195ede902177ff4170a1
  > All HTTP Spans: Status  code is 200 (span[tracetest.span.type="general" name = "validate request"] span[tracetest.span.type="http"])

    • Expected: attr:http.status_code = 200

      No Spans
      Received: resolution error: there are no matching spans to retrieve the attribute "http.status_code" from. To fix this error, create a selector matching at least one span.
  > span[tracetest.span.type="http" name="GET" http.method="GET"]

    • Expected: attr:http.route = "/api/v2/pokemon/${var:POKEMON_ID}"

      No Spans
      Received: resolution error: there are no matching spans to retrieve the attribute "http.route" from. To fix this error, create a selector matching at least one span.
  > All Database Spans: Processing time is less than 1s (span[tracetest.span.type="database"])
⠸ ✘ Artillery Engine: Import Pokemon (https://app.tracetest.io/organizations/ttorg_ced62e34638d965e/environments/ttenv_b42fa137465c6e04/test/artillery-engine-import-pokemon/run/75) - trace id: c2d4624f20060565b0b5e9f97ad7e7a7
  > All HTTP Spans: Status  code is 200 (span[tracetest.span.type="general" name = "validate request"] span[tracetest.span.type="http"])

    • Expected: attr:http.status_code = 200

      No Spans
      Received: resolution error: there are no matching spans to retrieve the attribute "http.status_code" from. To fix this error, create a selector matching at least one span.
  > span[tracetest.span.type="http" name="GET" http.method="GET"]

    • Expected: attr:http.route = "/api/v2/pokemon/${var:POKEMON_ID}"

      No Spans
      Received: resolution error: there are no matching spans to retrieve the attribute "http.route" from. To fix this error, create a selector matching at least one span.
  > All Database Spans: Processing time is less than 1s (span[tracetest.span.type="database"])
⠙ ✘ Artillery Engine: Import Pokemon (https://app.tracetest.io/organizations/ttorg_ced62e34638d965e/environments/ttenv_b42fa137465c6e04/test/artillery-engine-import-pokemon/run/73) - trace id: 878bb73b5234f764f4d289bf657b40cf
  > All HTTP Spans: Status  code is 200 (span[tracetest.span.type="general" name = "validate request"] span[tracetest.span.type="http"])

    • Expected: attr:http.status_code = 200

      No Spans
      Received: resolution error: there are no matching spans to retrieve the attribute "http.status_code" from. To fix this error, create a selector matching at least one span.
  > span[tracetest.span.type="http" name="GET" http.method="GET"]

    • Expected: attr:http.route = "/api/v2/pokemon/${var:POKEMON_ID}"

      No Spans
      Received: resolution error: there are no matching spans to retrieve the attribute "http.route" from. To fix this error, create a selector matching at least one span.
  > All Database Spans: Processing time is less than 1s (span[tracetest.span.type="database"])
--------------------------------------
Metrics for period to: 09:30:00(-0600) (width: 0.015s)
--------------------------------------

tracetest.tests_failed: ........................................................ 1
vusers.completed: .............................................................. 1
vusers.failed: ................................................................. 0
vusers.session_length:
  min: ......................................................................... 21797.1
  max: ......................................................................... 21797.1
  mean: ........................................................................ 21797.1
  median: ...................................................................... 21813.5
  p95: ......................................................................... 21813.5
  p99: ......................................................................... 21813.5


All VUs finished. Total time: 34 seconds

--------------------------------
Summary report @ 09:30:09(-0600)
--------------------------------

tracetest.tests_failed: ........................................................ 9
tracetest.tests_succeeded: ..................................................... 1
vusers.completed: .............................................................. 10
vusers.created: ................................................................ 10
vusers.created_by_name.tracetest_engine_test: .................................. 10
vusers.failed: ................................................................. 0
vusers.session_length:
  min: ......................................................................... 21797.1
  max: ......................................................................... 33447
  mean: ........................................................................ 28651.1
  median: ...................................................................... 28862.3
  p95: ......................................................................... 32542.3
  p99: ......................................................................... 32542.3

Useful Links

• For a detailed step-by-step tutorial on how to use this module: visit https://docs.tracetest.io/tools-and-integrations/artillery-engine
• If you want to get in touch with the Tracetest team: visit https://dub.sh/tracetest-community