@apica-io/url-xi
v4.2.6
Published
URL Check for integrations and API monitoring
Downloads
116
Readme
- URL-XI - Extended URL Rest Tester
- The Concept
- The test case definition
- Steps and requests
- Main properties
- Extractors
- Assertions
- Transformers
- Variables
- Random data generation
- Posting data with x-www-form-urlencoded content type
- JavaScript support
- Iterators
- Request data as a string
- Status code validation for request response.
- Override default management of request errors
- Supported syntax for requests
- The test case schema
- Running an URL-XI test in the command line
- Running url-xi as a http server
- Samples
- Projects
URL-XI - Extended URL Rest Tester
Url-xi can be used to test & monitor REST based API-s and ordinary HTML based http(s) requests. Supports all http requests GET,PUT,DELETE,POST, HEAD and OPTIONS.
Url-xi is a simplified version POSTMAN and using a similar concept, but has better support for a flow of request and correlation of request. It can additionally return other metrics than response times of the http requests as the main return value of the test case.
Url-xi has model based testing framework inspired by Mocha and Jest. A test case is built up in several steps. A step contains one or several API requests. Reporting is done on each step and the all all underlying requests. The tool is built for API monitoring from the bottom up. Detailed timings are reported per request. You can customize what should be reported as the result of a request.
The tool can be used stand-alone or as a check in Apica Synthetic Monitoring platform ASM. See Apica company homepage
System requirements
- Node js : Must be installed
Installation
Install from npm with 'npm install @apica-io/url-xi -g'
The Concept
A test case configuration is defined in a JSON file. The configuration can contain several http requests and you can chain them to a flow of request with correlation between the requests. Status and response time is reported for each request. Samples are found in the installation directory.
The test case definition
Steps and requests
The test case is divided into logical steps each step contains requests.Steps and requests are json arrays.
"steps": [
{
"name": "Get Events",
"idleBetweenRequests": "{{requestIdleTime}}",
"requests": [
{
"config": {
"method": "get",
"url": "/ticket-monster/rest/events/",
"params": {
"_": "{{$timestamp}}"
}
},Main properties
"$schema": "https://files-apicasystem-com.s3-eu-west-1.amazonaws.com/schemas/url-xi-schema-v1-0.json",
"name": "Ticket Monster Get Event",
"description": "Test case for ticket monster",
"flowControl": "Chained Flow",
"message":"Assert Test={{assertTest}}",
"baseURL": "http://ticketmonster.apicasystem.com",
"config": {},
"includes": [
{
"name": "testdata",
"scope": "project",
"type": "data",
"src": "my_test_data.json"
},
{
"name": "defaultVariables",
"scope": "project",
"type": "vars",
"src": "default_test_vars.json"
}
],| Property | Description |
| ------------- | ------------- |
| $schema | The json schema for the test definition |
| name | The test case name |
| description | A longer description of the test case |
| flowControl | Chained Flow or Individual tests. Chain Flow is default
||Chained flow stops on first request with errors|
|| Individual tests run all steps and reports number of successful steps as return value. |
| baseURL | The base url. Can be changed on the command line |
| config | Global config for all requests. Axios syntax |
| includes | Specification of included testa data and variables. |
Extractors
Extractors are used for correlation of request and for validation of response. The following type of extractors are supported:
- JSONPath
- XPath
- Regexp
- JavaScript
"extractors": [
{
"type": "xpath",
"expression": "//*[local-name() = 'GameId']/text()",
"variable": "gameId"
},
{
"type": "regexp",
"expression": "b:GameId>(\\d+)<",
"variable": "gameId2"
},
{
"type": "header",
"expression": "content-type",
"variable": "homePageContent"
},
{
"type": "regexp",
"expression": "<script\\s+src=\"(.+)\">\\s+<\/script>",
"variable": "javaScripts",
"array": true
},
]Special boolean flags that can be used in extractors
- array : Save the array of extracted values. Default is to save a random value in the array.
- index : Save a random index of extracted array instead of the random value.
- counter : Save the size of the extracted array.
Assertions
Are used for validation of response of requests. Assertions works togethers with extractors and variables. Result of assertions are stored in the test result.
"assertions": [
{
"type": "javaScript",
"value": "{{origin}}",
"description": "Returned origin should contain an IP address. Method={{method}}",
"expression": "/^((25[0-5]|(2[0-4]|1[0-9]|[1-9]|)[0-9])(\\.(?!$)|$)){4}$/.test(value)",
"failStep": true,
"reportFailOnly":true
}
]Transformers
Transformers are used when you need to transform values of extractors before the can be used for correlation. Transformers are based on regular expressions.
"transformers": [
{
"type":"replace",
"source": "{{videoHostPath}}/{{videoTemplate}}",
"target": "videoChunk",
"from": "$Number$",
"to": "{{$lapIdx1}}"
},
{
"type":"extract",
"source": "{{manifest}}",
"target": "videoHostPath",
"from": "^((?:\/\/|[^\/]+)*(.*))\/"
}
]property | required | description ---- |-----| -------------------------------- type | yes | Can be of value replace or extract ||| replace: Replace source and place result in variables defined in target ||| extract: from source and place result in variables defined in target. Regular expression with groups in from | source | yes | The source expression. Can contain mustache expression | target | yes | A comma separated list of variable names. Separated list only valid for extract with several groups | from | no | A value for replace from or the regular expression for extract | to | no | A value to replace to. Can contain mustache expression
Variables
Variables are used for storing values of extractors or as input parameters.
"variables": [
{
"key": "eventId",
"type": "number",
"usage": "",
"value": "'let arr=[1,2];arr[Math.floor(Math.random() * arr.length)]'"
},
{
"key": "requestIdleTime",
"type": "number",
"usage": "input",
"value": 1000,
"validation": "value > 999 && value <= 15000"
},
{
"key": "capacity",
"type": "number",
"usage": "returnValue",
"value": 0,
"unit": "seats"
},
{
"key": "venueName",
"type": "string",
"usage": "inResponse",
"value": ""
}
]- The usage property is important for variables.
- input: is input variables which can be changed with -i flag in the cli interface. You should include validation of them
- returnValue: Will be the return value of the test run
- inResponse: Is additional information in stored in the test result.
- You can set an initial value with JavaScript. It requires double dots to work. See above
- Validation is also done with JavaScript, but does not require double dots.
Dynamic variables
Variables which are not defined int variables sections are called dynamic variables. They are created by extractors and have keys not defined in the variables section.
Variable placeholders
All variables can be defined with the mustache syntax. A placeholder looks like this {{variableName}}
System variables
- $timestamp - Current timestamp (ms) epoch format
- $testName - Name of current test
- $stepName - Name of current step
- $lap - The lap number when an iterator is used. Starts with 0
- $lapIdx1 - The lap number when an iterator is used. Starts with 1
- $durationMs - The duration in ms of last request
Example: {{$testName}} - Return the name of the test.
Timings variables
All http timings for last request are defined in variables which you can use in assertions and scripts. The following example shows an assertion using the timing dnsTime. See the json report for all valid timings.
{
"description": "Validate dns time",
"failStep": false,
"expression": "value < 5",
"value": "{{$timings.dnsTime}}",
"type": "javaScript",
"reportFailOnly": false
}Random data generation
Random data generation with the NPM module faker is supported. Can be used as variables or be dynamically generated with mustache syntax for placeholders.
{
"key": "email",
"type": "string",
"usage": "inResponse",
"value": "{{$faker.internet.email}}"
}See: https://www.npmjs.com/package/faker for all placeholders
Posting data with x-www-form-urlencoded content type
You can use standard JSON for posting an application/x-www-form-urlencoded form with comma separated values.
{
"name": "Azure Portal Login (OAUTH2)",
"disabled":false,
"requests": [
{
"config": {
"method": "post",
"url": "https://login.microsoftonline.com/{{tentantId}}/oauth2/v2.0/token",
"data": {
"username": "{{username}}",
"password": "{{password}}",
"grant_type": "password",
"client_secret": "{{client_secret}}",
"client_id": "{{client_id}}",
"scope": "https://graph.microsoft.com/.default offline_access"
},
"headers": {
"Content-type": "application/x-www-form-urlencoded",
"Accept": "application/json; charset=utf-8"
}
},
"extractors": [
{
"type": "jsonpath",
"expression": "$.access_token",
"variable": "accessToken"
},
{
"type": "jsonpath",
"expression": "$.refresh_token",
"variable": "refreshToken"
}
],
"assertions": [
{
"type": "javaScript",
"value": "{{accessToken}}",
"description": "Login must return a valid access token.",
"expression": "value !== undefined",
"failStep": true,
"reportFailOnly": true
}
]
}
]
} Here we also show the feature that steps and requests can be disabled with the disabled boolean property
JavaScript support
You can run JavaScript as post and pre processing of request. Javascript can be injected both on step and request level. Often used as another way of extract values from response. Javascript also support the chai style of assertions. Look at the chai assertions api.
Javascript runs in a safe sandbox based on the Node VM with VM2 interface (see npm vm2). You can define the Javascript directly in the json as an string or an array. Javascript can also be defined in external files, when you run url-xi in project mode. If a project is defined to reusable Java Script files should be stored in lib sub directory in the project directory or project zip file.
"scripts": [
{
"scope": "after",
"name":"Get random event id",
"script": "getRandomEvent.js"
}
],JavaScript Example - getRandomEvent.js
/*
Get a random event id from events json array.
Put the extracted event id in the variable eventId
*/
var jsonData = responseData
expect(jsonData,"Must return any none empty array with at least 2 elements").to.have.lengthOf.above(1);
var randomIdx = parseInt(Math.floor(Math.random() * jsonData.length))
console.info('randomIdx',randomIdx, jsonData.length)
logger.debug("Random Index=%s , Response Type%s",randomIdx,responseType)
uxs.setVar('eventId', jsonData[randomIdx].id) JavaScript Example -prepareUpdate.js
This example shows what you can do in a before request script. The script updates the request configuration object.
/*
* Update configuration (data and url) before running the request.
*/
let check=uxs.getVar('check')
let target_sla=check.target_sla || 0
if(target_sla < 99.5)
target_sla = 99.5
let apiName= check.check_type_api
switch (check.check_type_ap) {
case 'url v2':
apiName='url-v2'
break
case 'cmdxtemplated':
apiName='command-v2'
break
}
let url=`checks/${apiName}/${check.id}`
logger.debug('url=%s',url)
requestConfig.url=url
requestConfig.data = {
"target_sla_2": target_sla ,
"target_sla": target_sla
}JavaScript API shortly
Method |Scope| Description --- | --- |--- responseBody |After|Response data . TypeArray and Object is converted to JSON string. responseData|After| AfterResponse data raw format. responseType |After| Type of response. requestConfig |Before| The request configuration. Can be changed by the script. timings|After| The http timings for corresponding request,step or test. Example timings.dnsTime uxs.setVar(name,value)|All| Set the variable name to value uxs.getVar(name) | All|Get variable value for variable named name expect (Chai Expression...) | After|Chai expect assertion in expect style assert (Chai Expression...) | After|Chai expect assertion in assert style logger |All |Log a message with log4js
Iterators
You can use an iterator to iterate a step in several laps. An iterator can be based on a number or an array. Variable substitution is supported for the value of the iterator. The variable must be of type array and extraction must have the array flag.
Iterator properties
Property | Required | Description --- | --- |--- varName | yes | Target variable name in the iteration. Will often contain a specific value for each lap of iteration value | yes | A value or an array of values. Can contain mustache expressions ||| Value is a numeric value: It is interpreted as the number of iterations ||| Value is an array: The array length is interpreted as number of iterations. The varName will contain the value item corresponding to the lap of the iteration maxLaps|no | Additional property for specify max number of laps for an array value. Can contain a mustache expression waitForValidResponse | no | Will do polling of requests in step until an assertion with failStep set is returning success
Extractor for iterator
{
"type": "regexp",
"expression": "<script\\s+src=\"(.+)\">\\s+<\/script>",
"variable": "javaScripts",
"array": true
}Step Example 1 - Get extracted Java Scripts
{
"name": "Get JavaScripts",
"iterator": {
"varName": "javaScript",
"value": "{{javaScripts}}"
},
"requests": [
{
"config": {
"method": "get",
"url": "{{javaScript}}"
},
"extractors": [],
"notSaveData": true
}
]
},Step example 2 - Run all http methods
{
"name": "HTTP Methods",
"iterator": {
"varName": "method",
"value": [
"get",
"post",
"patch",
"put",
"delete"
]
},
"requests": [
{
"config": {
"method": "{{method}}",
"url": "/{{method}}",
"data": "{\"testdata\":true,\"timestamp\":{{$timestamp}}}"
},
"extractors": [
{
"type": "header",
"expression": "server",
"variable": "server"
},
{
"type": "jsonpath",
"expression": "$.origin",
"variable": "origin"
}
],
"assertions": [
{
"type": "javaScript",
"value": "{{origin}}",
"description": "Returned origin should contain an IP address. Method={{method}}",
"expression": "/^((25[0-5]|(2[0-4]|1[0-9]|[1-9]|)[0-9])(\\.(?!$)|$)){4}$/.test(value)",
"failStep": true,
"reportFailOnly":true
}
]
}
]
}Step example 3 - Iterator which polls for valid data
{
"name": "Poll for new result",
"idleBetweenRequests": "{{requestIdleTime}}",
"iterator": {
"value": "{{pollCount}}",
"waitForValidResponse": true
}
}- Iterator count is in value property
- WaitForValidResponse will do polling of requests in step until an assertion with failStep set is returning success
- You also see a new feature here it is idleBetweenRequests. It can be used on test and step level
Request data as a string
Data in request as JSON is supported by default. If data is string format you must use an array for complex requests. Here comes a SOAP example with xml data.
{
"name": "Get Remaining Tickets for Game",
"requests": [
{
"config": {
"method": "post",
"url": "/CheckGamesService.svc",
"data": [
"<soap:Envelope xmlns:soap=\"http://www.w3.org/2003/05/soap-envelope\" xmlns:tem=\"http://tempuri.org/\">",
"<soap:Header xmlns:wsa=\"http://www.w3.org/2005/08/addressing\">",
"<wsa:To>http://sesthbwb09p.apica.local:8001/CheckGamesService.svc</wsa:To>",
"<wsa:Action>http://tempuri.org/ICheckGamesService/RemainingTicketsPerGameId</wsa:Action></soap:Header>",
"<soap:Body>",
"<tem:RemainingTicketsPerGameId>",
"<tem:gameID>{{gameId}}</tem:gameID>",
"<tem:isCachingOff>true</tem:isCachingOff>",
"</tem:RemainingTicketsPerGameId>",
"</soap:Body>",
"</soap:Envelope>"
],
"headers": {
"SOAPAction": "http://tempuri.org/ICheckGamesService/RemainingTicketsPerGameId"
}
},
"extractors": [
{
"type": "xpath",
"expression": "//*[local-name() = 'RemainingTicketsPerGameIdResult']/text()",
"variable": "remainingTickets"
}
]
}
]
}Status code validation for request response.
- Default is that a response status between 200 and 299 is interpreted as a successful request
- You can override that with the expectedStatus array defined in the request section.
{
"expectedStatus": [401],
"config": {
"method": "get",
"url": "/basic-auth/foo/error",
"auth": {
"username": "foo",
"password": "bar"
}
}
}Override default management of request errors
You can override the default management of request errrors with onRequestError property defined on step or request level Value| Description ----- |--- stopTest | Stop the complete test. It is the default directive. nextRequest | Continue with next request in the step without change the status of the request nextStep | Continue with the next step if the request failed. Do nothing for successful requests.
"name": "Home page",
"onRequestError":"nextRequest",Supported syntax for requests
Url-xi is based on the Axios framework. It means that the axios syntax for request configuration can be used. See: https://www.npmjs.com/package/axios
The test case schema
A test case is validated with by follow JSON schema.
"$schema":"https://files-apicasystem-com.s3-eu-west-1.amazonaws.com/schemas/url-xi-schema-v1-0.json",
"name": "HTTP-Bin Test HTTP Methods",The published schema can be found here: https://files-apicasystem-com.s3-eu-west-1.amazonaws.com/schemas/url-xi-schema-v1-0.json
Running an URL-XI test in the command line
url-xi -f samples/tm_order_tickets.json
url-xi -h
Usage: index [options]
Options:
-V, --version output the version number
-f, --file <test_file> The test configuration file
-r, --results <result_dir> The result directory
-xh, --xheaders <headers> extra headers (default: "{}")
-i, --inputs [inputs...] input variables. Format name=value format
-u, --url <url> base url
-l, --log_level <log_level> log level (default: "info")
-nd, --nodata no response/request data in report
-m, --mask Mask sensitive data from report
-po, --parse_only parse json only. No not run (default: false)
-prod, --production Production mode. Minimal logging and content in results (default: false)
--no_keep_alive No keep alive of http connections
-rn, --result_name <result_name> name of the result
-s, --server start as server
-p, --port <port> server port (default: "8070")
-tc, --time_calc <time_calc> Custom request-time calculation (default: "totalTime")
-of, --out_format <out_format> Output format in json result. CRS or Default (default: "default")
-proj --project <project> Project should be a directory or a zip file
-dk, --decryptKey <decryptKey> Cryptify Decrypt key
-ts, --table_server <table_server> Apica Table Server (Experimental testing)
-h, --help display help for commandSpecial command line options and arguments to options
Option | Argument | Description --- | --- |--- -f | A json file | The url-xi test case -r | Optional result directory | Result directory . A valid directory . Must exists -i | Input variables | Specify each input variable as name=vale -xh| Optional extra headers | Extra headers in result. Specify as json with header name and value pairs -l | Log level for log4js | See log levels for the log4js package. Default "info" -u | Base url |Change base url in test file -po | Parse only | Parse the test file only. Do not run -prod | Production usage | For usage in production with minimal logging in the result file. -dk | Decryption key | For usage in a project to decrypt encrypted files -of | Output format | crs=For usage in Apica ASM. Convert result to CRS format used in Apica ASM
Syntax for input variables
url-xi -f samples/default_test.json -i defIdleTime=1000 requestIdleTime=1000CLI Console Report
----- Process results [Ticketmonster Home Page] -----
----- [Test Summary] -----
Test Name: Ticketmonster Home Page
Total Response Time: 310
Start Time: 2020-12-18T10:09:00.894Z
End Time: 2020-12-18T10:09:01.222Z
Number of steps: 1
Total Content length: 493
Return value: 310
Result success: true
----- [Steps result] -----
Home page
[success=true, duration=310, content-length=493, start time=2020-12-18T10:09:00.896Z, ignore duration=false]
[GET] http://ticketmonster.apicasystem.com/ticket-monster/
[Success=true, Duration=310.12, Content-length=493,Start time=2020-12-18T10:09:00.896Z, Status=(200 : OK)]
Timings [ Wait=152.18, DNS=6.71, SSL/TLS handshake =0.00 TCP=73.43, FirstByte=83.08, Download=1.42]
Customize Request result reporting
Most API monitoring tools can report total response time of requests and summarize them to a total. Url-xi has a much more advanced feature for this which is tailored for monitoring. It will help you find performance and availability issues from several perspectives. You may want to investigate DNS response time or only the server response time (Time To First Byte). This is the options:
"requestTimeCalculation": {
"description": "Custom calculation of request response time. Default total response time in ms.",
"type": "string",
"enum": [
"TotalTime",
"Request",
"DNS",
"TimeToFirstBuffer",
"DownloadTime",
"ContentLength"
]Running url-xi as a http server
This is experimental. Does not support all rn-time options
url-xi -s
[2020-08-05T22:30:41.244] [INFO] url-xi - url-xi(1.8.1) started with [
'/usr/local/bin/node',
'/Users/janostgren/work/node/url-xi/dist/cli/index.js',
'-s'
]
[2020-08-05T22:30:41.252] [INFO] url-xi - URL XI server (version 1.1.5) started on http port 8066API
POST http://localhost:8066/api/url-xi/run - Run a test case POST http://localhost:8066/api/url-xi/parse - Parse a test case
Supported query parameter
- nodata = Produce result without data
- baseUrl = change the base URL. Qual as -u parameter in cli interface
- inputs = List of input variables . Example : inputs="api_key=DEMO_KEY"
curl -i -X POST -d @./samples/default_test.json http:/localhost:8066/api/url-xi/parse -H "Content-Type: application/json; charset=UTF-8"
HTTP/1.1 100 Continue
HTTP/1.1 200 OK
X-Powered-By: Express
Content-Type: application/json; charset=utf-8
Content-Length: 24
ETag: W/"18-Uv4N+TqqnzgG/uMPTz4ruEfRYrY"
Date: Wed, 05 Aug 2020 20:40:24 GMT
Connection: keep-alive
{"message":"Parsing ok"}Samples
Samples are found in the installation directory of url-xi. It is the global node directory followed by url-xi/samples
- Linux/Unix : /usr/local/lib/node_modules/url-xi/samples
- Windows : Dynamic. PC global installs happen under %APPDATA%:
$ ls -l /usr/local/lib/node_modules/url-xi/samples
total 80
-rw-r--r-- 1 janostgren admin 4051 26 Okt 1985 app-insight-demo.json
-rw-r--r-- 1 janostgren admin 4579 26 Okt 1985 cldemo_soap_game_service.json
-rw-r--r-- 1 janostgren admin 3015 26 Okt 1985 default_test.json
-rw-r--r-- 1 janostgren admin 10588 26 Okt 1985 http-bin-test.json
-rw-r--r-- 1 janostgren admin 417 26 Okt 1985 tm_home.json
-rw-r--r-- 1 janostgren admin 5944 26 Okt 1985 tm_order_tickets.jsonSimple Example
This i a minimal configuration
{
"name": "Ticketmonster Home Page",
"description": "Simple Scenario for the TM home page",
"baseURL": "http://ticketmonster.apicasystem.com",
"steps": [
{
"name": "Home page",
"requests": [
{
"config": {
"url": "/ticket-monster"
}
}
]
}
]
}Advanced Sample - Ticket Monster order tickets
Complete flow for order tickets in the Apica demo application TicketMonster. It also uses the project feature for setting up common variables, test data and JavaScripts.
{
"$schema": "https://files-apicasystem-com.s3-eu-west-1.amazonaws.com/schemas/url-xi-schema-v1-0.json",
"name": "Ticket Monster Order Tickets",
"description": "Order tickets in Ticket Monster. Setup AppDynamics integration headers",
"variables": [
{
"key": "email",
"type": "string",
"usage": "inResponse",
"value": "{{$faker.internet.email}}",
"description": "A random generated email"
},
{
"key": "bookingId",
"type": "number",
"usage": "inResponse"
},
{
"key": "ticketPrice",
"type": "number",
"usage": "inResponse"
},
{
"key": "ticketCategory",
"type": "string",
"usage": "inResponse"
},
{
"key": "ticketSection",
"type": "string",
"usage": "inResponse"
},
{
"key": "eventName",
"type": "string",
"usage": "inResponse"
},
{
"key": "venueName",
"type": "string",
"usage": "inResponse"
}
],
"includes": [
{
"name": "testdata",
"scope": "project",
"type": "data",
"src": "my_test_data.json"
},
{
"name": "defaultVariables",
"scope": "project",
"type": "vars",
"src": "default_test_vars.json"
}
],
"baseURL": "http://ticketmonster.apicasystem.com",
"config": {
"headers": {
"ApicaScenario": "{{$testName}}",
"ApicaCheckId": "92e02d76-63cf-4a54-a4d6-29b9488fdc1a ",
"AppDynamicsSnapshotEnabled": "true"
}
},
"steps": [
{
"name": "Home page",
"requests": [
{
"name":"Start page html",
"config": {
"method": "get",
"url": "/ticket-monster",
"headers": {
"ApicaStep": "{{$stepName}}"
}
},
"extractors": [
{
"type": "regexp",
"expression": "<script type=\"text\/javascript\".*src=\"(.*.js)\".*",
"variable": "javaScripts",
"array": true
},
{
"type": "regexp",
"expression": "<title>(.+)<\/title>",
"variable": "title"
}
],
"assertions": [
{
"description": "Home page must contain javascript references",
"failStep": true,
"reportFailOnly": false,
"value": "{{javaScripts}}",
"expression": "value.length > 1",
"type": "javaScript"
},
{
"description": "Page title must be Ticket Monster",
"type": "value",
"value": "{{title}}",
"expression": "Ticket Monster",
"failStep": true
}
]
}
]
},
{
"name": "Get JavaScripts",
"iterator": {
"varName": "javaScript",
"value": "{{javaScripts}}"
},
"requests": [
{
"name": "Javascript {{$lapIdx1}}",
"config": {
"method": "get",
"url": "/ticket-monster/{{javaScript}}"
},
"extractors": [],
"notSaveData": true
}
]
},
{
"name": "Get Events",
"requests": [
{
"name": "Get all events",
"config": {
"method": "get",
"url": "/ticket-monster/rest/events",
"headers": {
"ApicaStep": "{{$stepName}}"
},
"params": {
"_": "{{$timestamp}}"
}
},
"scripts": [
{
"scope": "after",
"name":"Get random event id",
"script": "getRandomEvent.js"
}
],
"assertions": [
{
"description": "A numeric event id must be extracted",
"failStep": true,
"reportFailOnly": false,
"type": "javaScript",
"value": "{{eventId}}",
"expression": "!isNaN(value) && Number(value) >0"
}
]
}
]
},
{
"name": "Get Tickets",
"requests": [
{
"name": "Get shows",
"config": {
"method": "get",
"url": "/ticket-monster/rest/shows",
"params": {
"_": "{{$timestamp}}",
"event": "{{eventId}}"
},
"headers": {
"ApicaStep": "{{$stepName}}"
}
},
"extractors": [
{
"type": "jsonpath",
"expression": "$[*].id",
"variable": "showId"
}
]
},
{
"name": "Select Tickets",
"config": {
"method": "get",
"url": "/ticket-monster/rest/shows/{{showId}}",
"params": {
"_": "{{$timestamp}}"
},
"headers": {
"ApicaStep": "{{$stepName}}"
}
},
"extractors": [
{
"type": "jsonpath",
"expression": "$.performances[*].id",
"variable": "performanceId"
},
{
"type": "jsonpath",
"expression": "$.event.name",
"variable": "eventName"
},
{
"type": "jsonpath",
"expression": "$.venue.name",
"variable": "venueName"
},
{
"type": "jsonpath",
"expression": "$.ticketPrices[*]",
"variable": "ticketPrices",
"index": true
},
{
"type": "jsonpath",
"expression": "$.ticketPrices[{{ticketPrices}}].id",
"variable": "ticketPriceId"
},
{
"type": "jsonpath",
"expression": "$.ticketPrices[{{ticketPrices}}].section.name",
"variable": "ticketSection"
},
{
"type": "jsonpath",
"expression": "$.ticketPrices[{{ticketPrices}}].ticketCategory.description",
"variable": "ticketCategory"
},
{
"type": "jsonpath",
"expression": "$.ticketPrices[{{ticketPrices}}].price",
"variable": "ticketPrice"
}
]
}
]
},
{
"name": "Checkout",
"requests": [
{
"name":"Create booking",
"config": {
"method": "post",
"url": "/ticket-monster/rest/bookings",
"data": {
"ticketRequests": [
{
"ticketPrice": "{{ticketPriceId}}",
"quantity": 1
}
],
"email": "{{email}}",
"performance": "{{performanceId}}"
},
"headers": {
"ApicaStep": "{{$stepName}}"
}
},
"extractors": [
{
"type": "jsonpath",
"expression": "$.id",
"variable": "bookingId"
}
]
}
]
},
{
"name": "Undo the ticket booking",
"requests": [
{
"name":"Delete booking",
"config": {
"method": "delete",
"url": "/ticket-monster/rest/bookings/{{bookingId}}",
"headers": {
"ApicaStep": "{{$stepName}}"
}
}
}
]
}
]
}
Result report example
This example is without result data. The production switch is used.
{
"name": "Ticket Monster Order Tickets",
"baseURL": "http://ticketmonster.apicasystem.com",
"type": "URL-XI",
"producer": "@apica-io/url-xi 3.4.0",
"resultVersion": "1.0",
"flowControl": "Chained Flow",
"returnValue": 1891,
"unit": "ms",
"success": true,
"durationMs": 1891,
"startTimestamp": 1621853129132,
"endTimestamp": 1621853131070,
"contentLength": 161557,
"timings": {
"socketWait": 192,
"dnsTime": 1,
"secureHandshake": 0,
"tcpConnect": 60,
"timeToFirstByte": 1066,
"downloadTime": 572,
"totalTime": 1891
},
"variables": [
{
"key": "email",
"type": "string",
"usage": "inResponse",
"value": "[email protected]",
"description": "A random generated email"
},
{
"key": "bookingId",
"type": "number",
"usage": "inResponse",
"value": 7107109
},
{
"key": "ticketPrice",
"type": "number",
"usage": "inResponse",
"value": 150
},
{
"key": "ticketCategory",
"type": "string",
"usage": "inResponse",
"value": "Adult"
},
{
"key": "ticketSection",
"type": "string",
"usage": "inResponse",
"value": "Section 73"
},
{
"key": "eventName",
"type": "string",
"usage": "inResponse",
"value": "Champions League"
},
{
"key": "venueName",
"type": "string",
"usage": "inResponse",
"value": "Camp Nou"
},
{
"key": "eventId",
"type": "number",
"usage": "inResponse",
"value": 1,
"description": "The found event id"
}
],
"steps": [
{
"name": "Home page",
"success": true,
"durationMs": 350,
"startTimestamp": 1621853129134,
"endTimestamp": 1621853129502,
"contentLength": 493,
"timings": {
"socketWait": 189,
"dnsTime": 1,
"secureHandshake": 0,
"tcpConnect": 60,
"timeToFirstByte": 97,
"downloadTime": 3,
"totalTime": 350
},
"ignoreDuration": false,
"requests": [
{
"name": "Start page html",
"url": "http://ticketmonster.apicasystem.com/ticket-monster/",
"method": "get",
"requestHeaders": {
"Accept": "application/json, text/plain, */*",
"ApicaScenario": "Ticket Monster Order Tickets",
"ApicaCheckId": "92e02d76-63cf-4a54-a4d6-29b9488fdc1a ",
"AppDynamicsSnapshotEnabled": "true",
"User-Agent": "url-xi-3.4.0",
"ApicaStep": "Home page"
},
"success": true,
"durationMs": 350.3891910000002,
"startTimestamp": 1621853129135,
"endTimestamp": 1621853129502,
"contentLength": 493,
"timings": {
"socketWait": 189,
"dnsTime": 1,
"secureHandshake": 0,
"tcpConnect": 60,
"timeToFirstByte": 97,
"downloadTime": 3,
"totalTime": 350
},
"status": 200,
"statusText": "OK",
"headers": {
"server": "nginx/1.10.3 (Ubuntu)",
"date": "Mon, 24 May 2021 10:45:29 GMT",
"content-type": "text/html",
"content-length": "493",
"connection": "keep-alive",
"last-modified": "Tue, 16 Jul 2019 09:23:34 GMT",
"x-powered-by": "Undertow/1"
}
}
],
"assertions": [
{
"source": "Start page html",
"description": "Home page must contain javascript references",
"status": "info",
"value": [
"resources/js/libs/modernizr-2.8.3.min.js",
"resources/js/libs/require.js"
],
"expression": "value.length > 1"
},
{
"source": "Start page html",
"description": "Page title must be Ticket Monster",
"status": "info",
"value": "Ticket Monster",
"expression": "Ticket Monster"
}
]
},
{
"name": "Get JavaScripts",
"success": true,
"durationMs": 531,
"startTimestamp": 1621853129502,
"endTimestamp": 1621853130037,
"contentLength": 91475,
"timings": {
"socketWait": 1,
"dnsTime": 0,
"secureHandshake": 0,
"tcpConnect": 0,
"timeToFirstByte": 176,
"downloadTime": 354,
"totalTime": 531
},
"ignoreDuration": false,
"requests": [
{
"name": "Javascript 1",
"url": "http://ticketmonster.apicasystem.com/ticket-monster/resources/js/libs/modernizr-2.8.3.min.js",
"method": "get",
"requestHeaders": {
"Accept": "application/json, text/plain, */*",
"ApicaScenario": "Ticket Monster Order Tickets",
"ApicaCheckId": "92e02d76-63cf-4a54-a4d6-29b9488fdc1a ",
"AppDynamicsSnapshotEnabled": "true",
"User-Agent": "url-xi-3.4.0"
},
"success": true,
"durationMs": 104.66474999999991,
"startTimestamp": 1621853129502,
"endTimestamp": 1621853129609,
"contentLength": 8849,
"timings": {
"socketWait": 0,
"dnsTime": 0,
"secureHandshake": 0,
"tcpConnect": 0,
"timeToFirstByte": 87,
"downloadTime": 17,
"totalTime": 105
},
"status": 200,
"statusText": "OK",
"headers": {
"server": "nginx/1.10.3 (Ubuntu)",
"date": "Mon, 24 May 2021 10:45:29 GMT",
"content-type": "application/javascript",
"content-length": "8849",
"connection": "keep-alive",
"last-modified": "Tue, 16 Jul 2019 09:23:34 GMT",
"x-powered-by": "Undertow/1"
}
},
{
"name": "Javascript 2",
"url": "http://ticketmonster.apicasystem.com/ticket-monster/resources/js/libs/require.js",
"method": "get",
"requestHeaders": {
"Accept": "application/json, text/plain, */*",
"ApicaScenario": "Ticket Monster Order Tickets",
"ApicaCheckId": "92e02d76-63cf-4a54-a4d6-29b9488fdc1a ",
"AppDynamicsSnapshotEnabled": "true",
"User-Agent": "url-xi-3.4.0"
},
"success": true,
"durationMs": 426.55309099999977,
"startTimestamp": 1621853129609,
"endTimestamp": 1621853130037,
"contentLength": 82626,
"timings": {
"socketWait": 0,
"dnsTime": 0,
"secureHandshake": 0,
"tcpConnect": 0,
"timeToFirstByte": 89,
"downloadTime": 337,
"totalTime": 427
},
"status": 200,
"statusText": "OK",
"headers": {
"server": "nginx/1.10.3 (Ubuntu)",
"date": "Mon, 24 May 2021 10:45:29 GMT",
"content-type": "application/javascript",
"content-length": "82626",
"connection": "keep-alive",
"last-modified": "Tue, 16 Jul 2019 09:23:34 GMT",
"x-powered-by": "Undertow/1"
}
}
]
},
{
"name": "Get Events",
"success": true,
"durationMs": 105,
"startTimestamp": 1621853130038,
"endTimestamp": 1621853130152,
"contentLength": 589,
"timings": {
"socketWait": 0,
"dnsTime": 0,
"secureHandshake": 0,
"tcpConnect": 0,
"timeToFirstByte": 105,
"downloadTime": 0,
"totalTime": 105
},
"ignoreDuration": false,
"requests": [
{
"name": "Get all events",
"url": "http://ticketmonster.apicasystem.com/ticket-monster/rest/events?_=1621853130038",
"method": "get",
"requestHeaders": {
"Accept": "application/json, text/plain, */*",
"ApicaScenario": "Ticket Monster Order Tickets",
"ApicaCheckId": "92e02d76-63cf-4a54-a4d6-29b9488fdc1a ",
"AppDynamicsSnapshotEnabled": "true",
"User-Agent": "url-xi-3.4.0",
"ApicaStep": "Get Events"
},
"success": true,
"durationMs": 105.08714000000009,
"startTimestamp": 1621853130038,
"endTimestamp": 1621853130152,
"contentLength": 589,
"timings": {
"socketWait": 0,
"dnsTime": 0,
"secureHandshake": 0,
"tcpConnect": 0,
"timeToFirstByte": 105,
"downloadTime": 0,
"totalTime": 105
},
"status": 200,
"statusText": "OK",
"headers": {
"server": "nginx/1.10.3 (Ubuntu)",
"date": "Mon, 24 May 2021 10:45:30 GMT",
"content-type": "application/json",
"content-length": "589",
"connection": "keep-alive",
"x-powered-by": "Undertow/1"
}
}
],
"assertions": [
{
"source": "Get all events",
"description": "A numeric event id must be extracted",
"status": "info",
"value": "1",
"expression": "!isNaN(value) && Number(value) >0"
}
]
},
{
"name": "Get Tickets",
"success": true,
"durationMs": 409,
"startTimestamp": 1621853130152,
"endTimestamp": 1621853130569,
"contentLength": 68584,
"timings": {
"socketWait": 0,
"dnsTime": 0,
"secureHandshake": 0,
"tcpConnect": 0,
"timeToFirstByte": 195,
"downloadTime": 214,
"totalTime": 409
},
"ignoreDuration": false,
"requests": [
{
"name": "Get shows",
"url": "http://ticketmonster.apicasystem.com/ticket-monster/rest/shows?_=1621853130152&event=1",
"method": "get",
"requestHeaders": {
"Accept": "application/json, text/plain, */*",
"ApicaScenario": "Ticket Monster Order Tickets",
"ApicaCheckId": "92e02d76-63cf-4a54-a4d6-29b9488fdc1a ",
"AppDynamicsSnapshotEnabled": "true",
"User-Agent": "url-xi-3.4.0",
"ApicaStep": "Get Tickets"
},
"success": true,
"durationMs": 206.78929000000016,
"startTimestamp": 1621853130153,
"endTimestamp": 1621853130363,
"contentLength": 34293,
"timings": {
"socketWait": 0,
"dnsTime": 0,
"secureHandshake": 0,
"tcpConnect": 0,
"timeToFirstByte": 89,
"downloadTime": 118,
"totalTime": 207
},
"status": 200,
"statusText": "OK",
"headers": {
"server": "nginx/1.10.3 (Ubuntu)",
"date": "Mon, 24 May 2021 10:45:30 GMT",
"content-type": "application/json",
"transfer-encoding": "chunked",
"connection": "keep-alive",
"x-powered-by": "Undertow/1"
}
},
{
"name": "Select Tickets",
"url": "http://ticketmonster.apicasystem.com/ticket-monster/rest/shows/2?_=1621853130363",
"method": "get",
"requestHeaders": {
"Accept": "application/json, text/plain, */*",
"ApicaScenario": "Ticket Monster Order Tickets",
"ApicaCheckId": "92e02d76-63cf-4a54-a4d6-29b9488fdc1a ",
"AppDynamicsSnapshotEnabled": "true",
"User-Agent": "url-xi-3.4.0",
"ApicaStep": "Get Tickets"
},
"success": true,
"durationMs": 202.4201109999999,
"startTimestamp": 1621853130363,
"endTimestamp": 1621853130569,
"contentLength": 34291,
"timings": {
"socketWait": 0,
"dnsTime": 0,
"secureHandshake": 0,
"tcpConnect": 0,
"timeToFirstByte": 106,
"downloadTime": 96,
"totalTime": 202
},
"status": 200,
"statusText": "OK",
"headers": {
"server": "nginx/1.10.3 (Ubuntu)",
"date": "Mon, 24 May 2021 10:45:30 GMT",
"content-type": "application/json",
"transfer-encoding": "chunked",
"connection": "keep-alive",
"x-powered-by": "Undertow/1"
}
}
]
},
{
"name": "Checkout",
"success": true,
"durationMs": 374,
"startTimestamp": 1621853130570,
"endTimestamp": 1621853130946,
"contentLength": 416,
"timings": {
"socketWait": 1,
"dnsTime": 0,
"secureHandshake": 0,
"tcpConnect": 0,
"timeToFirstByte": 373,
"downloadTime": 1,
"totalTime": 374
},
"ignoreDuration": false,
"requests": [
{
"name": "Create booking",
"url": "http://ticketmonster.apicasystem.com/ticket-monster/rest/bookings",
"method": "post",
"requestHeaders": {
"Accept": "application/json, text/plain, */*",
"Content-Type": "application/json;charset=utf-8",
"ApicaScenario": "Ticket Monster Order Tickets",
"ApicaCheckId": "92e02d76-63cf-4a54-a4d6-29b9488fdc1a ",
"AppDynamicsSnapshotEnabled": "true",
"User-Agent": "url-xi-3.4.0",
"ApicaStep": "Checkout",
"Content-Length": 98
},
"success": true,
"durationMs": 374.3788159999999,
"startTimestamp": 1621853130570,
"endTimestamp": 1621853130946,
"contentLength": 416,
"timings": {
"socketWait": 1,
"dnsTime": 0,
"secureHandshake": 0,
"tcpConnect": 0,
"timeToFirstByte": 373,
"downloadTime": 1,
"totalTime": 374
},
"status": 200,
"statusText": "OK",
"headers": {
"server": "nginx/1.10.3 (Ubuntu)",
"date": "Mon, 24 May 2021 10:45:31 GMT",
"content-type": "application/json",
"content-length": "416",
"connection": "keep-alive",
"x-powered-by": "Undertow/1"
}
}
]
},
{
"name": "Undo the ticket booking",
"success": true,
"durationMs": 121,
"startTimestamp": 1621853130947,
"endTimestamp": 1621853131069,
"contentLength": 0,
"timings": {
"socketWait": 0,
"dnsTime": 0,
"secureHandshake": 0,
"tcpConnect": 0,
"timeToFirstByte": 120,
"downloadTime": 0,
"totalTime": 121
},
"ignoreDuration": false,
"requests": [
{
"name": "Delete booking",
"url": "http://ticketmonster.apicasystem.com/ticket-monster/rest/bookings/7107109",
"method": "delete",
"requestHeaders": {
"Accept": "application/json, text/plain, */*",
"ApicaScenario": "Ticket Monster Order Tickets",
"ApicaCheckId": "92e02d76-63cf-4a54-a4d6-29b9488fdc1a ",
"AppDynamicsSnapshotEnabled": "true",
"User-Agent": "url-xi-3.4.0",
"ApicaStep": "Undo the ticket booking"
},
"success": true,
"durationMs": 120.96489199999996,
"startTimestamp": 1621853130947,
"endTimestamp": 1621853131069,
"contentLength": 0,
"timings": {
"socketWait": 0,
"dnsTime": 0,
"secureHandshake": 0,
"tcpConnect": 0,
"timeToFirstByte": 120,
"downloadTime": 0,
"totalTime": 121
},
"status": 204,
"statusText": "No Content",
"headers": {
"server": "nginx/1.10.3 (Ubuntu)",
"date": "Mon, 24 May 2021 10:45:31 GMT",
"connection": "keep-alive",
"x-powered-by": "Undertow/1"
}
}
]
}
]
}Projects
Project overview
The purpose of a project is to share common assets assets as
- JavaScript files
- Data like test data and certificates
- Variables so the can be reused in several test cases. A project is directory structure on the filesystem. You refer to the top level directory when executing in a project context. The -proj or -- project option is used on the command line for this purpose.
For variables and external test data files you need an includes object the test configuration.
"includes": [
{
"name": "Common Variables",
"scope": "project",
"type": "vars",
"src": "common_vars.json"
},
{
"name": "posts",
"scope": "project",
"type": "data",
"src": "posts.csv",
"options": {}
}
],Project directory structure
See sample in samples directory.
ls -R samples/jsonplaceholder
data lib vars
samples/jsonplaceholder/data:
posts.csv
samples/jsonplaceholder/lib:
getUserId.js reloadATS.js
samples/jsonplaceholder/vars:
common_vars.json| Sub Directory | Description | | ------------- | ------------- | | data | contains included data files on csv or json format. They can be used in iterators | | variables | contains shared variables. Json format | | lib | contains javascript files with shared code |
Use certificates for authentication
You can use client certificates for authentication. It requires project to use certificates. The solution is based on Node JS https.Agent class and its configuration
You should define an optional httpsAgent object on the root level, step level or request level in the test configuration file.
"httpsAgent": {
"rejectUnauthorized": false,
"pfx": "{{$cert.client.p12}}",
"password":"badssl.com",
"passphrase":"badssl.com"
}You also need to define an include object which describe the certificates files
"includes": [
{
"name": "client.p12",
"type": "certificate",
"scope": "project",
"src": "badssl.com-client.p12"
}The certificate badssl.com-client.p12 is stored in the project root directory.
Encrypt and decrypt files in a project
You can use the NPM package cryptify (https://www.npmjs.com/package/cryptify) for decrypting encrypted files in a project. The following file types supports encrypt/decrypt:
- certificates
- data files
- shared variables
You must supply a decryption key with -dk option. The file must been encrypted with the key using the command line version of cryptify. Install the package globally and use the tool.
Installation of cryptify
$ npm install -g cryptify
$ cryptify
Usage: cryptify [options] [command]
Options:
-v, --version Display the current version
-l, --list List available ciphers
-h, --help Display help for the command
Commands:
encrypt [options] <file...> Encrypt files(s)
decrypt [options] <file...> Decrypt files(s)
help <command> Display help for the command
Example:
$ cryptify encrypt file.txt -p 'Secret123!'
$ cryptify decrypt file.txt -p 'Secret123!'
Password Requirements:
1. Must contain at least 8 characters
2. Must contain at least 1 special character
3. Must contain at least 1 numeric character
4. Must contain a combination of uppercase and lowercaseExample of encrypt a file
$ cryptify encrypt common_vars.json --password URL-xi.4.data