carson
v0.11.0
Published
Supervise benchmarks of any RDF Query engine with ease
Downloads
3
Maintainers
Readme
Carson
Carson is an execution engine capable of supervising experimentations on any RDF Query engine. It can be used to measure performance, and various other metrics, on both clients and servers, and can be adapted to any RDF based system.
Why use Carson ?
When you want to perform a benchmark on a query engine, you often spend hours to build a workflow that can run your tests and record all the data you need across multiples servers. Carson allows you to skip this step and focus on what you want to benchmark instead of how to perform the benchmark.
Features
- Easy to configure: setup Carson for your system with one configuration file!
- Deploy reverse proxies on servers of a federation and perform measurements on any incoming request.
- Support many common metrics (execution time, answer's completeness, number of transferred tuples, etc).
- Written using ECMAScript 6.
- Intensively tested
Table of contents
- Installation
- Usage
- Getting started
- What Carson does not do
- Use Carson as a library
- Documentation
- Testing
- License
- Author
Prerequisite
Node.js: version 6.9.0 or higher
Installation
For a global installation (to use it in command line mode)
npm install -g carson
For a local installation (to use it as a library)
npm install --save carson
Usage
Carson can be used through command line (see below) or as a library (see this section).
Usage: carson <config-file> <output> [options]
Supervise benchmarks of any RDF Query engine with ease
Options:
-h, --help output usage information
-V, --version output the version number
-p, --progress display progress during execution
Getting started
To work, Carson only need one configuration file that describes your setup and the metrics you which to use. Let's take a simple example where we want to measure the execution time of one request made to an LDF server:
- First, we start a LDF server on port 5000 (see LDF documentation for this).
- We put a simple SPARQL query
SELECT * WHERE { ?s ?p ?o .}
in the fileall.sparql
. - Next, we use the following configuration:
{
"type": "http",
"servers": [
{
"host": "localhost",
"port": 5000,
"proxyPort": 5001
}
],
"input": {
"type": "single",
"value": "all.sparql"
},
"metrics": ["time"],
"format": {
"client": "application/sparql-results+json",
"federation": "rdf"
},
"run": "ldf-client http://localhost:5001/dbpedia -f $INPUT -t application/sparql-results+json"
}
- Then, we launch Carson with our configuration file and the path where we want to place the results:
carson config.json out/results.json
- The results will look like this (with some variations around the execution time of course):
{
"query#1": {
"requests": [
{
"id": "b52bd2af-b7ff-470e-ae3e-d01a9c65e3e1",
"time": 162,
"startTime": "17:41:41:876",
"endTime": "17:41:42:38",
"origin": "unknown",
"target": {
"port": 5000,
"host": "localhost"
}
}
],
"time": [
{
"time": 649,
"startTime": "17:41:41:439",
"endTime": "17:41:42:88",
"origin": "client",
"target": "federation"
}
]
},
"nbServers": 1,
"nbQueries": 1,
"hardware": {
"arch": "x64",
"platform": "linux",
"os": "Linux",
"cpu": {
"model": "Intel(R) Core(TM) i7-2670QM CPU @ 2.20GHz",
"core": 8
},
"memory": 8371781632
}
}
For more details about how to write a configuration file, please see the related documentation.
What Carson does not do
It's important to clarify what Carson is about: it's a tool which can supervise the execution of a benchmark setup for you. It will setup reverse proxies around the servers of your federation and run queries using specific commands, but you still need to take care of the following tasks:
- Provides the query engine you want to use
- Deploy the servers of your federation.
- Defines the queries used for the benchmark.
- Ensure that both clients & servers observe the format defined in the configuration file.
Use Carson as a library
You can also use Carson as a library and call it through your own code.
const Carson = require('carson');
const app = new Carson('path/to/output/results.json');
const config = {
// some awesome configuration here!
};
app.loadConfig(config)
.then(() => app.run())
.then(() => console.log('Execution completed!'))
.catch(error => console.error(error));
Documentation
Generate JSDoc documentation with the following command
cd carson/
# documentation will be generated in doc/ directory
npm run doc
Testing
cd carson/
# run tests
npm test
# compute code coverage with istanbul
npm run coverage