CompassQL

CompassQL is a visualization query language that powers chart specifications and recommendations in Voyager 2.

As described in our vision paper and Voyager 2 paper, a CompassQL query is a JSON object that contains the following components:

• Specification (spec) for describing a collection of queried visualizations. This spec's syntax follows a structure similar to Vega-Lite's single view specification. However, spec in CompassQL can have enumeration specifiers (or wildcards) describing properties that can be enumerated.1

• Grouping/Nesting method names (groupBy and nest) for grouping queried visualizations into groups or hierarchical groups.

• Ranking method names (orderBy and chooseBy) for ordering queried visualizations or choose a top visualization from the collection.

• Config (config) for customizing query parameters.

Internally, CompassQL engine contains a collection of constraints for enumerating a set of candidate visualizations based on the input specification, and methods for grouping and ranking visualization.

For example, the following CompassQL query has one wildcard for the mark property. The system will automatically generate different marks and choose the top visual encodings based on the effectiveness score.

{
  "spec": {
    "data": {"url": "data/cars.json"},
    "mark": "?",
    "encodings": [
      {
        "channel": "x",
        "aggregate": "mean",
        "field": "Horsepower",
        "type": "quantitative"
      },{
        "channel": "y",
        "field": "Cylinders",
        "type": "ordinal"
      }
    ]
  },
  "chooseBy": "effectiveness"
}

The examples/specs directory contains a number of example CompassQL queries.

To understand more about the structure of a CompassQL Query, look at the Query interface declaration.

• A query's spec property implements SpecQuery interface, which follows the same structure as Vega-Lite's UnitSpec (single view specification) but most of SpecQuery's properties have -Query suffixes to hint that its instance is a query that can contain wildcards to describe a collection of specifications.
• Since multiple encoding channels can be a wildcard, the encoding object in Vega-Lite is flatten as encodings which is an array of Encoding in CompassQL's spec.

Usage

Given a row-based array of data object, here are the steps to use CompassQL:

1. Specify a query config (or use an empty object to use the default configs)

var opt = {}; // Use all default query configs

For all query configuration properties, see src/config.ts.

2. Build a data schema.

var schema = cql.schema.build(data);

The data property is a row-based array of data objects where each object represents a row in the data table (e.g., [{"a": 1, "b":2}, {"a": 2, "b": 3}]).

You can reuse the same schema for querying the same dataset multiple times.

3. Specify a query. For example, this is a query for automatically selecting a mark:

var query = {
  "spec": {
    "data": {"url": "node_modules/vega-datasets/data/cars.json"},
    "mark": "?",
    "encodings": [
      {
        "channel": "x",
        "aggregate": "mean",
        "field": "Horsepower",
        "type": "quantitative"
      },{
        "channel": "y",
        "field": "Cylinders",
        "type": "ordinal"
      }
    ]
  },
  "chooseBy": "effectiveness"
};

4. Execute a CompassQL query.

var output = cql.recommend(query, schema);
var result = output.result; // recommendation result

The result object is an instance of SpecQueryModelGroup (ResultGroup<SpecQueryModel>), which is a root of the output ordered tree. Its items property can be either an array of SpecQueryModel or an array of SpecQueryModelGroup (for hierarchical groupings).

The SpecQueryModel is an class instance of a SpecQuery with helper methods. Note that, in the result, all of spec query models are completely enumerated and there would be no wildcard left.

5. Convert instances of SpecQueryModel in the tree, using SpecQueryModel's toSpec() class method and the mapLeaves method.

var vlTree = cql.result.mapLeaves(result, function(item) {
  return item.toSpec();
});

6. Now you can use the result. In this case, the tree has only 2 levels (the root and leaves). We can just get the top visualization by accessing the 0-th item.

For a full source code, please see index.html.

var topVlSpec = vlTree.items[0];

Note for Developers

• The root file of our project is src/cql.ts, which defines the top-level namespace cql for the compiled files. Other files under src/ reflect namespace structure. All methods for cql.xxx will be in either src/xxx.ts or src/xxx/xxx.ts. For example, cql.util.* methods are in src/util.ts, cql.query is in src/query/query.ts.

• TODO: constraints

  • List in Vy2 paper supplement..

Development Instructions

You can install dependencies with:

yarn install

You can use the following npm commands such as

npm run build
npm run lint
npm run test
npm run cover       // see test coverage  (see coverage/lcov-report/index.html)
npm run watch       // watcher that build, lint, and test
npm run test-debug  // useful for debugging unit-test with vscode
npm run clean       // useful for wiping out js files that's created from other branch

(See package.json for Full list of commands.)

To play with latest CompassQL in the vega-editor, use branch cql-vl3 in kanitw's fork, which has been updated to use Vega-Lite 3, Vega 5, and CompassQL ^0.21.1. (For CompassQL 0.7 or older, use branch compassql, which uses Vega-Lite 1.x).

Make sure to link CompassQL to the editor

cd COMPASSQL_DIR
npm link

cd VEGA_EDITOR_DIR
npm run vendor -- -l compassql

(You might want to link your local version of Vega-Lite as well.)

Main API

The main method is cql.recommend, which is in src/recommend.ts.

Directory Structure

• examples - Example CompassQL queries
  • examples/specs – All JSON files for CompassQL queries
  • examples/cql-examples.json - A json files listing all CompasssQL examples that should be shown in Vega-editor.
• src/ - Main source code directory.
  • src/cql.ts is the root file for CompassQL codebase that exports the global cql object. Other files under src/ reflect namespace structure.
  • All interface for CompassQL syntax should be declared at the top-level of the src/ folder.
• test/ - Code for unit testing. test's structure reflects src's' directory structure. For example, test/constraint/ test files inside src/constraint/.
• typings/ - TypeScript typing declaration for dependencies. Some of them are downloaded from the TypeStrong community.

Pro-Tip

• When you add a new source file to the project, don't forget to the file to files in tsconfig.json.