analytics-api

A node/express API that provides access to analytics queries in Snowflake DB for flows, checklists, goals, page views, segments, experiences and experiments.

Configuration

The environment for the analytics-api is configured via opscues.

$ opscues config get -a analytics-api -e dev

will show you the current dev config. You could also grab the current prod or staging config as well.

Development Setup

First, install dependencies. You will need a valid NPM_TOKEN in your $PATH to install the private the private opscues package.

$ npm install

Then, run the app in dev mode (almost the same as prod, with error tracking disabled):

$ APPCUES_ENV=dev APPCUES_APP=analytics-api APPCUES_FACET=main AWS_REGION=us-west-2 npm start

or run the app in development mode, with automatically watched changes (that restart the server).

$ AWS_REGION=us-west-2 APPCUES_FACET=local APPCUES_ENV=prod npm run dev

Config variables for this live in nodemon.json.

(Optional) Local Redis

The docker compose file includes redis:

$ docker-compose up -d

This defaults to localhost:6379 but you can simply use a different REDIS_URL if you would like (check package.json).

Local facet on prod expects a redis-server to be running locally.

$ APPCUES_FACET=local APPCUES_ENV=prod npm start

Unit Tests

Unit tests use Jest. Make sure you have it installed. If not, install it by running:

npm install jest --global

To run the unit tests:

$ npm test

or watch your changes

$ npm test -- --watch

Test a single file

Useful to test specific features grouped in a file:

npm test <path/to/file.test.js

E.g.

npm test test/queryBuilder/irToSql.test.js

Test a single test within a file

Useful to focusing on a single case.

npm test test/queryBuilder/irToSql.test.js -- \
  -t=<string with the name of the test>

E.g.

npm test test/queryBuilder/irToSql.test.js -- \
  -t="IR for single user profile in composed conditions: 1 level 'or'"

Watch a single test within a file

Can be used during development to have a faster/simpler feedback loop.

Here's an example to test a single case:

AWS_REGION=us-west-2 APPCUES_APP=analytics-api APPCUES_ENV=test \
  ./node_modules/.bin/jest --watch test/queryBuilder/irToSql.test.js \
  -t "IR for user profile with or/and composition using dimentions only"

Debugging

Check the Debugging doc.

Linting

PR will fail if the linter doesn't pass. In order to check it will you can run:

 npm run lint

If there's any error you can fix them automatically with:

 npm run lint-fix

Live Tests

Live tests are available in our test suite but are not run by default.

The following env vars are used:

• LIVE_TESTS set this to run live tests, unset it to skip
• LIVE_TESTS_URL defaults to http://localhost:3333
• LIVE_TESTS_JWT set to an account 22 JWT (get it with copy(await window.apc_auth.getCurrentCustomToken()) in the web console on Studio)

Usage (V2 Queries)

The V2 query interface, aka the Query Builder, is documented here.

V2 queries are also accessible via websocket, at the /v2/socket path. See lib/queryBuilder/expressSocketHandler.js for more information.

Usage (V1 Queries)

GET /v1/query and POST /v1/query are the legacy query interface.

An API key and secret that is authorized for the analytics-api role must be passed in the Authorization header as HTTP Basic Auth credentials.

The endpoints take the following parameters:

• account_id (required)
• account_api_key (required, must match account_id)
• type (required; one of dashboard_flow_stats, all_flow_stats, or flow_stats)
• flow_id (required only for flow_stats)

It returns a JSON array of row objects.

Example:

$ curl -u key:secret 'http://localhost:3333/v1/query?type=dashboard_flow_stats&account_id=22&account_api_key=xxx'

[
  {
    "flow_id": "-J_XahqCu5eEg4JOjJNO",
    "flow_started": 12,
    "flow_completed": 6
  },
  {
    "flow_id": "-Jo1IiM9ikj05bjzjint",
    "flow_started": 109,
    "flow_completed": 0
  },
  ...
]

Deployment

CircleCI is used for deploying changes to the Analytics API.

Staging

You can deploy changes to staging by clicking the hold before the deploy_staging step, and this can be done on any pushed commit to a branch or PR. Staging does get used by other services such as the staging CRX or studio, so do monitor for errors in Sentry and Datadog

Production

Production deploys happen after a PR has been merged, there's an additional deploy_prod step with a hold before it just like deploy_staging. As with staging, keep an eye on Datadog and Sentry.