customer-data-validators

Validation Scripts for Elevate Security Customer HR/Behaviors Dataset Formats.

Latest Integration Type definitions: type-config.yml (public link). spec.html (HTML version)

Prerequisite

• [ ] Install NodeJS (with npm)

Usage

The CLI is self-documented, just execute and it will output the available commands. Example, to check the src/tests/test_files/2019-10-30-00_58_46.csv file with people/v1 integration type:

./dist/es-cdv-cli.nodejs.js check src/tests/test_files/2019-10-30-00_58_46.csv people/v1  # return code = 3 because there are invalid lines and no valid lines
./dist/es-cdv-cli.nodejs.js check src/tests/test_files/valid.csv people/v1 # return code = 0 because file is valid
echo $? # will see 0

Return codes of the CLI

• 0: file contains valid rows, and no invalid rows (so, only valid or empty rows)
• 1: invalid command
• 2: no valid rows or invalid input file format
• 4: some invalid rows
• 6=2+4: some empty rows and some invalid rows
• 8: unexpected failure

As a CLI:

By cloning the source code

git clone [email protected]:ElevateSecurity/customer-data-validators.git
cd customer-data-validators
npm install
./dist/es-cdv-cli.nodejs.js -h # for help
./dist/es-cdv-cli.nodejs.js check src/tests/test_files/2019-10-30-00_58_46.csv people/v1 # running the command

If you prefer to install a specific version of the package and run it as a binary

npm install @elevate_security/customer-data-validators
npm link
customer-data-validator -h

If you don't want to install the package up-front and just want to execute the command as a one-off:

npx --package @elevate_security/customer-data-validators  customer-data-validator -h

If npx is not installed: npm install -g npx

In a browser application

We're importing the library as an external script, to always get the latest available patch/minor version of the pinned major version (here below, version 1.x.y). This avoids rebuilding the UI everytime a new version of the validators is published.

<script src="https://unpkg.com/@elevate_security/customer-data-validators@^1"></script>

Then the library is available as esCustomerDataValidator in the global object, you can use it like this:

const {CustomerDataValidator} = window.esCustomerDataValidator;
const results = await CustomerDataValidator(fileContent, 'people/v1')

Development

Edit the schema

Update the type-config.yml. Then run npm run convert to update the type-config.json.

Update js files

To rebuild the package, run npm run build:prod

Tests

For cli and functional tests:

npm run test

And for UI tests, run:

npm run test:ui

Publishing the library to NPM

This happens automatically when merging a feature PR to the master branch. After the CirclecI Job finishes on the master branch, CircleCI will create a PR to bump the version number. Make sure you merge this "Bump version" PR promptly to avoid version conflicts.

Public type-config.yml

Since some other projects like the scoring platform need to access the type definitions, CircleCI pushes the distribution files built from this repo to a public S3 bucket es-use1prod-assets (see .circleci/config.yml. Files are accessible from https://docs.elevatesecurity.com/__apps/customer-data-validators/master/config/type-config.yml (master can be replaced by any branch name from this repository and config/type-config.yml can be replaced by any file from the dist folder)