Hansel Segment Population Server

Step 1

Create a directory for your service and npm init to create a new npm project

Step 2

Install the Hansel Segment population server module npm install @hanselio_public/segment-population-server --save

Step 3

Inside your "index.js" file, start the server with the .boot() function. The .boot() function takes a json configuration object which contains mandatory properties and other optional properties that override the Hansel Segment Population Server's defaults.

const Server = require('@hanselio_public/segment-population-server');

Server.boot({
  AWS_REGION: "<AWS_REGION>",
  AWS_S3_BUCKET_NAME: "<AWS_S3_BUCKET_NAME>",
  AWS_CREDENTIALS_PROFILE: "<AWS_CREDENTIALS_PROFILE>",
  AWS_SDK_API_VERSION: "<AWS_SDK_API_VERSION>",
  REDIS_HOST: "<REDIS_HOST>",
  REDIS_PORT: <REDIS_PORT>,
  REDIS_DB: <REDIS_DB>,
  EXPORT_API_HOSTNAME: "<EXPORT_API_HOSTNAME>",
  EXPORT_API_PORT: <EXPORT_API_PORT>,
  EXPORT_API_ENDPOINT: "<EXPORT_API_ENDPOINT>",
  EXPORT_API_AUTHORIZATION_HEADER: "<EXPORT_API_AUTHORIZATION_HEADER>",
});

Properties

| Property | Mandatory | Default | Description | | ------------------------------- | --------- | ------- | ----------------------------------------------------------------------- | | AWS_REGION | Yes | "" | AWS region | | AWS_S3_BUCKET_NAME | Yes | "" | S3 bucket name | | AWS_CREDENTIALS_PROFILE | Yes | "" | AWS credentials | | AWS_SDK_API_VERSION | Yes | "" | AWS SDK version | | REDIS_HOST | Yes | "" | Host name of the redis instance | | REDIS_PORT | Yes | "" | Port of the redis instance | | REDIS_DB | Yes | "" | DB of the redis instance | | EXPORT_API_HOSTNAME | Yes | "" | Smartech segment request API hostname | | EXPORT_API_PORT | Yes | "" | Smartech segment request API port | | EXPORT_API_ENDPOINT | Yes | "" | Smartech segment request API endpoint | | EXPORT_API_AUTHORIZATION_HEADER | Yes | "" | Smartech segment request API authorization header | | APPLICATION_PORT | No | 5000 | Port on which the Segment Population Service runs | | AWS_S3_PRESERVE_SYNCED_FILE | No | true | Preserve the S3 file in the bucket after it has been processed? | | JOB_TTL | No | 5 | Time to live for a job (in minutes) | | JOB_START_DELAY | No | 5 | Delay before the job starts executing (in seconds) | | INMEM_CACHE_TTL | No | 1 | Meta data cache validity (in minutes) | | SYNC_API_INTERVAL | No | 5 | Interval at which the segment sync API is called (in minutes) | | REDIS_SYNC_INTERVAL | No | 360 | Interval after which a new sync request is made (in minutes) | | EXPORT_API_HOLDBACK | No | 30 | Holdback time before which a segment can be synced again (in minutes) |

Step 4

Run your server

forever start index.js

Step 5

APIs

1. /api/export_segment_complete

Incoming API that is used to notify the service that a new segment file is available for processing on S3.

Method: Post

Request Body:

{
    "segment_id":"<segment_id>",
    "filename":"<filename>",
    "request_id":"<request_id>"
}

2. /api/sync

This API is invoked every 5 mins and schedules worker jobs for all segments requested by the Node SDK. This API need not be invoked externally.

Method: Post

For developers

Steps to start

• Clone this repo
• npm install
• npm start
• To run the emulated SMT notification server node tempApp.js

Publish

• Dry Run: npm publish --access public --dry-run

• For Realz: npm publish --access public