Library to verify the health check of the APIs in @condor-labs

To ensure that developers handle a standard on the health check of their APIs, we have developed a library that will verify compliance with the response standard of the aPIs in @condor-labs, this library works as a middleware that will allow us to verify that it complies with company guidelines.

Compatibility

The minimum supported version of Node.js is v8.

How to use it

To use the library you just need to follow the following steps Install the library with npm

npm i -S mongoose@">=6.11.4 <8.0.0"
npm install @condor-labs/health-middleware

Import the library

const health = require("@condor-labs/health-middleware");
// OR
const {
  healthMonitor,
  statusType,
  dependencyServices,
} = require("@condor-labs/health-middleware");

The main method is healthMonitor, you can use it as default to show basic stats like cpu or memory.

Example

const express = require("express");
const { healthMonitor } = require("@condor-labs/health-middleware");
var app = express();
//only need app as parameter
healthMonitor(app);
app.listen(process.env.PORT || 3000);

As default the route is /health, if you consume you get a response like

Example output

{
  "status": "fail",
  "version": "1.0",
  "service": "health",
  "description": "",
  "instance": "Admins-Mac-mini.local",
  "checks": {
    "memory:utilization": [
      {
        "componentType": "system",
        "status": "fail",
        "observedValue": 96.9440221786499,
        "observedUnit": "percent",
        "time": "2021-09-14T17:08:48.477Z",
        "componentId": "memory0"
      }
    ],
    "cpu:utilization": [
      {
        "componentType": "system",
        "status": "pass",
        "observedValue": 13.860444612381132,
        "observedUnit": "percent",
        "time": "2021-09-14T17:08:48.478Z",
        "componentId": "cpu0"
      }
    ],
    "network:availability": [
      {
        "componentId": "network0",
        "componentType": "system",
        "status": "pass",
        "observedValue": true,
        "observedUnit": "availability",
        "time": "2021-09-14T08:48:49.105Z"
      }
    ]
  }
}

Modify healthMonitor parameters

As the health middleware knows you likely need customize your checks, you can use the following vars to change the behavior of the Health Middleware.

Some of the following vars are just to give a useful explaination about the service or application such as service and description.

const healthConfig = {
  service: "custom service",
  description: "my project",
  checks: [],
  dependencies: [],
  urls: ["https://www.google.com", "https://www.opendns.com"],
  path: "/custom",
  version: "1.0",
  alias: "machine0",
};

//use in the monitor
healthMonitor(app, healthConfig);

| param | description | required | | ------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------- | | service | it's a unique identifier of the service, in the application scope | true | | description | It is a human-friendly description of the service. | true | | checks | Array of objects that provides detailed health statuses of additional downstream systems and endpoints which can affect the overall health of the main API. | false | | dependencies | Array of objects that provides detailed information about connection to databases, it works like checks but whitout callbacks, the connection is made from the connection parameters entered. | false | | urls | Urls for network availability test, the field is not required: default: "https://www.google.com", "https://www.opendns.com") | false | | path | its a route where your health will live. default: /health | false | | version | Public version of the service (default: 1.0). | false | | alias | It is a short and human-friendly name for the instances | false |

Add a custom check

When you need to add a custom check for example the one from your database, you can use the prepareCheck function to do the job.

{ prepareCheck } = require("@condor-labs/health-middleware");
const checks = [prepareCheck("mongo", dbConnectExample)];

Or use a object to define the params

const checks = [{
      componentName: "oracle", //Object
      componentType: statusType.DATASTORE,
      callback: dbConnectExample,
      cacheTTL: cacheExpirationTime, //optional(by default cacheTTL will be 10), time in seconds
    },
  ],

Add a dependency

A dependency is a connection with a database which can affect the overall health of the main API. When you need to add a dependency for example the one from your Redis database, use an object to define the connection params. It works like checks but whitout callbacks, the connection is verified from the connection params entered.

const dependencies = [{
      service: dependencyServices.REDIS, // You can use REDIS, MONGODB, MYSQL or ELASTICSEARCH
      componentName: 'myredis',
      connection: {
        prefix: 'prefix', //optional(by default prefix will be null)
        host: '127.0.0.1',
        port: 6379, //optional(by default port will be 6379)
        password: 'password', //optional(by default password will be null)
      },
    },
    {
      service: dependencyServices.MONGODB,
      componentName: 'MyMongoDB',
      connection: {
        host: 'clusterN-shard-00-00.3wetr.mongodb.net',
        port: 27017, //optional(by default port will be 27017)
        database: 'myDatabase',
        user: 'user',
        password: 'password',
        replicaSet: 'atlas-xu7xxx-shard-0', //optional
        ssl: true,
        authSource: 'admin',
      },
    },
    {
      service: dependencyServices.MYSQL,
      componentName: 'Mysql',
      connection: {
        host: 'localhost',
        port: 3306, //optional(by default port will be 3306)
        user: 'root',
        password: 'example',
        database: 'sample', //optional
        multipleStatements: false, //optional
      },
    },
    {
      service: dependencyServices.ELASTICSEARCH,
      componentName: 'MyElasticSearch',
      connection: {
        protocol: '', // optional(null, '', 'http', 'https')
        user: '', // optional
        password: '', // optional
        host: 'localhost',
        port: 9200,
        ssl: {}, //optional
      },
    },
  ],

Example

const {
  healthMonitor,
  getStatus,
  statusType,
  dependencyServices,
} = require("@condor-labs/health-middleware");

const yourDBConnectTesting = () => {
  return true; //return a boolean
};
const newCheck = getStatus(yourDBConnectTesting, statusType.DATASTORE);
const healthConfig = {
  //only need modify params that you need it
  service: "service random",
  description: "my service with some random check",
  dependencies: [
    {
      service: dependencyServices.REDIS,
      componentName: "MyRedis",
      connection: {
        prefix: "demo",
        host: "127.0.0.1",
        port: 6379,
        password: "password",
      },
    },
    {
      service: dependencyServices.MONGODB,
      componentName: "MyMongoDB",
      connection: {
        host: "clusterN-shard-00-00.3wetr.mongodb.net",
        port: 27017,
        database: "myDatabase",
        user: "user",
        password: "password",
        replicaSet: "atlas-xu7xxx-shard-0",
        ssl: true,
        authSource: "admin",
      },
    },
    {
      service: dependencyServices.MYSQL,
      componentName: "Mysql",
      connection: {
        host: "localhost",
        port: 3306,
        user: "root",
        password: "password",
        database: "test",
        multipleStatements: false,
      },
    },
    {
      service: dependencyServices.ELASTICSEARCH,
      componentName: "MyElasticSearch",
      connection: {
        protocol: "",
        user: "",
        password: "",
        host: "localhost",
        port: 9200,
      },
    },
  ],
  checks: [
    {
      componentName: "oracle", //Object
      componentType: statusType.DATASTORE,
      callback: dbConnectExample,
      cacheTTL: 20, //20 sec in cache, set in 0 if you don't want have cache
    },
  ],
};
healthMonitor(app, healthConfig);

You will get something like this

{
  "status": "fail",
  "version": "1.1",
  "releaseTag": "1.1.0",
  "service": "custom service",
  "description": "",
  "instance": "Machine0|Admins-Mac-mini.local",
  "checks": {
    "memory:utilization": [
      {
        "componentId": "memory0",
        "componentType": "system",
        "status": "fail",
        "observedValue": 99.59919452667236,
        "observedUnit": "percent",
        "time": "2021-09-15T16:06:40.252Z"
      }
    ],
    "cpu:utilization": [
      {
        "componentId": "cpu0",
        "componentType": "system",
        "status": "pass",
        "observedValue": 14.022481829750083,
        "observedUnit": "percent",
        "time": "2021-09-15T16:06:40.255Z"
      }
    ],
    "network:availability": [
      {
        "componentId": "network0",
        "componentType": "system",
        "status": "pass",
        "observedValue": true,
        "observedUnit": "availability",
        "time": "2021-09-15T16:06:42.415Z"
      }
    ],
    "myredis:responseTime": [
      {
        "componentId": "MyRedis0",
        "componentType": "datastore",
        "observedValue": 148.39368800073862,
        "observedUnit": "ms",
        "status": "pass",
        "time": "2021-09-15T16:06:42.419Z",
        "hostname": "127.0.0.1",
        "resourceVersion": "6.2.7"
      }
    ],
    "mymongodb:responseTime": [
      {
        "componentId": "MyMongoDB0",
        "componentType": "datastore",
        "observedValue": 0.45793600007891655,
        "observedUnit": "ms",
        "status": "pass",
        "time": "2021-09-15T16:06:42.421Z",
        "hostname": "localhost",
        "resourceVersion": "4.1.13"
      }
    ],
    "mysql:responseTime": [
      {
        "componentId": "Mysql0",
        "componentType": "datastore",
        "observedValue": 220.51974400132895,
        "observedUnit": "ms",
        "status": "pass",
        "time": "2021-09-15T16:06:42.422Z",
        "hostname": "localhost",
        "resourceVersion": "5.7.38"
      }
    ],
    "myelasticsearch:responseTime": [
      {
        "componentId": "MyElasticSearch0",
        "componentType": "datastore",
        "observedValue": 56.33939500153065,
        "observedUnit": "ms",
        "status": "pass",
        "time": "2021-09-15T16:06:42.424Z",
        "hostname": "localhost",
        "resourceVersion": "7.4.0"
      }
    ],
    "oracle:responseTime": [
      {
        "componentId": "oracle0",
        "componentType": "datastore",
        "observedValue": 0.03558000177145004,
        "observedUnit": "ms",
        "status": "pass",
        "time": "2021-09-15T16:06:42.430Z"
      }
    ]
  }
}

Reporting Metrics out of the application for APIs

If you need to report the status of your API to a external service such as a Apps Inventory repository, you must provide a new param into the healthConfig object, this gives the ability to report the health of your service. This internally will check the status of your application and then, report the metric to the external endpoint.

Note: This functionality requires an external API to receive the reports of the service. Also, this feature will send request every 30secs by default to the external service by instance running.

to Add this new functionality to your API add to the config the following object:

  {
    ...defaultConfig,
    "webhook": {
      "enabled": true,
      "url": "http://localhost:3000/dev/apps/test_api/sections/health",
      "auth": "39KMqOATRgZBMZxEqsqk0cTQK",
      "intervalMS": 10000,
    },
  }

Reporting Metrics out of the application for NON-APIs

If your applications its not a API and you also need to report health statuses, you must use the following Implementation for workers:

Example

const { HealthMonitorPush } = require("../library/index");

const healthConfig = {
  service: "",
  checks: [],
  dependencies: [],
  description: "",
  urls: [],
  path: "",
  version: "",
  aliasMachine: "",
  timeoutMS: 3000,
  debug: true,
  webhook: {
    enabled: true,
    url: "http://localhost:3000/dev/apps/test_api/sections/health",
    auth: "39KMqOATRgZBMZxEqsqk0cTQK",
    intervalMS: 10000,
  },
};

HealthMonitorPush(healthConfig);

Notice in this case you dont need to send the Express application to the health middleware since we assume you're not working in a API. This feature is useful for workers and cron-tasks in NodeJS

Modify HealthMonitorPush parameters

const healthConfig = {
  webhook: {
    enabled: true,
    url: "http://localhost/apps/test_api/sections/health",
    auth: "39KMqOATRgZBMZxEqsqk0cTQK",
    intervalMS: 10000,
  },
};

| param | description | required | | ------------------ | ------------------------------------------------------------------------------------------------------------------------------- | -------- | | webhook.enabled | This value indicates that the sending of health to the apps inventory api will be activated. | true | | webhook.url | URL for sending health to the api apps inventory. | true | | webhook.auth | Auth token for sending health api apps inventory. | true | | webhook.intervalMS | Interval of how often the health will be sent to the api apps inventory. Default value: 30000 \|\| min: 10000 \|\| max: 60000 | false |

Typings for typescript

Basic types have been implemented to allow the library to work with TypeScript.

How to use it

Import the library

import healthCheck from "@condor-labs/health-middleware";
// OR
import {
  healthMonitor,
  statusType,
  dependencyServices,
} from "@condor-labs/health-middleware";

Modify healthMonitor parameters

import { HealthConfig } from "@condor-labs/health-middleware";

const healthConfig: HealthConfig = {
  service: "custom service",
  description: "my project",
  checks: [],
  dependencies: [],
  urls: ["https://www.google.com", "https://www.opendns.com"],
  path: "/custom",
  version: "1.0",
  aliasMachine: "machine0",
};

//use in the monitor
healthMonitor(app, healthConfig);

The following parameter is an example of how you can customize the limit for memory usage and CPU usage of the system for the fail and warn state.

const healthConfig = {
  ...configs
  rules: {
    memory:{
        warn: 85,
        fail: 95,
    },
    cpu:{
        warn: 70,
        fail: 80,
    },
  }
};

| param | description | required | | ----------- | --------------------------------------------------------------- | -------- | | memory | Object containing the limit values for the warn and fail states | false | | memory.warn | Number | false | | memory.fail | Number | false | | cpu | Object containing the limit values for the warn and fail states | false | | cpu.warn | Number | false | | cpu.fail | Number | false |

Example how to use for Express

const express = require("express");
const { healthMonitor } = require("../library");
const port = process.env.PORT || 3000;
var app = express();

const healthConfig = {
  service: "custom service",
  description: "",
  urls: ["https://www.google.com"],
  path: "/health",
  version: "1.0",
  aliasMachine: "",
  rules: {
    memory: {
      warn: 80,
      fail: 90,
    },
    cpu: {
      warn: 8,
      fail: 9,
    },
  },
};

healthMonitor(app, healthConfig);

app.get("/", (req, res) => {
  res.send("Hello Express");
});

app.listen(port, () => {
  console.log(`Server on port ${port}`);
});

Example how to use for Hapi

This library supports old and new versions of Api using:

• healthMonitorHapi (v16 or less)
• healthMonitorHapiv17 (v17 or higher, currently v22)

const Hapi = require("hapi");
const { healthMonitorHapi } = require("../library");

const server = new Hapi.Server();

const serverConfig = {
  host: "localhost",
  port: 3000,
  routes: {
    cors: {
      origin: ["*"],
      exposedHeaders: ["x-total-count"],
      credentials: true,
    },
    timeout: {
      server: 90000,
    },
    validate: {
      options: {
        abortEarly: false,
      },
    },
  },
};

const healthConfig = {
  service: "custom service",
  description: "",
  checks: [],
  dependencies: [],
  urls: "https://www.google.com",
  path: "/health",
  version: "1.0",
  aliasMachine: "",
  rules: {
    memory: {
      warn: 80,
      fail: 90,
    },
    cpu: {
      warn: 85,
      fail: 100,
    },
  },
};

server.connection(serverConfig);

healthMonitorHapi(server, healthConfig); // or healthMonitorHapiv17(...)

server.route({
  method: ["GET", "POST"],
  path: "/",
  handler: (request, reply) => {
    reply("Hello World!");
  },
});

server.start();
console.log("Server running on %s", server.info.uri);

Error Codes

The following are the health library error codes.

| CODE | Description | | --------------------------------- | ------------------------------------------------------------------------------- | | ERROR_PUSHING_HEALTH_TO_INVENTORY | Error response when sending health to inventory. | | ERROR_PUT_HEALTH_TO_INVENTORY | Error response when put health to inventory. | | CONFIGURATION_REQUIRED | Configuration is required to be able to publish the status of your application. | | ENABLED_REQUIRED | Enabled is required in true to be able to publish the health of your app. | | APP_REQUIRED | App is required. | | CALLBACK_EXPECTED | A callback function was expected for this check. |

Success Codes

The following are the health library success codes.

| CODE | Description | | ------------------------------- | ----------------------------------------------------- | | SUCCESS_RESPONSE_FROM_INVENTORY | Successful response when sending health to inventory. |

Notes

releaseTag in response json will be used to indicate the release under which the healt-check is executed.

The release tag only works when the RELEASE_TAG variable is set as environment var in the machine where the process is running.

How to Publish

Increasing package version

You will need to update the package.json file placed in the root folder.

identify the property version and increase the right number in plus one.

Login in NPM by console.

 npm login
 [Enter username]
 [Enter password]
 [Enter email]

If all is ok the console will show you something like this : Logged in as USERNAME on https://registry.npmjs.org/.

Uploading a new version

 npm publish --access public

Ref: https://docs.npmjs.com/getting-started/publishing-npm-packages

Note: you will need to have a NPM account, if you don't have one create one here: https://www.npmjs.com/signup

Contributors

The original author and current lead maintainer of this module is the @condor-labs development team.

More about Condorlabs Here.

License

MIT