Metrics library

Library to send metrics to our metrics service

To allow javascript developers to send metrics in a easy way without worring about how to connect and how to build the metrics in the statsd format, we have developed a library that will create an UDP connection to StatsD automatically and will expose some functions to generate the type of metrics that statsd accepts, also the library has middleware for the developers that use expressjs or hapijs, this middlewares will generate metrics default metrics that will allow us to collect data about the request like, latency, number of request, error, etc.

How to use it

To use the library you just need to follow the following steps Install the library with npm

npm install @condor-labs/metrics

Import the library

const metrics = require('@condor-labs/metrics');

When you import the library, it will automatically create an UDP connection to your localhost in the port 8125

If you want to pass custom parameters when you are instanciating the client you can do the following

metrics.connect({
  host: 'your.metrics.host'
  port: 8125,
  globalTags: {
    job: 'my-job'
    instance: 'my-instance'
  }
});

The previous code will create a new UDP connection that will send the metrics to your.metrics.host at port 8125, also the library will add a couple of labels to all the metrics you send from now on, these labels are job and instance. These labels will help you to differentiate your metrics from another metrics with the same name that could be sent from another service.

Another way you can pass the configurations to the client is through environment variables.

So let's say you are using dotenv in your app, then you just need to add the same variables to the .env file and create the client without params, the client will use the environment variables as its configuration.

So make sure you have these environment variables added to your .env file of your project or some environment file of your system.

env
STATSD_HOST=your.metrics.host
STATSD_PORT=8125
JOB=your-job
INSTANCE=your-instance

That’s all you need to do to have a fully functional metric instance that will send metrics to our service stack.

Now, this metrics instance will give you the following methods to play with:

// HELPERS

metrics.collectSystemInformation(); // this method will generate and send information about the machine (time, memory, cpu, network).
 
metrics.heartbeatSignal(interval); // this method will generate a bit every interval that will help you to know when a service has stopped from working, default to 1000 milliseconds.
 
metrics.restartSignal(); // this method will generate a signal when the service has restarted.

metrics.implementFullMonitoring(); // This method is a wrapper for  metrics.collectSystemInformation(), metrics.heartbeatSignal() [each minute by default] and metrics.restartSignal(). It will send the 3 metrics at the same time.

metrics.asyncTimer(aFunctionToWrap); // this method will help you to wrap an async function to wait until the function has finished its execution and send the time that the function has taken.

metrics.connect(connectionOptions); // this method will create a new UDP connection with the connection options, default to {}.
metrics.closeConnection(); // this method will close the UDP connection.

// METRICS GENERATORS
metrics.increment(metricName); // this method will send a counter to the metrics service.
metrics.gauge(metricName, value); // this method will send a gauge with a value to the metrics service.
metrics.timing(metricName, value); // this method will send a timer with a value to the metrics service.
metrics.set(metricName); // this method will send a set to the metrics service.

What metric types can I generate

As you could imagine, you can generate basically 4 types of metrics Counter, Gauges, Timing and Sets.

A counter is a metric that can only go up, it’s useful to count events.

A gauge is a metric that can go up or down, it’s useful to send values about the current state of something. i.e Memory or CPU consumption.

A timing is a metric that measure the time that some event takes, it’s also know as histogram, it’s useful to measure how much time take a task to be completed, how long takes a request to respond, how much time take an event to be processed.

A set is a metric that count how many unique events happened, it’s useful when you what to know how many unique views do you have in your system.

To know more about metrics type you could go to

• https://github.com/statsd/statsd/blob/master/docs/metric_types.md
• https://prometheus.io/docs/concepts/metric_types/
• https://statsd.readthedocs.io/en/v3.2.1/types.html
• https://www.librato.com/docs/kb/collect/collection_agents/stastd/

How to know what metric type should I use when generating a custom metric

Basically you can follow the following rules:

If you are measuring time always use a timer

• When you say something like:

  • How much time takes to process this …
  • What is the time this request takes …
  • How long take to complete this …

  // how much takes to run a query in millisecond
  metrics.timing('database_query', 1000);

If you are measuring something that reflect the current state of something then use a gauge (goes up and down)

• When you say something like:

  • How many events are we processing right now …
  • What is the current memory consumption …
  • How many opened connection do we have right now …

  // how much memory we are using in Kb
  metrics.gauge('memory', 1024);

  // how many events are we processing right now
  metrics.gauge('current_procesing_events', 100);

If you are just counting events or do you want to determine the frequency at which an event is happening then use a counter (only goes up)

• When you say something like:

  • How many request do we have until now…
  • How many logins do we have until now …
  • How many times have we process an event …
  • What is the rate that some event is happening …

  // Increment the http_request in 1
  metrics.increment('http_request');

  // Increment the logins in 1
  metrics.increment('logins');

If you are counting unique events use a set (this type is not commonly used)

• When you say something like:

  • How many unique user do we have …
  • How many unique events do we process …

  // Count the unique logins by userId
  metrics.set('unique_logins', { userId });

You can find more info about which metric type to use and when to use them here:

• https://www.librato.com/docs/kb/collect/collection_agents/stastd/
• https://thenewstack.io/collecting-metrics-using-statsd-a-standard-for-real-time-monitoring/>

Now let's see how should looks all the code together if we want to send metrics to test site from our example api:

// Import the library
const metrics = require('@condor-labs/metrics');

// Create the instance
metrics.connect({
  host: 'your.metrics.host'
  port: 8125,
  globalTags: {
    job: 'my-job'
    instance: 'my-instance'
  }
});

/// Send one metric of each type
metrics.increment('my_courter');
metrics.gauge('my_gauge', 10);
metrics.timing('my_timing', 10);

But if we pass the config through environment variables, the code will look like the following:

// Import the library
const metrics = require('@condor-labs/metrics');

// Create the instance
metrics.connect();

/// Send one metric of each type
metrics.increment('my_courter');
metrics.gauge('my_gauge', 10);
metrics.timing('my_timing', 10);

With just 5 lines you should be able to send 3 metrics to our metrics service on test site.

Typings for typescript

Basic types have been implemented to allow the library to work with TypeScript.

How to use it

import metrics, { Config } from '@condor-labs/metrics’;

Pass custom parameters when you are instantiating the client:

const settings: Config = {
    host: 'your.metrics.host',
    port: 8125,
    globalTags: {
      job: 'my-job',
      instance: 'my-instance',
    },
  };

  metrics.connect(settings);

If you want to create some utility so that when using the increment it receives parameters. You can do something like this:

import metrics, { Stat, Tags } from '@condor-labs/metrics';

const incrementMetrics = (stat: Stat, tags?: Tags): void => metrics.increment(stat, tags);

How to use the library with express.js

To use the library with express.js you just need to import the metrics library and use the middleware the library provides.

Note: that you can go here and see a functional example.

const express = require('express');
const app = express();

const metrics = require('@condorlabs/metrics');
const metricsMiddleware = require('@condorlabs/metrics/middleware/express');

const { PORT = 3000 } = process.env;

// Uncomment the following code to log errors from the library
metrics.socket.on('error', error => {
  console.error('Error in socket: ', error.message);
});

// Middleware to count the number of request and the duration of the requests
app.use(metricsMiddleware.requestMiddleware(metrics));

// Your routes should be after you setup the requestMiddleware and before the errorMiddleware
app.get('/', (req, res) => res.send('hello world'));

app.use(metricsMiddleware.errorMiddleware(metrics));

app.listen(PORT, () => {
  metrics.heartbeatSignal();
  metrics.restartSignal();
  metrics.collectAppInformation();
  metrics.collectSystemInformation();
  console.log(`API running on port ${PORT}`); //eslint-disable-line
});

process.on('SIGINT', function() {
  console.log('\nCaught interrupt signal'); //eslint-disable-line
  metrics.closeConnection(); // Close statsd connection
  process.exit();
});

How to use the library with hapi.js v16

To use the library with express.js you just need to import the metrics library and use the middleware the library provides.

Note: that you can go here and see a functional example.

const Hapi = require('hapi');

// import the metrics library
const metrics = require('@condorlabs/metrics');
const metricsMiddleware = require('@condorlabs/metrics/middleware/hapi/v16');

metrics.socket.on('error', error => {
  console.error('Error in socket: ', error.message);
});

const server = new Hapi.Server();
server.connection({ port: 3000, host: 'localhost' });

// Use the "setupHapiV16Middleware" function to generate metrics about any request
metricsMiddleware.setup(metrics, server);

server.route({
  method: 'GET',
  path: '/',
  handler: (request, reply) => {
    reply('hello world');
  }
});

async function start() {
  try {
    await server.start();
  } catch (err) {
    console.log(err);
    process.exit(1);
  }
  metrics.heartbeatSignal();
  metrics.restartSignal();
  metrics.collectAppInformation();
  metrics.collectSystemInformation();
  console.log('Server running on', server.info.uri);
}

process.on('SIGINT', function() {
  console.log('\nCaught interrupt signal'); //eslint-disable-line
  metrics.closeConnection(); // Closing statsd connection
  process.exit();
});

start();

How to use the library with hapi.js v17

[To use the library with hapi.js you just need to create and instance of the metrics client and use a middleware the library provides.](To use the library with express.js you just need to import the metrics library and use the middleware the library provides.

Note: that you can go here and see a functional example.)

const Hapi = require('hapi');

// import the metrics library
const metrics = require('@condorlabs/metrics');
const metricsMiddleware = require('@condorlabs/metrics/middleware/hapi/v16');

const server = Hapi.Server({ port: PORT, host: 'localhost' });

metricsMiddleware.setup(metrics, server);

server.route({
  method: 'GET',
  path: '/',
  handler: (request, h) => {
    return 'hello world';
  }
});

async function start() {
  try {
    await server.start();
  } catch (err) {
    console.log(err);
    process.exit(1);
  }
  metrics.heartbeatSignal();
  metrics.restartSignal();
  metrics.collectAppInformation();
  metrics.collectSystemInformation();
  console.log('Server running on', server.info.uri);
}

process.on('SIGINT', function() {
  console.log('\nCaught interrupt signal'); //eslint-disable-line
  metrics.closeConnection(); // Closing statsd connection
  process.exit();
});

start();

How to Publish

Increasing package version

You will need to update the package.json file placed in the root folder.

identify the property version and increase the right number in plus one.

Login in NPM by console.

 npm login
 [Enter username]
 [Enter password]
 [Enter email]

If all is ok the console will show you something like this : Logged in as USERNAME on https://registry.npmjs.org/.

Uploading a new version

 npm publish --access public

Ref: https://docs.npmjs.com/getting-started/publishing-npm-packages

Note: you will need to have a NPM account, if you don't have one create one here: https://www.npmjs.com/signup

Contributors

The original author and current lead maintainer of this module is the @condor-labs development team.

More about Condorlabs Here.

License

MIT