cwlogs-writable

Writable stream for AWS CloudWatch Logs, inspired by bunyan-cloudwatch.

• Features
• API Docs
• Quick Start
• Bunyan Example
• Capturing Log Record Stringification Errors
• Picking LogStream Names
• Recovering from Errors
• Custom Handling of InvalidSequenceTokenException AWS Errors
• CWLogsWritable Options
• Change Log
• License

Features

• Uses aws-sdk.
• Can be used anywhere Writable streams are allowed.
• Allows for recovery from AWS errors.
• Creates log groups and streams if they do not exist.
• Filtering of log events by the stream itself.
• Safe stringification of log events.

API Docs

There are two forms of the API docs:

• Normal API docs - Use this if you using cwlogs-writable as-is and not customizing/extending it's functionality.
• Extended API docs - Use this to also view protected methods that you can use to customize/extend cwlogs-writable.

Quick Start

Install the library using NPM into your existing node project:

npm install cwlogs-writable

Create and write to the stream.

var CWLogsWritable = require('cwlogs-writable');

// Make stream name as unique as possible (see "Picking LogStream Names").
var streamName = 'my-log-stream/' + Date.now()
  + '/' + Math.round(Math.random() * 4026531839 + 268435456).toString(16);

var stream = new CWLogsWritable({
  logGroupName: 'my-aws-log-group',
  logStreamName: streamName,

  // Options passed to the AWS.CloudWatchLogs service.
  cloudWatchLogsOptions: {
    // Change the AWS region as needed.
    region: 'us-east-1',

    // Example authenticating using access key.
    accessKeyId: '{AWS-IAM-USER-ACCESS-KEY-ID}',
    secretAccessKey: '{AWS-SECRET-ACCESS-KEY}'
  }
});

stream.write('example-log-message');

Also consider this checklist:

• Are my log stream names unique enough?
• What if a log record stringification fails?
• Should my logging recover from errors or fail fast?

Bunyan Example

var bunyan = require('bunyan');
var CWLogsWritable = require('cwlogs-writable');

// Make stream name as unique as possible (see "Picking LogStream Names").
var streamName = 'my-log-stream/' + Date.now()
  + '/' + Math.round(Math.random() * 4026531839 + 268435456).toString(16);

var logger = bunyan.createLogger({
  name: 'foo',
  streams: [
    {
      level: 'debug',

      // If 'raw' the CloudWatch log event timestamp will
      // be taken from the bunyan JSON (i.e. rec.time).
      type: 'raw',

      stream: new CWLogsWritable({
        logGroupName: 'my-aws-log-group',
        logStreamName: streamName,
        cloudWatchLogsOptions: { /* ... */ }
      })
    }
  ]
});

Picking LogStream Names

In AWS CloudWatch Logs a LogStream represents "a sequence of log events from a single emitter of logs".

The important part is "single emitter", as this implies that log events should not be put into a LogStream concurrently by multiple emitters.

This is enforced by the PutLogEvents API action which requires each call to include a "sequenceToken". That token is changed each time a call is successful, and the new token is used for the next call.

If an emitter provides an incorrect token, the API will respond with an InvalidSequenceTokenException.

To avoid this error, you must pick LogStream names that are unique to the emitter or at least include enough randomness.

// Example generation of LogStream name
var logStreamName = [

  // Environment identifier (e.g. "production")
  process.env.NODE_ENV || 'development',

  // Current UTC date
  new Date().toISOString().substr(0, 10),

  // EC2 instance ID, optionally provided as an env variable
  process.env.EC2_INSTANCE_ID || null,

  // Process ID
  'p' + process.pid,

  // Random hex string (from "10000000" to "ffffffff")
  Math.round(Math.random() * 4026531839 + 268435456).toString(16),

].filter(Boolean).join('/').replace(/[:*]/g, '');

Capturing Log Record Stringification Errors

Before log records are sent to AWS they must be stringified. cwlogs-writable uses safe stringification techniques to handle circular references that would normally cause JSON.stringify to fail.

Other errors thrown during stringification (e.g. one thrown by a property getter) will also be handled if the optional dependency safe-json-stringify is installed.

If it is not installed, cwlogs-writable will catch the error and emit a stringifyError event.

var stream = new CWLogsWritable({ ... });

// Catch errors if safe-json-stringify is not installed
stream.on('stringifyError', function(err, record) {
  console.log('Failed to stringify log entry!', err);

  // You can attempt to manually process the record argument,
  // but be careful as you will may hit the same error.
  //customBadRecordProcessing(record);
});

Recovering from Errors

By default cwlogs-writable will handle the two most common AWS errors, InvalidSequenceTokenException and DataAlreadyAcceptedException, to give your application as much resiliency as possible.

For all other errors, the default behavior of a CWLogsWritable stream is to emit an 'error' event, clear any queued logs, and ignore all further writes to the stream to prevent memory leaks.

To override this behavior you can provide a onError callback that will allow you to recover from these errors.

var CWLogsWritable = require('cwlogs-writable');

function onError(err, logEvents, next) {
  // Use built-in behavior if the error is not
  // from a PutLogEvents action (logEvents will be null).
  if (!logEvents) {
    next(err);
    return;
  }

  // Requeue the log events after a delay,
  // if the queue is small enough.
  if (this.getQueueSize() < 100) {
    setTimeout(function() {
      // Pass the logEvents to the "next" callback
      // so they are added back to the head of the queue.
      next(logEvents);
    }, 2000);
  }

  // Otherwise, log the events to the console
  // and resume streaming.
  else {
    console.error(
      'Failed to send logEvents: ' +
      JSON.stringify(logEvents)
    );

    next();
  }
}

// Make stream name as unique as possible (see "Picking LogStream Names").
var streamName = 'my-log-stream/' + Date.now()
  + '/' + Math.round(Math.random() * 4026531839 + 268435456).toString(16);

var stream = new CWLogsWritable({
  logGroupName: 'my-aws-log-group',
  logStreamName: streamName,
  cloudWatchLogsOptions: { /* ... */ },

  // Pass the onError callback to CWLogsWritable
  onError: onError
});

Custom Handling of InvalidSequenceTokenException AWS Errors

Frequent InvalidSequenceTokenException AWS errors may indicate a problem with the uniqueness of your LogStream name (see Picking LogStream Names).

If you are experiencing throttling on PutLogEvents or DescribeLogStreams actions, you may want to add custom handling of InvalidSequenceTokenException errors.

// Example of changing the logStreamName on
// InvalidSequenceTokenException errors
// to attempt to avoid further collisions.

function getLogStreamName() {
  return 'my-log-stream/' + Date.now()
    + '/' + Math.round(Math.random() * 4026531839 + 268435456).toString(16);
}

function onError(err, logEvents, next) {
  // Change the LogStream name to get a new
  // randomized value, and requeue the log events.
  if (err.code === 'InvalidSequenceTokenException') {
    this.logStreamName = getLogStreamName();
    next(logEvents);
  }

  // Default to built-in behavior.
  else {
    next(err);
  }
}

var stream = new CWLogsWritable({
  logGroupName: 'my-aws-log-group',
  logStreamName: getLogStreamName(),
  cloudWatchLogsOptions: { /* ... */ },

  // Disable the default handling of
  // InvalidSequenceTokenException errors
  // so onError will get them instead.
  retryOnInvalidSequenceToken: false,

  // Pass the onError callback to CWLogsWritable
  onError: onError
});

CWLogsWritable Options

• logGroupName

  Required
  Type: string

  AWS CloudWatch LogGroup name. It will be created if it doesn't exist.

• logStreamName

  Required
  Type: string

  AWS CloudWatch LogStream name. It will be created if it doesn't exist.

• cloudWatchLogsOptions

  Optional
  Type: object
  Default: {}

  Options passed to AWS.CloudWatchLogs service.

• writeInterval

  Optional
  Type: string | number
  Default: "nextTick"

  Amount of wait time after a Writable#_write call to allow batching of log events. Must be a positive number or "nextTick". If "nextTick", process.nextTick is used. If a number, setTimeout is used.

• retryableDelay

  Optional
  Type: string | number
  Default: 150

• retryableMax

  Optional
  Type: number
  Default: 100

  Maximum number of times an AWS error marked as "retryable" will be retried before the error is instead passed to CWLogsWritable#onError.

• maxBatchCount

  Optional
  Type: number
  Default: 10000

  Maximum number of log events allowed in a single PutLogEvents API call.

• maxBatchSize

  Optional
  Type: number
  Default: 1048576

  Maximum number of bytes allowed in a single PutLogEvents API call.

• ignoreDataAlreadyAcceptedException

  Optional
  Type: boolean
  Default: true

  Ignore DataAlreadyAcceptedException errors. This will bypass CWLogsWritable#onError. See cwlogs-writable/issues/10.

• retryOnInvalidSequenceToken

  Optional
  Type: boolean
  Default: true

  Retry on InvalidSequenceTokenException errors. This will bypass CWLogsWritable#onError. See cwlogs-writable/issues/12.

• onError

  Optional
  Type: function

  Called when an AWS error is encountered. Overwrites CWLogsWritable#onError method.

• filterWrite

  Optional
  Type: function

  Filter writes to CWLogsWritable. Overwrites CWLogsWritable#filterWrite method.

• objectMode

  Optional
  Type: boolean
  Default: true

  Passed to the Writable constructor. See https://nodejs.org/api/stream.html#stream_object_mode.

Change Log

See CHANGELOG.md

License

The MIT License (MIT)

Copyright (c) 2017 Andre Mekkawi <[email protected]>

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.