GELF Transformer (ver. 2)

This project is based on Pino Gelf

GELF transformer is a tool which receives json formatted logs from the stdin and transforms them into GELF format GELF It can also use a custom mapping schema to fill the output log with more data.

Contents

• Installation
• Usage
• Getting Started
• Examples
• GELF
• Log Level Mapping
• Acknowledgements

Installation

npm i -g gelf-transformer

Usage

Pipeline approach

If your application is pushing logs to the standard output then pipe them to gelf transformer.

node your-app.js | gelf-transformer log <options>

Getting Started

command log

gelf-transformer log --help

Switch | Description | Default | Notes ---|---|---|--- -h | Host | 127.0.0.1 | Graylog server host -p | Port | 12201 | Graylog server port -m | Maximum Chunk Size | 1420 | -c | Custom schema | false | You can provide a schema which will define which information from your original logs will be visible in the graylog formatted log -v | Verbose mode | false | Output GELF to console -t | Start sending logs to Graylog | false | It will start to send logs to the defined graylog server

Examples

Custom Fields

Given the log message (formatted as JSON for readability):

{
  "pid":16699,
  "hostname":"han",
  "name":"gelf-test-app",
  "level":30,
  "time":1481840140708,
  "msg":"request completed",
  "customField":"test",
  "res":{"statusCode":304},
  "responseTime":8,
  "req":{
    "method":"GET",
    "headers":{
      "host":"localhost:3000",
      "user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_1) AppleWebKit/602.2.14 (KHTML, like Gecko) Version/10.0.1 Safari/602.2.14"}
    },
  "v":1
}

Given custom schema json file (my_custom_schema.json):

{
  "title": "GELF Schema",
  "type": "object",
  "properties": {
    "_status_code": {
      "type": "integer",
      "source": "res.statusCode"
    },
    "_user_agent": {
      "type": "string",
      "source": "req.headers.user-agent"
    },
    "customField": {
      "type": "string"
    }
  }
}

And the usage:

node server.js | gelf-transformer log -v -c my_custom_schema.json

Gelf Transformer will show the following message to your Graylog server (formatted here as JSON for readability):

{
  "version":"1.1",
  "host":"han",
  "short_message":"request completed",
  "full_message":"request completed",
  "timestamp":1481840140.708,
  "level":6,
  "facility":"gelf-test-app",
  "_status_code":304,
  "_user_agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_1) AppleWebKit/602.2.14 (KHTML, like Gecko) Version/10.0.1 Safari/602.2.14",
  "customField":"test"
}

GELF

Right now automatic mapping of fields is done as follows:

Output GELF | Input log | Notes ---|---|--- version | - | Hardcoded to 1.1 per GELF docs host | hostname | short_message | msg | This message is truncated to 64 characters full_message | msg | msg is not truncated timestamp | time | level | level | Default level codes from Pino are mapped to SysLog levels1 facility | name | deprecated

Log Level Mapping

Default behaviour

By default Gelf Transfomer will log level from a Pino format to syslog format:

Pino Log Level Value | Pino Log Level Name | SysLog Level ---|---|--- 10 | Trace | Debug 20 | Debug | Debug 30 | Info | Info 40 | Warn | Warning 50 | Error | Error 60 | Fatal | Critical

Note: A log messages without a level map to SysLog Critical

Override log level from Schema

TBD

Acknowledgements

The implementation of Pino GELF is based in large part on pino-syslog and gelf-node.