better-tail

Node.js implementation of UNIX tail command using streams. No dependencies :no_entry_sign:.

:book: Table of contents

• :fire: Motivation
• :floppy_disk: Installation
• :beginner: Usage
• :nut_and_bolt: Parameters
  • :dart: Target
  • :speech_balloon: Options
    • bytes
    • follow
    • lines
    • retry
    • sleepInterval
    • encoding
• :scroll: Methods
  • unfollow
• :calendar: Events
  • data
  • error
  • end
• :construction: What’s coming next?
• :beetle: Debugging
• :game_die: Running tests
• :busts_in_silhouette: Contributing
• :1234: Versioning
• :octocat: Authors
• :pray: Acknowledgments

:fire: Motivation

I needed to tail files for a corporate side project, made a few research on what was available, but nothing could 100% match my needs:

• did not find any package allowing to tail with or without following file
• did not find any package allowing to target file’s N last lines
• lots of them extends EventEmitter rather than Stream
• lots of them are unmaintained since years
• majority have unneeded dependencies

At first, I wanted to tweak Luca Grulla’s tail package to add the features I needed. But, boy, it’s written in Coffeescript :man_facepalming:. What a nightmare (at least, for me).

So, I ended up decaffeinating it and rewriting it in pure JS. Then tweaked it as I wanted.

What I came up with was great, working and all, but then I stumbled upon an issue where lines were mixed when file from which I was reading was being appended too fast. This is an open issue of tail.

:thinking: “Why not use streams after all ?” Makes sense.

Let me introduce you to better-tail then !

Note: not that it’s better than others, but other package names are either squatted or unavailable.

:point_down::point_down::point_down:

:floppy_disk: Installation

Node.js version 8 or higher is required.

npm install better-tail --save # or yarn add better-tail

:beginner: Usage

const Tail = require('better-tail')

const tail = new Tail('path-to-file-to-tail')

tail.on('line', function (line) {
  console.log(line)
}).on('error', function (err) {
  console.error(err)
})

:bulb: Good to know

New line at end of target file

If target file ends with a newline, last line and data events will emit an empty string.

This behavior is not to be expected with follow option set to true.

Truncating target file

If target file content is truncated while reading, line and data events will be emitted with following message :

better-tail: file truncated

Reading will then start again with original options.

Example:

new Tail(target, { follow: true, lines: 1 })

// Data is flowing
`Some content
is written
to target file
and suddenly
`

// Data is truncated
`Some content
is written
to tar
`

// Emitted events would be
`better-tail: file truncated`

// Followed by
`to tar`

We only asked for last line (lines: 1), so after truncating, tailing « restarts » and only emits last line of target file.

:nut_and_bolt: Parameters

Only target parameter is required.

:dart: Target

First parameter is the target to tail data from. It can be:

• the string path to target file
• a readable stream of target file
• a file descriptor of target file

const fs = require('fs')

// Tail file from path
new Tail('./some-file')

// Tail file from readable stream
new Tail(fs.createReadStream('./some-file', { encoding: 'utf8' }))

// Tail file from file descriptor
new Tail(fs.openSync('./some-file', 'r'))

:speech_balloon: Options

Second parameter is an object of following options (heavily inspired by UNIX tail command options):

bytes (default: undefined)

Number of bytes to tail. Can be prepended with + to start tailing from given byte.

If this option is set, it supersedes lines option.

// Will return lines in last 42 bytes of target content
new Tail(target, { bytes: 42 })

// Will return lines starting from byte 42 until target content end
new Tail(target, { bytes: '+42' })

follow (default: false)

Keeps polling target file for appended data until you stop following it.

Related:

• sleepInterval option
• unfollow method

const tail = new Tail(target, { follow: true })

tail.on('line', function (line) {
  console.log(line)
}).on('error', function (err) {
  console.error(err)
})

setTimeout(function () {
  tail.unfollow()
}, 5000)

lines (default: 10)

Number of lines to tail. Can be prepended with + to start tailing from given line.

This option is superseded by bytes option.

// Will return last 42 lines of target content
new Tail(target, { bytes: 42 })

// Will return lines starting from line 42 until target content end
new Tail(target, { bytes: '+42' })

retry (default: false)

Can either be a boolean:

• true: keep trying to open target file until it is accessible
• false: emit an error if file is inaccessible

Or an object with following properties:

• interval: wait for given milliseconds between retries
• timeout: abort retrying after given milliseconds
• max: abort retrying after given retries count

If given an object, it means target file opening will be retried upon failure.

This option uses sleepInterval option value by default to wait between retries if interval property is not set.

// Will retry indefinitly, approximately each second (default value of sleepInterval option), until file is accessible
new Tail(target, { retry: true })

// Will wait for approximately 5 seconds between retries
new Tail(target, { retry: { interval: 5000 } })

// Will retry for approximately 5 seconds before aborting
new Tail(target, { retry: { timeout: 5000 } })

// Will retry 3 times before aborting
// Note: first try is not counted, so file access will be tried 4 times
new Tail(target, { retry: { max: 3 } })

// Will retry 5 times or approximately 3 seconds before aborting
new Tail(target, { retry: { timeout: 3000, max: 5, interval: 500 } })

sleepInterval (default: 1000)

Sleep for approximately N milliseconds between target file polls.

This option has no effect if follow option is set to false.

// Will poll target file data approximately each 5 seconds
new Tail(target, { follow: true, sleepInterval: 5000 })

encoding (default: 'utf8')

Set target file's content encoding.

Supported encodings are those supported by Node.js:

• utf8
• utf16le
• latin1
• base64
• hex
• ascii
• binary
• ucs2

:bulb: Seeking help for testing this option

:scroll: Methods

unfollow

Stop following target file. This method will also destroy underlying stream.

This method has no effect if follow option is set to false.

:calendar: Events

line

Decoded lines are emitted through this event.

new Tail(target).on('line', function (line) {
  console.log(line)
})

data

Lines encoded to a Buffer object with given encoding option are emitted through this event.

new Tail(target).on('data', function (chunk) {
  console.log(chunk.toString('utf8'))
})

error

Errors will be emitted through this event. If an error is emitted, underlying stream is destroyed.

Errors can be emitted in following scenarios:

• an invalid option was provided
• an invalid target was provided
• could not guess target type
• failed to access target (or retry error)
• failed to get target content
• failed to get target size
• failed to create read stream to read target content

end

This event is emitted each time target file content end is reached. Therefore, it can fire multiple times if follow option is set to true.

:construction: What’s coming next?

• add support for readable streams not targetting files

:beetle: Debugging

Debugging is built-in and can be triggered in two ways:

• defining the environment variable DEBUG_TAIL (to any value)
• defining the constructor debug option to:
  • true
  • a function that takes a message as it first argument

If DEBUG_TAIL environment variable is defined, but debug option is not set, debugging will default to console.log. Same behavior apply if debug option is set to true.

If debug option is a function, it will be run with a single message argument.

:game_die: Running tests

This package is tested against multiple different scenarios with Mocha and chai (through expect BDD style).

In order to run tests locally, you have to:

• clone this repository
• install development dependencies with npm install (or yarn install)
• run tests with npm test (or yarn test)

Tests are run in bail mode. This means that whenever a test fails, all following tests are aborted.

Debugging tests buffers

Because tests work with buffers, it can be tedious to debug them and understand why they fail.

To make this easier, you can define a DEBUG_BUFFERS environment variable (to any value) in order to enable « buffers debugging mode ». This mode will simply write two files for each tests failing (each file is prefixed by test number x):

• expected result: x_expected.log
• received result: x_received.log

:busts_in_silhouette: Contributing

See CONTRIBUTING.md.

:1234: Versioning

This project uses SemVer for versioning. For the versions available, see the tags on this repository.

:octocat: Authors

• Nicolas Goudry - Initial work - nicolas-goudry

:pray: Acknowledgments

Obviously, Luca Grulla for inspiring me to do this.