line-apply

A CLI tool to transform a text stream by applying a JS function to each line

Features:

• take the JS function to apply from a file
• the function may return async results
• preview the transformation results with the --diff option

Summary

• Install
• How To
  • Basic
  • Async
  • Diff mode
  • Filter mode
  • Use sub-function
  • Pass additional arguments
• See also

Install

npm i -g line-apply

How To

Basic

cat some_data | line-apply some_transform_function.js > some_data_transformed
# Which can also be written
line-apply some_transform_function.js < cat some_data > some_data_transformed

where some_transform_function.js just needs to export a JS function. This should work both with the ESM export syntax

// some_transform_function.js
export default function (line) {
  if (line.length > 10) {
    return line.toUpperCase()
  } else {
    // returning null or undefined drops the entry
  }
}

or with the CommonJS export syntax

// some_transform_function.js
module.exports = function (line) {
  if (line.length > 10) {
    return line.toUpperCase()
  } else {
    // returning null or undefined drops the entry
  }
}

Async

That function can also be async:

import { getSomeExtraData } from './path/to/get_some_extra_data.js'

// some_async_transform_function.js
export default async function (doc) {
  if (line.length > 10) {
    const id = line.match(/(Q[\d+]) /)[1]
    const data = await getSomeExtraData(id)
    return `${line} ${data}`
  } else {
    // returning null or undefined drops the entry
  }
}

Diff mode

As a way to preview the results of your transformation, you can use the diff mode

cat some_data | line-apply some_transform_function.js --diff

which will display a colored diff of each line before and after transformation.

Filter mode

Use the js function only to filter lines: lines returning true will be let through. No transformation will be applied.

cat some_data | line-apply some_transform_function.js --filter

Use sub-function

Given a function_collection.js file like:

// function_collection.js
export function uppercase (line) {
  return line.toUpperCase()
}

export function lowercase (line) {
  return line.toLowerCase()
}

You can use those subfunction by passing their key as an additional argument

cat some_data | line-apply ./function_collection.js uppercase
cat some_data | line-apply ./function_collection.js lowercase

This should also work with the CommonJS syntax:

// function_collection.cjs
module.exports = {
  uppercase: line => line.toUpperCase(),
  lowercase: line => line.toLowerCase(),
}

Pass additional arguments

Any remaining argument will be passed to the function

# Pass '123' as argument to the exported function
cat some_data | line-apply ./function.js 123
# Pass '123' as argument to the exported sub-function foo
cat some_data | line-apply ./function_collection.js foo 123

See also

• ndjson-apply
• json-apply