dfq-js

Library for querying fixed-width or delimited data files from the CLI or Node.js

Using CLI Commands

#install

> npm i -g dfq

count

count <source>

Counts the number of rows in the given source file

> dfq count sample.txt
1024

sample

sample <source> [--offset n=0] [--length=10]

Returns a sample of rows in the source

> dfq sample sample.txt --length 3
Line 1...
Line 2...
Line 3...

extract

extract fixed [--begin <n=0>] [--end <n=EOL>]

extract delim [--index <n>] [--delimiter <s=,>] [--qualifier <s=">]

Returns a sample of columns in the source

> dfq extract fixed sample.txt --begin 10 --end 15
Column 2 (Line 1)...
Column 2 (Line 2)...
Column 2 (Line 3)...

distinct

distinct fixed <source> [--begin <n=0>] [--end <n=EOL>]

distinct delim <source> [--index <n>] [--delimiter <s=,>] [--qualifier <s=">]

Output a list of distinct values in the given position

> dfq distinct fixed sample.txt --begin 10 --end 15
2
3
5
11

filter

filter fixed [--key <s>] [--begin <n=0>] [--end <n=EOL>]

filter delim [--key <s>] [--index <n>] [--delimiter <s=,>] [--qualifier <s=">]

Output lines where the given key is found within the given range

> dfq filter fixed sample.txt --key xyz --begin 10 --end 15
Line 1...
Line 8...
Line 9...

sum

sum fixed <source> [--begin <n=0>] [--end <n=EOL>]

sum delim <source> [--index <n>] [--delimiter <s=,>] [--qualifier <s=">]

Output the sum of values in the given position

> dfq sum fixed sample.txt --begin 10 --end 15
5178.52

Node.js Functions

Most, if not all, of the above functionality is available directly through Node.js.

import { count, sample, extractDelim, extractFixed, distinctDelim, distinctFixed, filterDelim, filterFixed, sumDelim, sumFixed } from 'dfq/commands'

const length = await count(filename)
const rows = await sample(filename, offset, length)
const extractValues = await extractDelim(filename, index, delimiter, qualifier)
const extractValues2 = await extractFixed(filename, begin, end)
const dinstinctValues = await distinctDelim(filename, index, delimiter, qualifier)
const dinstinctValues2 = await distinctFixed(filename, begin, end)
const matchingRows = await filterDelim(filename, key, index, delimiter, qualifier)
const matchingRows2 = await filterFixed(filename, key, begin, end)
const total = await sumDelim(filename, key, index, delimiter, qualifier)
const total2 = await sumFixed(filename, key, begin, end)