klaw-redux
v1.0.1
Published
File system walker with Readable stream interface, designed for easy use.
Downloads
6
Maintainers
Readme
Node.js - klaw-redux
A redux version of Klaw designed to be more like klaw-sync, and more user-friendly as a result.
Install
npm i --save klaw-redux
Name
klaw
is walk
backwards :p
Sync
If you need the same functionality but synchronous, you can use klaw-sync.
Usage
klaw(directory, [options])
Returns a Readable stream that iterates
through every file and directory starting with dir
as the root. Every read()
or data
event
returns an object with two properties: path
and stats
. path
is the full path of the file and
stats
is an instance of fs.Stats.
directory
: The directory to recursively walk. Typestring
.options
: Readable stream options and the following:queueMethod
(string
, default:'shift'
): Either'shift'
or'pop'
. Onreaddir()
array, call eithershift()
orpop()
.pathSorter
(function
, default:undefined
): Sorting function for Arrays.fs
(object
, default:graceful-fs
): Use this to hook into thefs
methods or to usemock-fs
nodir
(bool
, default:false
): Return only files (excludes all directories)nofile
(bool
, default:true
): Return only directories (excludes all files)filter
(function
, default:undefined
): Function that gets one argument fn({path: '', stats: {}}) and returns true to include or false to exclude the itemdepthLimit
(number
, default:undefined
): The number of times to recurse before stopping. -1 for unlimited.
Streams 1 (push) example:
const klaw = require('klaw')
const items = [] // files, directories, symlinks, etc
klaw('/some/dir')
.on('data', item => items.push(item.path))
.on('end', () => console.dir(items)) // => [ ... array of files]
Streams 2 & 3 (pull) example:
const klaw = require('klaw')
const items = [] // files, directories, symlinks, etc
klaw('/some/dir')
.on('readable', function () {
let item
while ((item = this.read())) {
items.push(item.path)
}
})
.on('end', () => console.dir(items)) // => [ ... array of files]
Filter functions:
const klaw = require('klaw')
const path = require('path')
const filterFunc = function(item) {
const basename = path.basename(item.path) // get the file name
return basename === '.' || basename[0] !== '.' // exclude the file if it's name begins with a dot
}
klaw('/some/dir', { filter: filterFunc })
.on('data', item => {
// only items of none hidden folders will reach here
})
Error Handling
Listen for the error
event.
Example:
const klaw = require('klaw')
klaw('/some/dir')
.on('readable', function () {
let item
while ((item = this.read())) {
// do something with the file
}
})
.on('error', (err, item) => {
console.log(err.message)
console.log(item.path) // the file the error occurred on
})
.on('end', () => console.dir(items)) // => [ ... array of files]
Aggregation / Filtering / Executing Actions (Through Streams)
On many occasions you may want to filter files based upon size, extension, etc. Or you may want to aggregate stats on certain file types. Or maybe you want to perform an action on certain file types.
You should use the module through2
to easily
accomplish this.
Install through2
:
npm i --save through2
Example (skipping directories):
const klaw = require('klaw')
const through2 = require('through2')
const excludeDirFilter = through2.obj(function (item, enc, next) {
if (!item.stats.isDirectory()) this.push(item)
next()
})
const items = [] // files, directories, symlinks, etc
klaw('/some/dir')
.pipe(excludeDirFilter)
.on('data', item => items.push(item.path))
.on('end', () => console.dir(items)) // => [ ... array of files without directories]
Example (totaling size of PNG files):
const klaw = require('klaw')
const path = require('path')
const through2 = require('through2')
let totalPngsInBytes = 0
const aggregatePngSize = through2.obj(function (item, enc, next) {
if (path.extname(item.path) === '.png') {
totalPngsInBytes += item.stats.size
}
this.push(item)
next()
})
klaw('/some/dir')
.pipe(aggregatePngSize)
.on('data', item => items.push(item.path))
.on('end', () => console.dir(totalPngsInBytes)) // => total of all pngs (bytes)
Example (deleting all .tmp files):
const fs = require('fs')
const klaw = require('klaw')
const through2 = require('through2')
const deleteAction = through2.obj(function (item, enc, next) {
this.push(item)
if (path.extname(item.path) === '.tmp') {
item.deleted = true
fs.unlink(item.path, next)
} else {
item.deleted = false
next()
}
})
const deletedFiles = []
klaw('/some/dir')
.pipe(deleteAction)
.on('data', item => {
if (!item.deleted) return
deletedFiles.push(item.path)
})
.on('end', () => console.dir(deletedFiles)) // => all deleted files
You can even chain a bunch of these filters and aggregators together. By using multiple pipes.
Example (using multiple filters / aggregators):
klaw('/some/dir')
.pipe(filterCertainFiles)
.pipe(deleteSomeOtherFiles)
.on('end', () => console.log('all done!'))
Example passing (piping) through errors:
Node.js does not pipe()
errors. This means that the error on one stream, like
klaw
will not pipe through to the next. If you want to do this, do the following:
const klaw = require('klaw')
const through2 = require('through2')
const excludeDirFilter = through2.obj(function (item, enc, next) {
if (!item.stats.isDirectory()) this.push(item)
next()
})
const items = [] // files, directories, symlinks, etc
klaw('/some/dir')
.on('error', err => excludeDirFilter.emit('error', err)) // forward the error on
.pipe(excludeDirFilter)
.on('data', item => items.push(item.path))
.on('end', () => console.dir(items)) // => [ ... array of files without directories]
Searching Strategy
Pass in options for queueMethod
, pathSorter
, and depthLimit
to affect how the file system
is recursively iterated. See the code for more details, it's less than 50 lines :)
License
Original code and concept is credited to JP Richardson It lives at the klaw repo under the MIT license.
MIT
Copyright (c) 2017 Josh Harris