files-by-directory

List files by directory, recursively, using asynchronous iteration.

This can be particularly useful for directory structures with lots of files, or slow files systems, since you can start treating the results straight away, without having to wait for the entire structure to be scanned.

Installation

Prerequisites: Node.js 6+, npm 3+.

npm install --save files-by-directory

API

filesByDirectory(paths: string[], options = {}): AsyncIterator<string[]>

Scan directories recursively, and generate 1 array per directory, containing the file paths.

# Directory structure:
level1
├── level2a
│   ├── level3
│   │   ├── file3a
│   │   └── file3b
│   └── file2a
├── level2b
│   └── file2b
├── file1a
└── file1b

const filesByDirectory = require('files-by-directory');

for await (const files of filesByDirectory(['level1'])) {
  console.log(files);
  console.log('---');
}

[
  'level1/file1a',
  'level1/file1b'
]
---
[
  'level1/level2a/file2a'
]
---
[
  'level1/level2a/level3/file3a',
  'level1/level2a/level3/file3b'
]
---
[
  'level1/level2b/file2b'
]
---

Notes:

• If a path is encountered twice, it is only generated once.
• By default symbolic links are treated as regular files, even though they link to directories.

When set to true, resolves any symbolic link to the directory it's pointing to, while preserving the link's path.

# Directory structure:
level1
├── level2
│   ├── file2a
│   └── file2b
├── file1a
├── link-to-directory -> level2
└── link-to-file -> file1a

for await (const files of filesByDirectory(['level1']/*, { followSymlinks: false }*/} )) {
  console.log(files);
}
// [ 'level1/file1a', 'level1/link-to-directory', 'link-to-file' ]
// [ 'level1/level2/file2a', 'level1/level2/file2b' ]

for await (const files of filesByDirectory(['level1'], { followSymlinks: true })) {
  console.log(files);
}
// [ 'level1/file1a', 'level1/link-to-file' ]
// [ 'level1/level2/file2a', 'level1/level2/file2b' ]
// [ 'level1/link-to-directory/file2a', 'level1/link-to-directory/file2b' ]

When set to true, excludes symbolic links from results:

# Directory structure:
level1
├── level2a
│   ├── file2a
│   └── file2b
├── level2b -> level2a
├── file1a
└── file1b -> file1a

for await (const files of filesByDirectory(['level1']/*, { excludeSymlinks: false }*/} )) {
  console.log(files);
}
// [ 'level1/level2b', 'level1/file1a', 'level1/file1b' ]
// [ 'level1/level2a/file2a', 'level1/level2a/file2b' ]

for await (const files of filesByDirectory(['level1'], { excludeSymlinks: true })) {
  console.log(files);
}
// [ 'level1/file1a', 'level1/file1b' ]
// [ 'level1/level2a/file2a', 'level1/level2a/file2b' ]

When set to true, proceed directories (recursively) before files.

# Directory structure:
level1
├── level2a
│   ├── level3
│   │   ├── file3a
│   │   └── file3b
│   └── file2a
├── level2b
│   └── file2b
├── file1a
└── file1b

for await (const files of filesByDirectory(['level1']/*, { directoriesFirst: false }*/} )) {
  console.log(files);
}
// [ 'level1/file1a', 'level1/file1b' ]
// [ 'level1/level2a/file2a' ]
// [ 'level1/level2a/level3/file3a', 'level1/level2a/level3/file3b' ]
// [ 'level1/level2b/file2b' ]

for await (const files of filesByDirectory(['level1'], { directoriesFirst: true })) {
  console.log(files);
}
// [ 'level1/level2a/level3/file3a', 'level1/level2a/level3/file3b' ]
// [ 'level1/level2a/file2a' ]
// [ 'level1/level2b/file2b' ]
// [ 'level1/file1a', 'level1/file1b' ]

When set to true, includes an entry containing the directory.

# Directory structure:
level1
├── level2
│   ├── file2a
│   └── file2b
├── file1a
└── file1b

for await (const files of filesByDirectory(['level1']/*, { showDirectories: false }*/} )) {
  console.log(files);
}
// [ 'level1/file1a', 'level1/file1b' ]
// [ 'level1/level2/file2a', 'level1/level2/file2b' ]

for await (const files of filesByDirectory(['level1'], { showDirectories: true })) {
  console.log(files);
}
// [ 'level1', 'level1/file1a', 'level1/file1b' ]
// [ 'level1/level2', 'level1/level2/file2a', 'level1/level2/file2b' ]

for await (const [directory, ...files] of filesByDirectory(['level1'], { showDirectories: true })) {
  console.log(directory, files);
}
// level1 [ 'level1/file1a', 'level1/file1b' ]
// level1/level2 [ 'level1/level2/file2a', 'level1/level2/file2b' ]

When set to false, includes empty files lists.

# Directory structure:
level1
├── level2a (only directories)
│   └── level3
│       └── file3a
├── level2b
│   └── (empty)
└── file1a

for await (const files of filesByDirectory(['level1']/*, { skipEmptyDirectories: true }*/} )) {
  console.log(files);
}
// [ 'level1/file1a' ]
// [ 'level1/level2a/level3/file3a' ]

for await (const files of filesByDirectory(['level1'], { skipEmptyDirectories: false } )) {
  console.log(files);
}
// [ 'level1/file1a' ]
// []
// [ 'level1/level2a/level3/file3a' ]
// []

Note: this can be useful when combined with showDirectories option:

const options = { skipEmptyDirectories: false, showDirectories: true };
for await (const [directory, ...files] of filesByDirectory(['level1'], options)) {
  console.log(directory, files);
}
// level1 [ 'level1/file1a' ]
// level1/level2a []
// level1/level2a/level3 [ 'level1/level2a/level3/file3a' ]
// level1/empty-directory []

Asynchronous iteration

Asynchronous iteration using for-await-of syntax requires Node 10+. For older version of NodeJS, either use:

• Babel with @babel/transform-async-generator-functions, or
• Use async/await syntax without for-await-of (NodeJS 8+, see demo-node-8.js)
• Use Promises with a custom invoke function (NodeJS 6+, see demo-node-6.js)

Contributing

Please refer to the guidelines for contributing.

License

Created with npm-package-skeleton.