@bluelovers/fast-glob

v3.0.4

Published

3 years ago

old typescript api for fast-glob

Downloads

7,428

0High
0Medium
0Low

bluelovers

bluebird fast fast-glob glob implementation patterns promise

fast-glob

It's a very fast and efficient glob library for Node.js.

This package provides methods for traversing the file system and returning pathnames that matched a defined set of a specified pattern according to the rules used by the Unix Bash shell with some simplifications, meanwhile results are returned in arbitrary order. Quick, simple, effective.

this fork

fork from https://github.com/mrmlnc/fast-glob

export async, default
allow old typescript api async<T>, sync<T>
wrap to bluebird (Optional)

usage

base

import FastGlob, { Options, EntryItem } from '@bluelovers/fast-glob';

bluebird

import FastGlob, { Options, EntryItem } from '@bluelovers/fast-glob/bluebird';

use other Bluebird like or any object with promisify method

const promisify = require('@bluelovers/fast-glob/util').promisify;
const Bluebird = require("bluebird");

const FastGlob = promisify(require('@bluelovers/fast-glob'));

// use other Bluebird like or any object with promisify method
const FastGlob = promisify(require('@bluelovers/fast-glob'), Bluebird);

const my_promisify = (fn) => {
 return function (...argv)
 {
  return Promise.resolve().then(function ()
  {
   return fn(...argv)
  })
 }
}

const FastGlob = promisify(require('@bluelovers/fast-glob'), my_promisify);

const FastGlob = promisify(require('@bluelovers/fast-glob'), {
    promisify: my_promisify
});

Highlights

Fast. Probably the fastest.
Supports multiple and negative patterns.
Synchronous, Promise and Stream API.
Object mode. Can return more than just strings.
Error-tolerant.

Donation

Do you like this project? Support it by donating, creating an issue or pull request.

Old and modern mode

This package works in two modes, depending on the environment in which it is used.

Old mode. Node.js below 10.10 or when the stats option is enabled.
Modern mode. Node.js 10.10+ and the stats option is disabled.

The modern mode is faster. Learn more about the internal mechanism.

Pattern syntax

:warning: Always use forward-slashes in glob expressions (patterns and ignore option). Use backslashes for escaping characters.

There is more than one form of syntax: basic and advanced. Below is a brief overview of the supported features. Also pay attention to our FAQ.

:book: This package uses a micromatch as a library for pattern matching.

Basic syntax

An asterisk (*) — matches everything except slashes (path separators), hidden files (names starting with .).
A double star or globstar (**) — matches zero or more directories.
Question mark (?) – matches any single character except slashes (path separators).
Sequence ([seq]) — matches any character in sequence.

:book: A few additional words about the basic matching behavior.

Some examples:

src/**/*.js — matches all files in the src directory (any level of nesting) that have the .js extension.
src/*.?? — matches all files in the src directory (only first level of nesting) that have a two-character extension.
file-[01].js — matches files: file-0.js, file-1.js.

Advanced syntax

Escapes characters (\\) — matching special characters ($^*+?()[]) as literals.
POSIX character classes ([[:digit:]]).
Extended globs (?(pattern-list)).
Bash style brace expansions ({}).
Regexp character classes ([1-5]).
Regex groups ((a|b)).

:book: A few additional words about the advanced matching behavior.

Some examples:

src/**/*.{css,scss) — matches all files in the src directory (any level of nesting) that have the .css or .scss extension.
file-[[:digit:]].js — matches files: file-0.js, file-1.js, …, file-9.js.
file-{1..3}.js — matches files: file-1.js, file-2.js, file-3.js.
file-(1|2) — matches files: file-1.js, file-2.js.

Installation

npm install fast-glob

API

Asynchronous

fg(patterns, [options])

Returns a Promise with an array of matching entries.

const fg = require('fast-glob');

const entries = await fg(['.editorconfig', '**/index.js'], { dot: true });

// ['.editorconfig', 'services/index.js']

Synchronous

fg.sync(patterns, [options])

Returns an array of matching entries.

const fg = require('fast-glob');

const entries = fg.sync(['.editorconfig', '**/index.js'], { dot: true });

// ['.editorconfig', 'services/index.js']

Stream

fg.stream(patterns, [options])

Returns a ReadableStream when the data event will be emitted with matching entry.

const fg = require('fast-glob');

const stream = fg.sync(['.editorconfig', '**/index.js'], { dot: true });

for await (const entry of stream) {
	// .editorconfig
	// services/index.js
}

patterns

Required: true
Type: string | string[]

Any correct pattern(s).

:1234: Pattern syntax
:warning: This package does not respect the order of patterns. First, all the negative patterns are applied, and only then the positive patterns. If you want to get a certain order of records, use sorting or split calls.

[options]

Required: false
Type: Options

See Options section.

Options

Common options

concurrency

Type: number
Default: os.cpus().length

Specifies the maximum number of concurrent requests from a reader to read directories.

:book: The higher the number, the higher the performance and load on the file system. If you want to read in quiet mode, set the value to a comfortable number or 1.

cwd

Type: string
Default: process.cwd()

The current working directory in which to search.

deep

Type: number
Default: Infinity

Specifies the maximum depth of a read directory relative to the start directory.

For example, you have the following tree:

dir/
└── one/            // 1
    └── two/        // 2
        └── file.js // 3

// With base directory
fg.sync('dir/**', { onlyFiles: false, deep: 1 }); // ['dir/one']
fg.sync('dir/**', { onlyFiles: false, deep: 2 }); // ['dir/one', 'dir/one/two']

// With cwd option
fg.sync('**', { onlyFiles: false, cwd: 'dir', deep: 1 }); // ['one']
fg.sync('**', { onlyFiles: false, cwd: 'dir', deep: 2 }); // ['one', 'one/two']

:book: If you specify a pattern with some base directory, this directory will not participate in the calculation of the depth of the found directories. Think of it as a cwd option.

followSymbolicLinks

Type: boolean
Default: true

Indicates whether to traverse descendants of symbolic link directories.

:book: If the stats option is specified, the information about the symbolic link (fs.lstat) will be replaced with information about the entry (fs.stat) behind it.

fs

Type: FileSystemAdapter
Default: fs.*

Custom implementation of methods for working with the file system.

export interface FileSystemAdapter {
    lstat?: typeof fs.lstat;
    stat?: typeof fs.stat;
    lstatSync?: typeof fs.lstatSync;
    statSync?: typeof fs.statSync;
    readdir?: typeof fs.readdir;
    readdirSync?: typeof fs.readdirSync;
}

ignore

Type: string[]
Default: []

An array of glob patterns to exclude matches. This is an alternative way to use negative patterns.

dir/
├── package-lock.json
└── package.json

fg.sync(['*.json', '!package-lock.json']);            // ['package.json']
fg.sync('*.json', { ignore: ['package-lock.json'] }); // ['package.json']

suppressErrors

Type: boolean
Default: false

By default this package suppress only ENOENT errors. Set to true to suppress any error.

:book: Can be useful when the directory has entries with a special level of access.

throwErrorOnBrokenSymbolicLink

Type: boolean
Default: false

Throw an error when symbolic link is broken if true or safely return lstat call if false.

:book: This option has no effect on errors when reading the symbolic link directory.

Output control

absolute

Type: boolean
Default: false

Return the absolute path for entries.

fg.sync('*.js', { absolute: false }); // ['index.js']
fg.sync('*.js', { absolute: true });  // ['/home/user/index.js']

:book: This option is required if you want to use negative patterns with absolute path, for example, !${__dirname}/*.js.

markDirectories

Type: boolean
Default: false

Mark the directory path with the final slash.

fs.sync('*', { onlyFiles: false, markDirectories: false }); // ['index.js', 'controllers']
fs.sync('*', { onlyFiles: false, markDirectories: true });  // ['index.js', 'controllers/']

objectMode

Type: boolean
Default: false

Returns objects (instead of strings) describing entries.

fg.sync('*', { objectMode: false }); // ['src/index.js']
fg.sync('*', { objectMode: true });  // [{ name: 'index.js', path: 'src/index.js', dirent: <fs.Dirent> }]

The object has the following fields:

name (string) — the last part of the path (basename)
path (string) — full path relative to the pattern base directory
dirent (fs.Dirent) — instance of fs.Direct

:book: An object is an internal representation of entry, so getting it does not affect performance.

onlyDirectories

Type: boolean
Default: false

Return only directories.

fg.sync('*', { onlyDirectories: false }); // ['index.js', 'src']
fg.sync('*', { onlyDirectories: true });  // ['src']

:book: If true, the onlyFiles option is automatically false.

onlyFiles

Type: boolean
Default: false

Return only files.

fg.sync('*', { onlyFiles: false }); // ['index.js', 'src']
fg.sync('*', { onlyFiles: true });  // ['index.js']

stats

Type: boolean
Default: false

Enables an object mode with an additional field:

stats (fs.Stats) — instance of fs.Stats

fg.sync('*', { stats: false }); // ['src/index.js']
fg.sync('*', { stats: true });  // [{ name: 'index.js', path: 'src/index.js', dirent: <fs.Dirent>, stats: <fs.Stats> }]

:book: Returns fs.stat instead of fs.lstat for symbolic links when the followSymbolicLinks option is specified.
:warning: Unlike object mode this mode requires additional calls to the file system. On average, this mode is slower at least twice. See old and modern mode for more details.

unique

Type: boolean
Default: true

Ensures that the returned entries are unique.

fg.sync(['*.json', 'package.json'], { unique: false }); // ['package.json', 'package.json']
fg.sync(['*.json', 'package.json'], { unique: true });  // ['package.json']

If true and similar entries are found, the result is the first found.

Matching control

braceExpansion

Type: boolean
Default: true

Enables Bash-like brace expansion.

:1234: Syntax description or more detailed description.

dir/
├── abd
├── acd
└── a{b,c}d

fg.sync('a{b,c}d', { braceExpansion: false }); // ['a{b,c}d']
fg.sync('a{b,c}d', { braceExpansion: true });  // ['abd', 'acd']

caseSensitiveMatch

Type: boolean
Default: true

Enables a case-sensitive mode for matching files.

dir/
├── file.txt
└── File.txt

fg.sync('file.txt', { caseSensitiveMatch: false }); // ['file.txt', 'File.txt']
fg.sync('file.txt', { caseSensitiveMatch: true });  // ['file.txt']

dot

Type: boolean
Default: false

Allow patterns to match entries that begin with a period (.).

:book: Note that an explicit dot in a portion of the pattern will always match dot files.

dir/
├── .editorconfig
└── package.json

fg.sync('file.txt', { dot: false }); // ['package.json']
fg.sync('file.txt', { dot: true });  // ['.editorconfig', 'package.json']

extglob

Type: boolean
Default: true

Enables Bash-like extglob functionality.

:1234: Syntax description.

dir/
├── README.md
└── package.json

fg.sync('*.+(json|md)', { extglob: false }); // []
fg.sync('*.+(json|md)', { extglob: true });  // ['README.md', 'package.json']

globstar

Type: boolean
Default: true

Enables recursively repeats a pattern containing **. If false, ** behaves exactly like *.

dir/
└── a
    └── b

fg.sync('**', { onlyFiles: false, globstar: false }); // ['a']
fg.sync('**', { onlyFiles: false, globstar: true });  // ['a', 'a/b']

baseNameMatch

Type: boolean
Default: false

If set to true, then patterns without slashes will be matched against the basename of the path if it contains slashes.

dir/
└── one/
    └── file.md

fg.sync('*.md', { baseNameMatch: false }); // []
fg.sync('*.md', { baseNameMatch: true });  // ['one/file.md']

FAQ

How to write patterns on Windows?

Always use forward-slashes in glob expressions (patterns and ignore option). Use backslashes for escaping characters. With the cwd option use a convenient format.

Bad

[
	'directory\\*',
	path.join(process.cwd(), '**')
]

Good

[
	'directory/*',
	path.join(process.cwd(), '**').replace(/\\/g, '/')
]

:book: Use the normalize-path or the unixify package to convert Windows-style path to a Unix-style path.

Read more about matching with backslashes.

Why are parentheses match wrong?

dir/
└── (special-*file).txt

fg.sync(['(special-*file).txt']) // []

Refers to Bash. You need to escape special characters:

fg.sync(['\\(special-*file\\).txt']) // ['(special-*file).txt']

Read more about matching special characters as literals.

How to exclude directory from reading?

You can use a negative pattern like this: !**/node_modules or !**/node_modules/**. Also you can use ignore option. Just look at the example below.

first/
├── file.md
└── second/
    └── file.txt

If you don't want to read the second directory, you must write the following pattern: !**/second or !**/second/**.

fg.sync(['**/*.md', '!**/second']);                 // ['first/file.md']
fg.sync(['**/*.md'], { ignore: ['**/second/**'] }); // ['first/file.md']

:warning: When you write !**/second/**/* it means that the directory will be read, but all the entries will not be included in the results.

You have to understand that if you write the pattern to exclude directories, then the directory will not be read under any circumstances.

How to use UNC path?

You cannot use Uniform Naming Convention (UNC) paths as patterns (due to syntax), but you can use them as cwd directory.

fg.sync('*', { cwd: '\\\\?\\C:\\Python27' /* or //?/C:/Python27 */ });
fg.sync('Python27/*', { cwd: '\\\\?\\C:\\' /* or //?/C:/ */ });

Compatible with `node-glob`?

Benchmarks

Server

Link: Vultr Bare Metal

Processor: E3-1270v6 (8 CPU)
RAM: 32GB
Disk: SSD (Intel DC S3520 SSDSC2BB240G7)

You can see results here for latest release.

Nettop

Link: Zotac bi323

Processor: Intel N3150 (4 CPU)
RAM: 8GB
Disk: SSD (Silicon Power SP060GBSS3S55S25)