gulp-restructure-tree

A powerful, declarative way to restructure complex directory structures with Gulp. Use globs, RegExps and functions arranged in a tree structure to rename files.

Quick start

Installation

$ npm install gulp-restructure-tree --save-dev

Usage

restructureTree(treeMap[, options])

Example

Describe your file tree with plain javascript objects to move or filter files in your Gulp pipeline:

const restructureTree = require('gulp-restructure-tree')

gulp.src('./**')
  .pipe(restructureTree({
    'src': {
      '**.md': null,             // Exclude all markdown files in src/
      'utils/': 'libs/'          // Move all file in src/utils/ to libs/
    },
    'log-*.txt': 'logs/$1.log', // Rename files matching ./src/log-*
  }))
  .pipe(gulp.dest('./dist'))

// src/utils/react-if/dist/index.js  ⇒ dist/libs/react-if/dist/index.js
// log-1491831591372.txt             ⇒ dist/logs/1491831591372.log

gulp-restructure-tree gives you many more ways to match files and directories, read on to learn more.

Table of Contents

• Quick start
  • Installation
  • Usage
  • Example
• API
  • treeMap
    • Branches
    • Leaves
  • options
• Comprehensive example
• Developing
  • Tests
• Known issues

API

restructureTree( treeMap[, options] )

Arguments:

treeMap:

A tree structure described with plain javascript objects. Keys are branches and non-object values are leaves.

• branches: A branch represents a path segment, the name of one directory or file. Simple bash-style *, ** glob patterns can be used, or Regular Expression for more complicated matching requirements.
  • foo: Regular string branches match filenames literally.
  • *: A wildcard can be included anywhere in the path to match any number of characters in a filename.
    e.g.: fo*ar will match foobar but will not match nested directories foo/bar.
  • **: A branch name can begin with a "globstar" pattern, in that case it will recursively match as many directories as necessary to complete the rest of the tree.1
    e.g.: **az will match foo/bar/baz.
  • foo/: If the last branch in a route ends with a /, it will match everything inside that directory.2
    This is equivalent to having a branch followed by a globstar pattern branch. e.g.: { 'foo/': 'bar' } is the same as { 'foo': { '**': 'bar' }}.
  • [/foo[a-z-]+/]: A branch name can be a Regular Expression. It will match any branch that matches it. No RegExp flags are supported.3
  • [["foo","bar*","baz"]]: A branch name can also be an array, in which case it will match if any of the array elements matches.3
• leaves: Any value that is either a function, a string, or null, represents a destination. Files whose paths match the path leading up to the leaf will be moved to the specified destination.
  • {string} foo/: A string destination path that ends with a / is interpreted as a directory destination. Matched files will keep their filenames, and the folder hierarchy that starts at the point the file path and the treeMap path "diverges" will be conserved4.
  • {string} foo: A string destination path that doesn't end with a / is interpreted as a file destination. Matched files will be renamed to match the new destination filename, and all of their original folder hierarchy discarded.
    To avoid conflicts when using a string destination file, either make sure to match only a single file, or match multiple files with globs or RegExps branches and include back references ($1, $2, ...) in the destination string.
  • (pathObject) => {string|pathObject}: Takes a parsed pathObject, and returns either a path string that will be interpreted the same as a string leaf, or a pathObject that will be applied directly.5
  • null: The file will be removed from the stream.

options:

• dryRun {boolean=false}: Don't move files, only log operations that would be executed (implies verbose mode).
• verbose {boolean=false}: Log file move operations to stdout.
• logUnchanged {boolean=false}: In verbose mode, also log operations that don't result in any path change.
• onlyFiles {boolean=false}: Don't process directories.
• bypassCache {boolean=false}: Force recompile of treeMap argument. 6

(In case of doubts, see PathMatcher.test.js for a more comprehensive spec.)

Comprehensive example (Gulp V4)

// gulpfile.babel.js
import restructureTree from 'gulp-restructure-tree'

// Map starting directory structure with a tree having the destinations as its
//  leaves.
const treeMap = {
  app: {
    modules: { 
      '*': {
      // Use wildcard globs: * and ** (recursive).

        '**.css': 'styles/', 
        // A string value is interpreted as a destination path.
        // A destination with a trailing / is interpreted as a directory, without
        //  it is interpreted as a destination file and the file will be renamed.
        // app/modules/basket/alt/christmas.css ⇒ styles/basket/alt/christmas.css
        
        gui: {
          '**GUI.jsx': 'templates/$1-$3.js',
          // Insert back references, here whatever directory was matched by the *
          //  just after 'modules' will serve as the first part of the filename.
          // This "trailing" ** counts as two capture groups, one for
          //   the folders and one for the ending file ("**/*").
          // app/modules/basket/gui/itemsGUI.jsx ⇒ templates/basket-items.js 
        },
        
        [/(.*)(\.min)?\.jsx?/]: 'scripts/$2.js',
        // Use RegExp by converting them to strings, capture groups are used for the
        //  back references.
        // app/modules/basket/itemsRefHandling.min.js ⇒ scripts/itemsRefHandling.js
        
        'img/': 'assets/$1/$3.compressed',
        // A trailing branch with a trailing / is interpreted as a recursive
        //   match (equivalent to a trailing '**' branch).
        // app/modules/basket/img/cart.png ⇒ assets/basket/cart.png.compressed
      }
    },
    helpers: {
      'jquery-*': 'libs/global/',
      // Rules are evaluated from top to bottom so place more specific rules on top.
      // If you placed this rule after the following one, it would match on the
      //  first one and never reach this one.
      // app/helpers/jquery-3.1.1.min.js ⇒ libs/global/jquery-3.1.1.min.js

      [/.*\.jsx?/]:
              ({ name, extname }) =>
                      `libs/${extname === 'jsx' ? 'react/' : ''}${name}/${name}.js`
      // Function leaves take a parsed path as argument and return a destination
      //  path as a string or parsed path object.
      // app/helpers/modalify.js ⇒ libs/modalify/modalify.js
      // app/helpers/dropdown.jsx ⇒ libs/react/dropdown/dropdown.js
    }
  },
  documentation: {
      '/': './docs',
      // Match folder by selecting it with a '/' rule. This will move the empty
      //  directory.
      // documentation ⇒ docs
      
      [['COPYRIGHT', 'LICENSE', 'CHANGELOG-*']]: './',
      // Use arrays to match a set of filenames without bothering with RegExps.
      // documentation/LICENSE ⇒ LICENSE
      
      '**': null,
      // Discard files by routing them to a null leaf.
      // documentation/module-organization.html ⇒ [Removed from output]
  },
}

export function move() {
  gulp.src('./src/**', { base: './src' })
    .pipe(restructureTree(treeMap))
    .pipe(gulp.dest('./dist')
}

Developing

$ git clone https://github.com/SewanDevs/ceiba-router.git
$ npm run build      # Compile project with Webpack
$ npm run watch      # Run Webpack in watch mode

Tests

$ npm run test       # Run Jest on all .test.js files
$ npm run testWatch  # Run Jest in watch mode

Known issues

• For a cleaner and more concise tree structure definition, this plugin asks you to describe your file tree as a plain javascript object, and so relies on object keys iteration order which is standardized as being undetermined (i.e. applications should assume object keys are unordered). However, iteration order is deterministic and consistent on both V8 (Node.js, Chrome) and SpiderMonkey (Firefox): string keys are itered in definition order, except for keys that are integers, which come first and are ordered by numerical order. gulp-restructure-tree will warn you if you use such keys in your tree and suggest making the branch a simple RegExp as a workaround to keep the definition order (e.g.: "7"→"/7/").

[1]: A globstar introduces two capture groups, one that will match the last folder matched, and one that will match the file name (e.g.: with { "foo/**": "quux/$1/$2" }: ./foo/bar/baz/qux⇒./quux/baz/qux).

[2]: Internally, a globstar is appended to the route, which means you can reference them in capture groups (e.g.: with { "foo/": "qux/$1/$2.bak" }: ./foo/bar/baz⇒./qux/bar/baz.bak).

[3]: RegExp and Array branches are included using computed property name syntax. The string representation of ReExps and Array instances is used to differentiate them from plain branches. Thus, a branch that begins and ends with a / will be interpreted as a RegExp branch, and a branch that has a , in it will be interpreted as an array branch. If you want to match a literal comma in a filename, you must escape it (e.g.: "foo\\,bar.txt" will match the file foo,bar.txt).

[4]: The concept of divergence is defined as follow: until a tree path is composed of regular string branches and it matches the file path, the paths are not diverging. The non-diverging matching directories will be "eaten" from the resulting path (e.g.: with { foo: { bar: { 'b*z/': 'dest/' }}}: foo/bar/baz/qux⇒dest/baz/qux, the folders foo and bar have been omitted in the resulting path, but baz has been kept since it isn't an "exact" match).

[5]: The pathObject sent to function destinations has an additional key in addition to those provided by path.parse: full, which is the full relative path name as provided by the vinyl stream. It is provided as a convenience and is ignored in the function's return value.

[6]: treeMap objects are compiled and cached in memory. It won't be recompiled if you pass the same object to gulp-restructure-tree multiple times. This means splitting definitions in multiple sub-trees for the different tasks shouldn't significantly improve performance. This also means that if you want to mutate the same treeMap object in between calls to gulp-restructure-tree, your changes won't be accounted for unless you pass in the bypassCache option.