vinyl-tasks

A JavaScript task runner using Orchestrator and Vinyl adapters. Works with Gulp.

Anyone who has used Gulp knows of its power.

The truth is, Gulp is merely a wrapper around other libraries. The real power lies in the Vinyl filesystem adapter (which happens to have also been written by the Gulp Organization).

When combined with Orchestrator to run tasks in maximum concurrency and manage events, some amazing things can be accomplished.

However, Gulp also comes with its share of "limitations". Most importantly, there is no supported node JS interface. Gulp expects to be run via the CLI. This can be a huge limitation when developing robust automation workflows.

It is also very difficult to pass run-time options to Gulp tasks without manually parsing command-line options, and arguably impossible to chain reusable streams in a single pipeline for maximum performance (at least out-of-the-box).

To be clear, Gulp does what it is supposed to extremely well. However, many people find the toolset limiting and would prefer a promise-based (i.e. Bluebird) JavaScript interface to build more robust automation workflows.

Although some of people's concerns will be addressed in Gulp 4 (which will take on a new flavor), there will continue to be value in using vinyl-tasks. The 3.x branch fo Gulp will remain tried and true, and the abundance of community-built plugins will continue to work for both the 3.x and 4.x branches of Gulp.

vinyl-tasks + gulp plugins = ❤

As an alternative to Gulp, in an attempt to address some of the current limitations, vinyl-tasks was created. It supports everything Gulp supports, in a simplified interface with enhanced functionality.

Also, vinyl-tasks is built on the same foundation and thus, the learning curve is a breeze.

Installation

$ npm install --save-dev vinyl-tasks

Usage

The vinyl-tasks interface consists of three main methods:

1. create(task)
2. filter(filterName, pattern, options)
3. pipeline(taskNames, options)

If looking for more information on how vinyl-tasks and Gulp coexist, learn about how to make the transition.

create(task)

Create a task runner for the given task object.

The task object contains all information necessary to define a task. Additionally, it returns a task runner that can be used any number of times, passing different options to modify the behavior of the run.

The task runner, when invoked, returns a Bluebird promise that is resolved or rejected based on the success/failure of the task run.

Additionally, tasks that are set up to be chainable can take advantage of the vinyl-tasks pipeline for maximum performance.

const runner = tasks.create({
  name: 'something',
  callback: callback
});

runner(options)
    .then(function() {
      console.log('everything is done!');
    })
    .catch(function(err) {
      console.error('something messed up :-(');
      console.error(err);
    })
    .done();

See the single task example or pipeline example for discrete implementation details.

task.name

{string} - The unique name for identification. (required)

task.callback

{function} - The callback that when invoked, returns the task function for performing all operations related to the task. (required)

The returned task function is assumed to be synchronous. For the task function to be asynchronous, one of the following must hold true:

1. task.callback returns a task function that accepts a callback.
2. task.callback returns a task function that returns a promise.
3. task.callback returns a task function that returns a stream.

These rules are dictated by orchestrator.fn and are cohesive with the Gulp API.

Additionally, the task has the opportunity to be chainable for use with vinyl-tasks pipeline.

tasks.create({
  name: 'some-sync',
  callback: someSyncCallback
});

function someSyncCallback(options) {
  return function() {
    doSyncThing();
  };
}

tasks.create({
  name: 'some-async',
  callback: someAsyncCallback
});

function someAsyncCallback(options) {
  return function(callback) {
    doAsyncThing(function() {
      callback();
    }); 
  };
}

tasks.create({
  name: 'some-promise',
  callback: somePromiseCallback
});

function somePromiseCallback(options) {
  return function() {
    return doPromiseThing().then(function() {
      console.log('Promise fulfilled!');
    });
  };
}

tasks.create({
  name: 'some-stream',
  callback: someStreamCallback
});

function someStreamCallback() {
  return function() {
    return vfs.src('**/*.foo.js').pipe(streamPlugin()).pipe(vfs.dest('build'));
  }
}

See the single task example or pipeline example for more detailed implementations.

task.chainable

{boolean} - At runtime, create a top-level pipeline for use with vinyl-tasks pipeline. Defaults to true.

By default, all tasks registered with the create interface are assumed to be chainable. A chainable task is an immutable pipeline that is invoked at runtime.

lazypipe does an exceptional job at creating such pipelines.

A chainable task should not create a vinyl stream; the stream will be created automatically by vinyl-tasks. Instead, it should filter the incoming stream.

The following example illustrates the difference between implementing a chainable task and a task that is not chainable:

tasks.create({
  name: 'not-chainable',
  callback: notChainable,
  chainable: false
});

tasks.create({
  name: 'chainable',
  callback: chainable
  chainable: true // superfluous; 'chainable: true' is the default behavior.
});

function notChainable() {
  return function() {
    return vfs.src('**/*.foo.js') // could be swapped for gulp.src
        .pipe(streamPlugin())
        .pipe(vfs.dest('build')); // could be swapped for gulp.dest
  };
}

function chainable() {
  return lazypipe() // must return a lazy-evaluated stream
  
    // will filter from ['**/*', '!node_modules'] - @see filter(filterName, options)
    .pipe(tasks.filter('chainable.foo', '**/*.foo.js'))
    .pipe(streamPlugin)
    .pipe(vfs.dest, 'build')
}

chainable tasks are most useful in larger pipelines.

task.color

The color to use when logging certain characteristics about the task.

Can be any color supported by chalk. Defaults to cyan.

task.hooks

A function that when invoked, returns an object containing hooks for tapping into the vinyl-tasks workflow.

Since vinyl-tasks can be used both individually (using the task runner returned by create) or in sequence (using vinyl-tasks pipeline, the tasks can be built in a reusable fashion.

In order to perform common operations regardless of whether a task is run individually or in a pipeline, hooks have been established for tapping into the vinyl-tasks workflow.

The following demonstrates all of the possible hooks and their meanings:

tasks.create({
  name: 'something',
  callback: callback,
  hooks: hooks
});

function hooks(options) {
  return {
    before: function() {
      console.log('I am doing something before the task starts');
    }
    
    done: function() {
      console.log('I am doing something after the task completes successfully');
    },
    
    /**
     * Provide the opportunity to prevent a task from running,
     * whether individually or when a part of a sequence.
     *
     * Should return `true` or `false` to validate the operation.
     * Additionally, can return a promise that should be fulfilled
     * with the value of `true` or `false`.
     */
    validate: function() {
      console.log('I am preventing the task from running');
      return false; // or return a promise
    }
  }
}

task.label

The label to use when logging, to identify the task action.

For the following task configuration:

task.create({
  name: 'something',
  callback: callback,
  label: 'the thing that things the thing'
});

...the resulting STDOUT would be:

Running the thing that things the thing...
Done

If not set, label will default to the task name:

Running something...
Done

filter(filterName, pattern, options)

Create a filter function that filters a vinyl stream when invoked. Additionally stores the filter in memory for use with filter.restore.

Uses gulp-filter to register a lazy-evaulated filter that is invoked at runtime. Useful in conjunction with creating chainable tasks; the filter can be used to limit the scope of a pipeline, and then restored afterwards for chainability.

The filterName should be namespaced. It is used to identify the filter, and all tasks share the same internal memory storage for filters. This name is also used for lookup when using filter.restore later in the task.

The pattern and options arguments are dictated by gulp-filter.filter.

tasks.create({
  name: 'something',
  callback: callback
});

function callback() {
  return lazypipe()
    .pipe(tasks.filter('something.foo', '**/*.foo.js'))
    .pipe(streamPlugin)
    .pipe(vfs.dest, 'build');
}

filter.restore(filterName)

Restore a filter that was created through the filter interface.

Assuming a filter was created for chainability, it is often useful to create the filter with options.restore set to true. In doing so, a chainable task can limit the scope of a pipeline, and then restore it afterwards so subsequent tasks can assume the same incoming vinyl stream.

tasks.create({
  name: 'something',
  callback: callback
});

function callback() {
  return lazypipe()
    
    // will filter from ['**/*', '!node_modules'], because task.chainable === true (default)
    .pipe(tasks.filter('something.foo', '**/*.foo.js', {restore: true}))
    .pipe(streamPlugin)
    .pipe(vfs.dest, 'build')
    
    // restores the stream to ['**/*', '!node_modules']
    .pipe(tasks.filter.restore('something.foo'))
}

filter.restore does not sanity check that the filter exists.

pipeline(taskNames, options)

Run a single continuous pipeline of multiple tasks, by piping the vinyl stream from one task to the next.

Accepts an {Array} of task names, and an (object} of options that are passed through the entire pipeline.

taskNames can only contain the task names for tasks that have been registered via the create interface. Additionally, the registered tasks must all be chainable.

tasks.pipeline(['task1', 'task2', 'task3'], options)
    .then(function() {
      console.log('everything is done!');
    })
    .catch(function(err) {
      console.error('something messed up :-(');
      console.error(err);
    })
    .done();

For implementation details, see the complete pipeline example.

Options

When using the task runner returned by create, or when using the vinyl-tasks pipeline, options can be passed to the interface at runtime to modify the behavior of a particular run. The options then become available to all callbacks and hooks. Additionally, vinyl-tasks understands the following options internally and uses them internally at runtime:

quiet

{boolean} - Suppress all output (suppress STDOUT and STDERR).

verbose

{*} - Show more output. Can be true, false, or a {Number} indicating the verbosity level. The higher the level, the more output is displayed.

Transitioning from Gulp

To reiterate, Gulp is a thin layer around vinyl-fs and orchestrator.

Although vinyl-tasks does not use Gulp directly, it does not argue with the philosophies of Gulp. In fact, vinyl-tasks depends on Gulp. Much of the module is built on the same principles and hard work done by the Gulp organization, as well as the community-driven plugins that make Gulp so easy to use.

By extending the functionality and concepts of Gulp (and orchestrator), it is possible to take advantage of the power of these tools to do incredible things.

Therefore "Transitioning from Gulp" is a bit of a misnomer, since vinyl-tasks is built on top of the same foundation.

To create a "Gulp" task (again, a misnomer) using the vinyl-tasks interface would mean to disable all of the extra functionality, and just take advantage of the node / Bluebird promise interface.

To illustrate, here is a rudimentary task written for Gulp:

const gulp = require('gulp');

gulp.task(['something'], callback);

function callback() {
  doSyncThing();
}

.. and its usage:

$ gulp something

Here is the same tasks (and its usage), using the create interface provided with vinyl-tasks:

const tasks = require('vinyl-tasks');

const runner = tasks.create({
  name: 'something',
  callback: callback,
  chainable: false
});

// "options" not needed, just demonstrating that they can be used in any capacity at runtime
function callback(options) {
  return function() {
    if (options.wow) {
      doSyncThing();
    }
  };
}

// Usage
const options = {
  wow: 'you mean',
  i: 'can pass',
  runtime: 'options?'
};

runner(options)
    .then(function() {
      console.log('everything is done!');
    })
    .catch(function(err) {
      console.error('something messed up :-(');
      console.error(err);
    })
    .done();

HOWEVER, to truly take advantage of the power of vinyl-tasks, consider tacking on the additional functionality, as demonstrated in the single task example and pipeline example.

Example (single task)

The following example demonstrates all of the functionality in vinyl-tasks, for a single task. More detailed explanations of each block of code can be found in the related sections of documentation.

Additionally, multiple tasks can be run in a single pipeline.

For an arbitrary task, called "something":

const gulpif = require('gulp-if');             // it is always useful to take advantage of `Gulp` plugins
const lazypipe = require('lazypipe');          // make a lazy pipeline for chainability
const streamPlugin = require('stream-plugin'); // arbitrary stream plugin
const tasks = require('vinyl-tasks');          // this module
const vfs = require('vinyl-fs');               // `gulp.src` and `gulp.dest` actually come from `vinyl-fs`

const runner = tasks.create({
  name: 'something',
  callback: callback,
  color: 'magenta',
  hooks: hooks,
  label: 'the thing that things the thing'
});

function callback(options) {
  return lazypipe()
      
      // will filter from ['**/*', '!node_modules']
      .pipe(task.filter('something.foo.js', '**/*.foo.js'))
      .pipe(gulpif, options.wow, streamPlugin) 
      .pipe(vfs.dest, 'build')
      
      // restores the stream to ['**/*', '!node_modules']
      .pipe(task.filter.restore('something.foo.js'));
}

function hooks(options) {
  return {
    before: function() {
      console.log('I am doing something before the task starts');
    }
    
    done: function() {
      console.log('I am doing something after the task completes');
    },
    
    validate: function() {
      console.log('I am preventing the task from running');
      return false; // or return a promise
    }
  }
}

// Usage
const options = {
  wow: 'you mean',
  i: 'can pass',
  runtime: 'options?'
};

runner(options)
    .then(function() {
      console.log('everything is done!');
    })
    .catch(function(err) {
      console.error('something messed up :-(');
      console.error(err);
    })
    .done();

Example (pipeline)

The following example demonstrates how to run multiple tasks in a single pipeline using vinyl-tasks. More detailed explanations of each block of code can be found in the related sections of documentation.

This example does not show the full configuration/setup for each task. For more details on setting up a single task, see the single task example.

const tasks = require('vinyl-tasks');

tasks.create({
  name: 'task1',
  callback: callback1
});

tasks.create({
  name: 'task2',
  callback: callback2
});

tasks.create({
  name: 'task3',
  callback: callback3
});

// Usage
const options = {
  wow: 'you mean',
  i: 'can pass',
  runtime: 'options?'
};

// options get passed through entire pipeline to all task callbacks/hooks
tasks.pipeline(['task1', 'task2', 'task3'], options)
    .then(function() {
      console.log('everything is done!');
    })
    .catch(function(err) {
      console.error('something messed up :-(');
      console.error(err);
    })
    .done();

Contributing

License

The MIT License (MIT)

Copyright (c) 2016 Justin Helmer

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.