browserify-diff
v0.1.2
Published
Module to allow browserify bundle caching and fast minimal cache updates based on version diffs.
Downloads
10
Readme
Module to allow browserify bundle caching and fast minimal cache updates based on version diffs.
The goal of this project is to allow intelligent caching for browserify bundles. With regular browserify bundles you don't have many options for caching of individual modules: when one module changes the entire build will change and thus is required to be fetched entirely. This is very inefficient, especially when you have a large bundle and release often, like we do at Magnet.me. This project provides the necessary tools to enable caching of individual modules without having to fetch all modules individually.
The core of this tool is a new bundle format: the browserify-json-bundle. The browserify plugin bundled in this module changes the output format of a browserify bundle into this browserify-json-bundle format, and additionally can generate a diff with the previous export. The embedded module loader will load a full bundle the first time it is executed. On successive calls, usually on successive page loads, it will only fetch a patch and apply it to the cached bundle (if possible). A tool is also embedded to generate stack all diffs since a specified version, such that only one diff needs to be send (more isn't requested either).
This module provides:
- A browserify plugin that will generate browserify-json-bundles and bundle diffs.
- A bundle loader that can load a browserify-json-bundle and fetch and apply patches.
- A tool that can generate a diff from a base version to latest.
- A set of browserify option overrides.1
Sourcemaps are at the moment of writing not yet properly supported. They might accidentally work, but We're working on it.
Installation
npm install browserify-diff --save
Examples
In your build chain:
var browserify = require('browserify');
var browserifyDiff = require('browserify-diff');
var fs = require('fs');
//Required!!!
var options = browserifyDiff.wrapOptions(myBrowserifyOptions);
var version = getNextVersion(); //e.g. git tag, date, semver, etc.
browserify('./app.js', options)
.plugin(browserifyDiff.plugin, {
version : version,
//We need the previous build to generate a changeset.
//This can simply be a pre-build version, but you can also generate it from the previous
//changesets (which is done in the demo)
previousBuild : getPreviousBuild(),
changesetFile : 'changeset_' + version + '.json'
})
.bundle()
.pipe(fs.createWriteStream('bundle.json'));In your back-end (NodeJS-express example) 2:
var browserifyDiff = require('browserify-diff');
var changesets = loadAllChangesets(); //Should return an array of changesets, such as those generated by the plugin
var diffGenerator = new browserifyDiff.Generator(changesets);
var express = require('express');
var app = express();
//resource for full version is trivial and thus not included here
app.get('diff', function(req, res) {
var baseVersion = req.query.since;
var diff = diffGenerator.getDiffSince(baseVersion);
if (diff) {
res.setHeader('Content-Type', 'application/json');
res.json(diff);
} else {
res.status(204);
}
res.end();
});
In your front-end code:
<!DOCTYPE html>
<html>
<head>
<script src="/node_modules/browserify-diff/bundleLoader.js"></script>
<script>
bundleLoader({
sourceUrl : 'bundle.js',
diffUrl : 'diff?since=%v'
});
</script>
</head>
</html>API
var browserifyDiff = require('browserify-diff');
browserifyDiff.browserifyOptions
An object of options that MUST be passed to browserify for browserify-diff to work.1
browserifyDiff.wrapOptions(options)
Extends a set of user defined options with the required browserify options.
browserifyDiff.plugin(b, opts)
The browserify plugin function. This should not be called directly, but used as a plugin. E.g.:
browserify('./app.js', options)
.plugin(browserifyDiff.plugin, pluginOpts)In this example pluginOpts will be passed as opts to the plugin by browserify.
arguments
b- A browserify instance- [
opts.version] - The version of the build. - [
opts.changesetFile] - The path where the changeset will be stored. - [
opts.previousBuild] - A bundle instance that will be used to derive the changeset from. If not set it is assumed that this is the first build.
browserifyDiff.Generator(changesets)
A constructor function to construct a generator for diffs.
arguments
changesets- An array of all changesets, which will be used to construct the diffs.
browserifyDiff.Generator#getDiffSince(version)
Generates a diff between version and the latest version, derived from the changesets passed to the constructor.
Demo
The /demo folder contains a simple demo. The build.js script is an executable build script that uses index.js as entry point for the bundle and stores the compiled bundle as bundles/latest.json. Additionally this script will generate a new changeset and place it in the changesets folder.
The server.js script can be used to run an example app on port 8000. The app currently does not do much, it initially loads the bundles/initial.json bundle, and on successive loads it will fetch the diffs and applies those. The initial.json bundle is not the latest version, hence you can use the demo to see how the original request goes, how an update is applied, and what happens when no update is present.
Remarks
- The plugin architecture of browserify unfortunately does not give full control: the options are already passed to the sub components before the plugins are called. This makes it impossible to override the options from the plugin. Therefore the user (unfortunately) has to do that manually.
- If you don't have a NodeJS application you should provide your own implementation of a diff merger, since only one diff is requested at a time. The merger embedded in this module (from the browserify-json-bundle-diff) does only require a CommonJS environment, not necessarily NodeJS. It can thus be used in any environment that supports CommonJS modules. E.g. At Magnet.me we're going to use Java's Nashorn implementation with jvm-npm to run the merger inside the JVM.