raptor-optimizer-rrr

v1.0.12-beta

Published

2 years ago

Module to support the optimized delivery of web application resources

Downloads

0High
0Medium
0Low

rragan

Raptor Optimizer

The Raptor Optimizer allows you to easily share JavaScript code between the client and server while also providing first-level support for optimally delivering JavaScript, CSS, images and other assets to the browser.

This tool offers many different optimizations such as a bundling, lazy loading, compression and fingerprinted resource URLs. Plugins are provided to support pre-processors and compilers such as Less, Stylus and Raptor Templates. This developer-friendly tool does not require that you change the way that you already code and can easily be adopted by existing applications.

Example

Install some modules from npm:

npm install raptor-optimizer-cli --global
npm install change-case

Create the main Node.js JavaScript module file:

main.js:

var changeCase = require('change-case');
console.log(changeCase.titleCase('hello world')); // Output: 'Hello World'

Create a StyleSheet for the page:

style.css

body {
    background-color: #5B83AD;
}

Create an HTML page to host the application:

my-page.html:

<!doctype html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>Raptor Optimizer Demo</title>
</head>
<body>
    <h1 id="header">Raptor Optimizer Demo</h1>
</body>
</html>

Run the following command:

raptor-optimizer style.css --main main.js --inject-into my-page.html --watch

Output:

Output for page "my-page":
  Resource bundle files:
    static/my-page-9ac9985e.js
    static/my-page-1ae3e9bf.css
  HTML slots file:
    build/my-page.html.json
  Updated HTML file:
    my-page.html

Open up my-page.html in your web browser and in the JavaScript console you will see the output of our program running in the browser, as well as a page styled by style.css.

As you can see, with the Raptor Optimizer you no longer have to struggle with managing complex build scripts. Simply let the Raptor Optimizer worry about generating all of the required optimized resource bundles and injecting them into your page so that you can just focus on writing clean and modular code.

There's also a JavaScript API, taglib and a collection of plugins to make your job as a front-end web developer easier. Please read on to learn how you can easily utilize the Raptor Optimizer in your application.

Design Philosophy

Dependencies should be declarative or discovered via static code analysis
Common module loading patterns should be supported
Must be extensible to support all types of resources
Maximize productivity in development
Maximize performance in production
Must be easy to adopt and not change how you write your code
Can be used at runtime or build time
Must be configurable

Features

Optimize Client-side Dependencies
- Supports all types of dependencies (JavaScript, CSS, images, Less, CoffeeScript, etc.)
- Configurable resource bundling
- JavaScript minification
- CSS minification
- Fingerprinted resource URLs
- Prefix resources with CDN host name
- Optional Base64 image encoding inside CSS files
- Custom output transforms
- Declarative browser-side package dependencies using simple optimizer.json files
- Generates the HTML markup required to include optimized resources
- etc.
Browser-side Node.js Module Loader
- Full support for Isomorphic JavaScript
- Conflict-free CommonJS module loader for the browser
- Complete compatibility with Node.js
  - Supports module.exports, exports, require, require.resolve, __dirname, __filename, process, etc.
- Supports the package.json browser field
- Full support for browserify shims and transforms
- Maintains line numbers in wrapped code
Developer Friendly
- Disable bundling and minification in development
- Line numbers are maintained for Node.js modules source
- Extremely fast incremental builds!
  - Only modified bundles are written to disk
  - Disk caches are utilized to avoid repeating the same work
Dependency Compilation
- Less
- Raptor Templates
- Dust
- etc.
Extensible
- Custom dependency compilers
- Custom code transforms
- Custom bundle writers
- Custom plugins
Configurable
- Configurable resource bundles
- Enable/disable transforms
- Development-mode versus production-mode
- Enable/disable fingerprints
- etc.
Flexible
- Integrate with build tools
- Use with Express or any other web development framework
- JavaScript API, CLI and taglib
Future
- Automatic image sprites
- Automatic image compression

Another Client-side Bundler?

While Browserify is a popular client-side bundler, you will find that it has certain limitations. Browserify is very JavaScript-centric and was originally designed to only support the transport of Node.js modules to the browser. The starting point for Browserify is always a Node.js JavaScript module file which means that it would not be very suitable for a project that includes "plain" JavaScript code or only CSS. Compared to Browserify, the Raptor Optimizer was designed be a more general solution. In addition to supporting the transport of Node.js modules to the browser, it also supports optimizing CSS, non-Node.js JavaScript code and even images. In fact, the Raptor Optimizer provides almost all of the features of Browserify (including support for Browserify shims and transforms) and extends it to support much more (configurable bundles, asynchronous loading, incremental builds, etc.).

Webpack, another client-side bundler, overloads the core Node.js module loader API with non-standard and proprietary methods and features. As a result, if you were to write code that conforms to what Webpack expects then that code will likely only run on the client and not on a Node.js server, and you will be tied to Webpack. In comparison, the Raptor Optimizer relies only on the standard Node.js API so that your JavaScript code will be truly isomorphic and work on both the server and the client.

Here's just a sampling of the non-standard features that Webpack introduces:

require("!style!css!./style.css"); // Non-standard require syntax
require("./style.css"); // Node.js does not know how to handle *.css modules
require.ensure(dependencies, callback) // require.ensure is not a Node.js method
require.include(request) // require.include is not a Node.js method

A unique feature of the Raptor Optimizer is that in addition to generating optimized JS and CSS bundles, it also generates the HTML markup to include those bundles. By giving the Raptor Optimizer control over the <script> and <link> tags, this tool can change how resources are bundled or add/remove fingerprints to bundles, etc., without requiring any change to application code.

Tutorials

Tutorial: Command Line Interface

Sample App: To try out and experiment with the code for this tutorial, please see the following project:raptor-samples/optimizer-cli

Install the command line interface for the Raptor Optimizer:

npm install raptor-optimizer-cli --global

In an empty directory, initialize a new Node.js project using the following command:

mkdir my-app
cd my-app
npm init

Install required modules into the new project:

npm install jquery
npm install raptor-optimizer-less

Create the following files:

add.js:

module.exports = function(a, b) {
    return a + b;
};

main.js:

var add = require('./add');
var jquery = require('jquery');

jquery(function() {
    $(document.body).append('2+2=' + add(2, 2));
});

style.less:

@headerColor: #5B83AD;

#header {
    color: @headerColor;
}

my-page.html:

<!doctype html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>Raptor Optimizer Demo</title>
</head>
<body>
    <h1 id="header">Raptor Optimizer Demo</h1>
</body>
</html>

Finally, run the following command to generate the optimized resource bundles for the page and to also inject the required <script> and <link> tags into the HTML page:

raptor-optimizer style.less \
    --main main.js \
    --inject-into my-page.html \
    --plugins raptor-optimizer-less \
    --development

If everything worked correctly then you should see output that includes output similar to the following:

Output for page "my-page":
  Resource bundle files:
    static/add.js
    static/raptor-modules-meta.js
    static/main.js
    static/node_modules/jquery/dist/jquery.js
    static/raptor-modules-1.0.1-beta/client/lib/raptor-modules-client.js
    static/style.less.css
  HTML slots file:
    build/my-page.html.json
  Updated HTML file:
    my-page.html

The updated my-page.html file should be similar to the following:

<!doctype html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>Raptor Optimizer Demo</title>
    <!-- <optimizer-head> -->
    <link rel="stylesheet" type="text/css" href="static/style.less.css">
    <!-- </optimizer-head> -->
</head>
<body>
    <h1 id="header">Raptor Optimizer Demo</h1>
    <!-- <optimizer-body> -->
    <script type="text/javascript" src="static/raptor-modules-1.0.1-beta/client/lib/raptor-modules-client.js"></script>
    <script type="text/javascript" src="static/add.js"></script>
    <script type="text/javascript" src="static/raptor-modules-meta.js"></script>
    <script type="text/javascript" src="static/node_modules/jquery/dist/jquery.js"></script>
    <script type="text/javascript" src="static/main.js"></script>
    <script type="text/javascript">$rmod.ready();</script>
    <!-- </optimizer-body> -->
</body>
</html>

If you open up my-page.html in your web browser you should see a page styled with Less and the output of running main.js.

Now try again with production mode:

raptor-optimizer style.less \
    --main main.js \
    --inject-into my-page.html \
    --plugins raptor-optimizer-less \
    --production

Output for page "my-page":
  Resource bundle files:
    static/my-page-2e3e9936.js
    static/my-page-169ab5d9.css
  HTML slots file:
    build/my-page.html.json
  Updated HTML file:
    my-page.html

The updated my-page.html file should be similar to the following:

<!doctype html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>Raptor Optimizer Demo</title>
    <!-- <optimizer-head> -->
    <link rel="stylesheet" type="text/css" href="static/my-page-169ab5d9.css">
    <!-- </optimizer-head> -->
</head>
<body>
    <h1 id="header">Raptor Optimizer Demo</h1>
    <!-- <optimizer-body> -->
    <script type="text/javascript" src="static/my-page-2e3e9936.js"></script>
    <script type="text/javascript">$rmod.ready();</script>
    <!-- </optimizer-body> -->
</body>
</html>

With the --production option enabled, all of the resources are concatenated together, minified and fingerprinted – perfect for high performance web applications running in production.

Tutorial: JSON Configuration File

Sample App To try out and experiment with the code for this tutorial, please see the following project:raptor-samples/optimizer-config

The number of command line arguments can get unwieldy so it is better to split out configuration into a separate JSON file. Let's now create a configuration file and configure a few bundles to make things more interesting:

raptor-optimizer-config.json:

{

    "plugins": [
        "raptor-optimizer-less"
    ],
    "fileWriter": {
        "outputDir": "static",
        "fingerprintsEnabled": true
    },
    "minify": true,
    "resolveCssUrls": true,
    "bundlingEnabled": true,
    "bundles": [
        {
            "name": "jquery",
            "dependencies": [
                "require: jquery"
            ]
        },
        {
            "name": "math",
            "dependencies": [
                "require: ./add"
            ]
        }
    ]
}

In addition, let's put our page dependencies in their own JSON file:

my-page.optimizer.json:

{
    "dependencies": [
        "style.less",
        "require-run: ./main"
    ]
}

Now run the page optimizer using the newly created JSON config file and JSON dependencies file:

raptor-optimizer ./my-page.optimizer.json --inject-into my-page.html --config raptor-optimizer-config.json

Because of the newly configured bundles, we'll see additional JavaScript bundles written to disk as shown below:

Output for page "my-page":
  Resource bundle files:
    static/math-169c956d.js
    static/jquery-24db89d9.js
    static/my-page-beed0921.js
    static/my-page-169ab5d9.css
  HTML slots file:
    build/my-page.html.json
  Updated HTML file:
    my-page.html

Tutorial: Asynchronous/Lazy Loading

Sample AppTo try out and experiment with the code for this tutorial, please see the following project:raptor-samples/optimizer-async

Asynchronously (i.e. lazy loading) of additional dependencies is also supported by the Raptor Optimizer as shown in the following sample code:

var foo = require('foo');
var raptorLoader = require('raptor-loader');
exports.doSomething = function(callback) {
    raptorLoader.async(function(err) {
        if (err) {
            // Handle the case where one or more of the
            // dependencies failed to load.
        }

        // Any modules that are required within the scope
        // of this function will be loaded asynchronously
        var bar = require('bar');
        var baz = require('baz');

        // Do something with bar and baz...

        callback();
    });
}

Tutorial: JavaScript API

Sample App To try out and experiment with the code for this tutorial, please see the following project:raptor-samples/optimizer-js-api

For added flexibility there is a JavaScript API that can be used to optimize pages as shown in the following sample code:

var raptorOptimizer = require('raptor-optimizer');
raptorOptimizer.configure('raptor-optimizer-config.json');
raptorOptimizer.optimizePage({
        name: 'my-page'
        dependencies: [
            "style.less",
            "require-run: ./main"
        ]
    },
    function(err, optimizedPage) {
        if (err) {
            // Handle the error
        }

        var headHtml = optimizedPage.getHeadHtml();
        // headHtml will contain something similar to the following:
        // <link rel="stylesheet" type="text/css" href="static/my-page-169ab5d9.css">

        var bodyHtml = optimizedPage.getBodyHtml();
        // bodyHtml will contain something similar to the following:
        //  <script type="text/javascript" src="static/my-page-2e3e9936.js"></script>
    });

Tutorial: Optimizer Taglib

Sample App To try out and experiment with the code for this tutorial, please see the following project:raptor-samples/optimizer-taglib

For the ultimate in usability, a taglib is provided for Raptor Templates (and Dust) to automatically optimize a page and inject the required HTML markup to include the optimized JavaScript and CSS bundles. Sample Raptor Template is shown below:

my-page.rhtml:

<!-- Declare the top-level dependencies for the page: -->
<optimizer-page name="my-page" package-path="./my-page.optimizer.json"/>

<!doctype html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>Raptor Optimizer Demo</title>

    <!-- <link> tags will be injected below: -->
    <optimizer-head/>
</head>
<body>
    <h1 id="header">Raptor Optimizer Demo</h1>

    <!-- <script> tags will be injected below: -->
    <optimizer-body/>
</body>
</html>

Using Raptor Templates and the Optimizer taglib, you can simply render the page using code similar to the following:

var template = require('raptor-templates').load('my-page.rhtml');
template.render({}, function(err, html) {
    // html will include all of the required <link> and <script> tags
});

Tutorial: Client/Server Template Rendering

Sample App To try out and experiment with the code for this tutorial, please see the following project:raptor-samples/optimizer-templates

To demonstrate rendering of the same template on the server and the client we will start with the following Raptor template:

template.rhtml

Hello ${data.name}!

NOTE: The sample app for this tutorial includes sample code that illustrates how to also render both a Dust template and a Handlebars template on both the client and server.

We will then create a main.js file to render the template to the console:

main.js:

var template = require('raptor-templates')
    .load(require.resolve('./template.rhtml'));

template.render(
    {
        name: 'Frank'
    },
    function(err, html) {
        console.log('Template output: ' + html);
    });

NOTE: The reason we use require.resolve('./template.rhtml') instead of require('template.rhtml') is that Node.js does not understand how to load .rhtml modules and the use of the require.extensions has been deprecated. require.resolve() is used to get the resolved path for the template and the raptor-templates module uses that path to load template into memory.

Running node main.js on the server will produce the following output in the console:

Template output: Hello Frank!

In order to automatically detect and compile required *.rhtml templates we will need to install the raptor-optimizer-rhtml plugin using the following command:

npm install raptor-optimizer-rhtml

We can then optimize the page using the following command:

raptor-optimizer style.less \
    --main main.js \
    --inject-into my-page.html \
    --plugins raptor-optimizer-rhtml

After opening my-page.html in your web browser you should then see the same output written to the browser's JavaScript console.

Tutorial: Runtime Optimization with Express

Sample App To try out and experiment with the code for this tutorial, please see the following project:raptor-samples/optimizer-express

The Raptor Optimizer has a smart caching layer and is fast enough so that it can be used at runtime as part of your server application. The easiest way to use the Raptor Optimizer at runtime is to use the taglib and simply render the page template to the response output stream.

The first time the page renders, the page will be optimized and cached and the output of the optimization will be used to produce the final page HTML. After the first page rendering, the only work that will be done by the Raptor Optimizer is a simple cache lookup.

By default, the Raptor Optimizer writes all optimized resource bundles into the static/ directory at the root of your application. In addition, by default, all resource URLs will be prefixed with /static. If resources are to be served up by the local Express server we will need to register the appropriate middleware as shown in the following sample code:

server.js

var express = require('express');
var compression = require('compression');
var serveStatic = require('serve-static');

// Load the page template:
var template = require('raptor-templates')
    .load(require.resolve('./template.rhtml')

var app = express();

// Enable gzip compression for all HTTP responses:
app.use(compression());

// Any URL that begins with "/static" will be served up
// out of the "static/" directory:
app.use('/static', serveStatic(__dirname + '/static'));

app.get('/', function(req, res) {
    // Render the page template as normal:
    template.render({
            name: 'Frank'
        },
        res);
});
...

app.listen(8080);

Installation

The following command should be used to install the raptor-optimizer module into your project:

npm install raptor-optimizer --save

If you would like to use the available command line interface, then you should install the raptor-optimizer-cli module globally using the following command:

npm install raptor-optimizer-cli --global

Usage

Command Line Interface

The raptor-optimizer module includes a command line interface (CLI) that can be used to generate optimized resource bundles from the command line.

A simple usage that writes out a JavaScript bundle and a CSS bundle to the static/ directory that includes all of the required dependencies is shown below:

raptor-optimizer foo.js style.less --main main.js --name my-page

With additional options:

raptor-optimizer jquery.js style.less \
    --main main.js \                         # Entry JavaScript module for the browser
    --name my-page \                         # Give the page bundle files a name
    --out static                             # Output directory
    --url-prefix http://mycdn/static/ \      # URL prefix
    --fingerprint \                          # Include fingerprints
    --html \                                 # Head and body HTML
    --minify \                               # Minify JavaScript and CSS
    --inject-into my-page.html \             # Inject HTML markup into a static HTML file
    --plugin my-plugin \                     # Enable a custom plugin
    --watch                                  # Watch for file changes and re-optimize automatically

For additional help from the command line, you can run the following command:

raptor-optimizer --help

Alternatively, you can create a JSON configuration file and use that instead:

raptor-optimizer --config optimizer-config.json

For more documentation on the Command Line Interface please see the raptor-optimizer-cli docs.

Configuration

Default Configuration

{
    "fileWriter": {
        "outputDir": "static",     // Write all bundles into the "static" directory
        "fingerprintsEnabled": true  // Include fingerprint in output files
    }
}

Complete Configuration

{
    // Plugins with custom dependency compilers, writers, etc.:
    "plugins": [ // Optimizer plugins (see Available Plugins below)
        // Plugins with default config:
        "raptor-optimizer-less",
        "raptor-optimizer-rhtml",
        // Plugin with custom configuration:
        {
            "plugin": "raptor-optimizer-my-plugin",
            "config": { ... }
        },
        ...
    ],
    // Configure the default bundle file writer:
    "fileWriter": {
        "outputDir": "static",     // Where to write the bundles
        "urlPrefix": "http://mycdn/static",    // Generate URLs with specified prefix
        "fingerprintsEnabled": true, // Include fingerprint in output files?
        "includeSlotNames": false  // Include slot name in output files?
    },
    "minify": true, // If true then the "raptor-optimizer-minify-js" and
                    // "raptor-optimizer-minify-css" plugins will be
                    // enabled (defaults to false)
    "resolveCssUrls": true, // If true then the "raptor-optimizer-resolve-css-urls" plugin
                            // will be enabled (defaults to true)
    "bundlingEnabled": true, // If true then resources will be bundled (defaults to true)
    // Pre-configured bundles that apply to all pages:
    "bundles": [
        {
            "name": "bundle1",
            "dependencies": [
                "foo.js",
                "baz.js"
            ]
        },
        {
            "name": "bundle2",
            "dependencies": [
                "bar.js"
            ]
        }
    ]
}

JavaScript API

Configuring the Default Page Optimizer

var optimizer = require('raptor-optimizer');
optimizer.configure(config);

If the value of the config argument is a String then it is treated as a path to a JSON configuration file.

Optimizing a Page

The following code illustrates how to optimize a simple set of JavaScript and CSS dependencies using the default configured optimizer:

var optimizer = require('raptor-optimizer');
optimizer.optimizePage({
        name: 'my-page',
        dependencies: [
            'foo.js',
            'bar.js',
            'baz.js',
            'qux.css'
        ]
    },
    function(err, optimizedPage) {
        if (err) {
            console.log('Failed to optimize page: ', err);
            return;
        }

        var headHtml = optimizedPage.getHeadHtml();
        /*
        String with a value similar to the following:
        <link rel="stylesheet" type="text/css" href="/static/my-page-85e3288e.css">
        */

        var bodyHtml = optimizedPage.getBodyHtml();
        /*
        String with a value similar to the following:
        <script type="text/javascript" src="/static/bundle1-6df28666.js"></script>
        <script type="text/javascript" src="/static/bundle2-132d1091.js"></script>
        <script type="text/javascript" src="/static/my-page-1de22b65.js"></script>
        */

        // Inject the generated HTML into the <head> and <body> sections of a page...
    });

Creating a New Page Optimizer

var pageOptimizer = optimizer.create(config);
pageOptimizer.optimizePage(...);

Dependencies

To optimize a page the Raptor Optimizer walks a dependency graph. A dependency can either be a JavaScript or CSS resource (or a file that compiles to either JavaScript or CSS) or a dependency can be a reference to a set of transitive dependencies. Some dependencies are inferred from scanning source code and other dependencies can be made explicit by listing them out in code or in an optimizer.json file.

It's also possible to register your own custom dependency types. With custom dependency types, you can control how resources are compiled or a custom dependency type can be used to resolve additional dependencies during optimization.

A dependency can be described using a simple String path as shown in the following code:

[
    "style.less",
    "../third-party/jquery.js"
]

In the examples, the dependency type is inferred from the filename extension. Alternatively, the dependency type can be made explicit using either one of the following formats:

[
    "less: style.less",
    { "type": "js", "path": "../third-party/jquery.js" }
]

You can also create a dependency that references dependencies in a separate optimizer.json file. For example:

[
    // Relative path:
    "./some-module/optimizer.json",

    // Look for "my-module/optimizer.json" in "node_modules":
    "my-module/optimizer.json"
]

If the path does not have a file extension then it is assumed to be a path to an optimizer.json file so the following short-hand works as well:

[
    "./some-module"
    "my-module"
]

Conditional Dependencies

The Raptor Optimizer supports conditional dependencies. Conditional dependencies is a powerful feature that allows for a page to be optimized differently based on certain criteria (e.g. "mobile device" versus "desktop"). For caching reasons, the criteria for conditional dependencies should be based on a set of enabled "extensions". An extension is just an arbitrary name that can be enabled/disabled before optimizing a page. For example, to make a dependency conditional such that is only included for mobile devices you can do the following:

{
    "dependencies": [
        { "path": "hello-mobile.js", "if-extension": "mobile" }
    ]
}

If needed, a JavaScript expression can be used to describe a more complex condition as shown in the following sample code:

{
    "dependencies": [
        {
            "path": "hello-mobile.js",
            "if": "extensions.contains('mobile') || extensions.contains('ipad')"
        }
    ]
}

Enabling Extensions

The code below shows how to enable extensions when optimizing a page:

Using the JavaScript API:

pageOptimizer.optimizePage({
    dependencies: [
        { path: 'hello-mobile.js', 'if-extension': 'mobile' }
    ],
    extensions: ['mobile', 'foo', 'bar']
})

Using the Raptor Templates taglib:

<optimize-page ... extensions="['mobile', 'foo', 'bar']">
    ...
</optimize-page>

Node.js-style Module Support

The Raptor Optimizer provides full support for transporting Node.js modules to the browser. If you write your modules in the standard Node.js way (i.e. using require, module.exports and exports) then the module will be able to be loaded on both the server and in the browser.

This functionality is offered by the core raptor-optimizer-require plugin which introduces a new require dependency type. For example:

[
    "require: ./path-to-some-module"
]

If you want to include a module and have it run when loaded (i.e. self-executing) then you should use the require-run dependency type:

[
    "require-run: ./main"
]

The raptor-optimizer-require plugin will automatically scan the source for for any required module to include any additional modules that are required by a particular module (done recursively). For a require to automatically be detected it must be in the form require("<module-name>") or require.resolve("<module-name>").

The raptor-optimizer-require plugin will automatically wrap all Node.js modules so that the psuedo globals (i.e. require, module, exports, __filename and __dirname) are made available to the module source code.

The raptor-optimizer-require plugin also supports browserify shims and browserify transforms.

For more details on how the Node.js modules are supported on the browser, please see the documentation for the raptor-samples/raptor-optimizer-require plugin.

Bundling

By default, all dependencies required for a page will be bundled into a single JavaScript bundle and a single CSS bundle. However, The Raptor Optimizer allows application-level bundles to be configured to allow for consistent bundles across pages and for multiple bundles to be included on a single page. Because the Raptor Optimizer also generates the HTML markup to include page bundles, the page itself does not need to be changed if the bundle configuration is changed.

If a page has a dependency that is part of an application-level bundle then the dependency will be included as part of the application-level bundle instead of being aggregated with the page-level bundle.

Bundling Example

Given the following configured bundles:

{
    ...
    "bundles": [
        {
            "name": "bundle1",
            "dependencies": [
                "foo.js",
                "baz.js"
            ]
        },
        {
            "name": "bundle2",
            "dependencies": [
                "bar.js"
            ]
        }
    ]
}

Optimizing a page that does not include any dependencies in application-level bundles:

raptor-optimizer app.js style.css --name my-page -c optimizer-config.json

Output:

Output for page "my-page":
  Resource bundle files:
    static/my-page.js
    static/my-page.css
  HTML slots file:
    build/my-page.html.json

Optimizing a page that includes "foo.js" that is part of "bundle1":

raptor-optimizer app.js foo.js style.css --name my-page -c optimizer-config.json

Output:

Output for page "my-page":
  Resource bundle files:
    static/my-page.js
    static/bundle1.js
    static/my-page.css
  HTML slots file:
    build/my-page.html.json

For reference, the following is the content of build/my-page.html.json after running the last command:

{
    "body": "<script type=\"text/javascript\" src=\"static/my-page.js\"></script>\n<script type=\"text/javascript\" src=\"static/bundle1.js\"></script>",
    "head": "<link rel=\"stylesheet\" type=\"text/css\" href=\"static/my-page.css\">"
}

For more information on working with bundles. Please see the bundling docs.

Asynchronous Module Loading

The Raptor Optimizer supports asynchronously loading dependencies using the lightweight raptor-loader module as shown in the following sample code:

require('raptor-loader').async(function() {
    // All of the requires nested in this function block will be lazily loaded.
    // When all of the required resources are loaded then the function will be invoked.
    var foo = require('foo');
    var bar = require('bar');
});

You can also specify additional explicit dependencies if necessary:

require('raptor-loader').async(
    [
        'style.less',
        'some/other/optimizer.json'
    ],
    function() {
        // All of the requires nested in this function block will be lazily loaded.
        // When all of the required resources are loaded then the function will be invoked.
        var foo = require('foo');
        var bar = require('bar');
    });

You can also choose to declare async dependencies in an optimizer.json file:

{
    "dependencies": [
        ...
    ],
    "async": {
        "my-module/lazy": [
            "require: foo",
            "require: bar",
            "style.less",
            "some/other/optimizer.json"
        ]
    }
}

The async dependencies can then be referenced in code:

require('raptor-loader').async(
    'my-module/lazy',
    function() {
        var foo = require('foo');
        var bar = require('bar');
    });

Available Plugins

Below is a list of plugins that are currently available:

Core plugins
- raptor-optimizer-require: Node.js-style require for the browser (similar to browserify)
- raptor-optimizer-minify-css: Minify CSS files using sqwish
- raptor-optimizer-minify-js: Minify JavaScript files using uglify-js
- raptor-optimizer-resolve-css-urls: Replace each resource URL in a CSS file with an optimized resource URL
Third-party plugins
- raptor-optimizer-dust: Compile Dust template files to JavaScript
- raptor-optimizer-handlebars: Compile Handlebars template files to JavaScript
- raptor-optimizer-less: Compile Less files to CSS
- raptor-optimizer-rhtml: Compile Raptor Template files to JavaScript
- raptor-optimizer-sass: Compile Sass files to CSS
- raptor-optimizer-stylus: Compile Stylus files to CSS

To use a third-party plugin, you must first install it using npm install. For example:

npm install raptor-optimizer-less --save

If you create your own plugin please send a Pull Request and it will show up above. Also, do not forget to tag your plugin with raptor-optimizer-plugin and raptor-optimizer in your package.json so that others can browse for it using npm

Optimizer Taglib

If you are using Raptor Templates or Dust you can utilize the available taglib for the Raptor Optimizer to easily optimize page dependencies and embed them into your page.

Using the Optimizer Taglib with Raptor Templates

npm install raptor-optimizer --save
npm install raptor-templates --save

You can now add the optimizer tags to your page templates. For example:

<optimizer-page name="my-page" package-path="./optimizer.json"/>

<!doctype html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>Test Page</title>
    <optimizer-head/>
</head>
<body>
    <h1>Test Page</h1>
    <optimizer-body/>
</body>
</html>

You will then need to create an optimizer.json in the same directory as your page template. For example:

optimizer.json:

{
    "dependencies": [
        "jquery.js",
        "foo.js",
        "bar.js",
        "style.less"
    ]
}

Now when the page renders you will get something similar to the following:

<!doctype html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>Test Page</title>
    <link rel="stylesheet" type="text/css" href="/static/my-page-85e3288e.css">
</head>
<body>
    <h1>Test Page</h1>
    <script type="text/javascript" src="/static/bundle1-6df28666.js"></script>
    <script type="text/javascript" src="/static/bundle2-132d1091.js"></script>
    <script type="text/javascript" src="/static/my-page-1de22b65.js"></script>
</body>
</html>

The optimized result is cached so you can skip the build step!

You can also configure the default page optimizer used by the optimizer tags:

require('raptor-optimizer').configure({...});

Using the Optimizer Taglib with Dust

You should follow the same steps as above, except you must install the dustjs-linkedin module and then use require('raptor-optimizer/dust').registerHelpers(dust) to register the helpers:

Install required dependencies:

npm install raptor-optimizer --save
npm install dustjs-linkedin --save

var dust = require('dustjs-linkedin');
require('raptor-optimizer/dust').registerHelpers(dust);

Finally, in your Dust templates you can use the new optimizer helpers as shown below:


{@optimizer-page name="my-page" packagePath="./optimizer.json" /}

<!doctype html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>Test Page</title>
    {@optimizer-head /}
</head>
<body>
    <h1>Test Page</h1>
    {@optimizer-body /}
</body>
</html>

Extending the Raptor Optimizer

Only read below if you are building plugins or transforms to further enhance the raptor-optimizer module.

Custom Plugins

A plugin can be used to change how the optimizer operates. This includes the following:

Register a custom dependency to support dependencies that compile to JS or CSS
- Examples:
  - Register a dependency handler for "less" files to compiles Less to CSS
  - Register a dependency handler for "rhtml" files to compiles Raptor Template files to JS
Register a custom bundle writer
- Examples:
  - Upload bundles to a resource server that backs a CDN instead of writing them to disk
Register output transforms
- Examples:
  - Add an output transform to minify JavaScript code
  - Add an output transform to minify CSS code
  - Add an output transform to remove console.log from JS code
  - Add an output transform to resolve image URLs in CSS files
Configure the optimizer
- Examples:
  - Allow a plugin to automatically configure the optimizer for production usage

A plugin is simply a Node.js module that exports a function with the following signature:

/**
 * A plugin for the Raptor Optimizer
 * @param  {raptor-optimizer/lib/PageOptimizer} optimizer An instance of a PageOptimizer that can be configured
 * @param  {Object} The plugin configuration provided by the user
 */
module.exports = function(pageOptimizer, config) {
    // Register dependency types:
    pageOptimizer.dependencies.registerJavaScriptType('my-js-type', require('./dependency-my-js-type'));
    pageOptimizer.dependencies.registerStyleSheetType('my-css-type', require('./dependency-my-css-type'));
    pageOptimizer.dependencies.registerPackageType('my-package-type', require('./dependency-my-package-type'));

    // Add an output transform
    pageOptimizer.addTransform(require('./my-transform'));

    // Register a custom Node.js/CommonJS module compiler for a custom filename extension
    // var myModule = require('./hello.test');
    pageOptimizer.dependencies.registerRequireExtension('test', function(path, context, callback) {
        callback(null, "exports.sayHello = function() { console.log('Hello!'); }");
    });
};

Custom Dependency Types

There are three types of dependencies that are supported:

JavaScript dependency: Produces JavaScript code
CSS dependency: Produces CSS code
Package dependency: Produces a package of additional JavaScript and CSS dependencies

Each of these dependencies is described in the next few sections. However, it is recommended to also check out the source code of available plugins listed above (e.g. raptor-optimizer-less).

Custom JavaScript Dependency Type

If you would like to introduce your own custom dependency types then you will need to have your plugin register a dependency handler. This is illustrated in the following sample code:

module.exports = function myPlugin(optimizer, config) {
    optimizer.dependencies.registerJavaScriptType(
        'my-custom-type',
        {
            // Declare which properties can be passed to the dependency type
            properties: {
                'path': 'string'
            },

            // Validation checks and initialization based on properties:
            init: function() {
                if (!this.path) {
                    throw new Error('"path" is required');
                }

                // NOTE: resolvePath can be used to resolve a provided relative path to a full path
                this.path = this.resolvePath(this.path);
            },

            // Read the resource:
            read: function(context, callback) {
                var path = this.path;

                fs.readFile(path, {encoding: 'utf8'}, function(err, src) {
                    if (err) {
                        return callback(err);
                    }

                    myCompiler.compile(src, callback);
                });

                // NOTE: A stream can also be returned
            },

            // getSourceFile is optional and is only used to determine the last modified time
            // stamp and to give the output file a reasonable name when bundling is disabled
            getSourceFile: function() {
                return this.path;
            }
        });
};

Once registered, the above dependency can then be referenced from an optimizer.json as shown in the following code:

{
    "dependencies": [
        "my-custom-type: hello.file"
    ]
}

If a custom dependency supports more than just a path property, additional properties could be provided as shown in the following sample code:

{
    "dependencies": [
        {
            "type": "my-custom-type",
            "path": "hello.file",
            "foo": "bar",
            "hello": true
        }
    ]
}

Custom CSS Dependency Type

If you would like to introduce your own custom dependency types then you will need to have your plugin register a dependency handler as shown in the following sample code:

module.exports = function myPlugin(optimizer, config) {
    optimizer.dependencies.registerStyleSheetType(
        'my-custom-type',
        handler);
};

The handler argument for a CSS dependency has the exact same interface as a handler for a JavaScript dependency (described earlier).

Custom Package Type

A custom package dependency can be used to dynamically resolve additional dependencies at optimization time. The sample package dependency handler below illustrates how a package dependency can be used to automatically include every file in a directory as a dependency:

var fs = require('fs');
var path = require('path');

optimizer.dependencies.registerPackageType('dir', {
    properties: {
        'path': 'string'
    },

    init: function() {
        if (!this.path) {
            throw new Error('"path" is required');
        }

        this.path = this.resolvePath(this.path); // Convert the relative path to an absolute path

        if (fs.statSync(this.path).isDirectory() === false) {
            throw new Error('Directory expected: ' + this.path);
        }
    },

    getDependencies: function(context, callback) {
        var dir = this.path;

        fs.readdir(dir, function(err, filenames) {
            if (err) {
                return callback(err);
            }

            // Convert the filenames to full paths
            var dependencies = filenames.map(function(filename) {
                return path.join(dir, filename);
            });

            callback(null, dependencies);
        });
    },

    getDir: function() {
        // If the dependencies are associated with a directory then return that directory.
        // Otherwise, return null
        return this.path;
    }
});

Custom Output Transforms

Registered output transforms are used to process bundles as they are written to disk. As an example, an output transform can be used to minify a JavaScript or CSS bundle. Another example is that an output transform may be used to remove console.log statements from output JavaScript code. Transforms should be registered by a plugin using the pageOptimizer.addTransform(transform) method.

As an example, the following unhelpful transform will convert all JavaScript source code to upper case:

module.exports = function (pageOptimizer, pluginConfig) {
    pageOptimizer.addTransform({

        // Only apply to JavaScript code
        contentType: 'js', //  'css' is the other option

        // Give your module a friendly name (helpful for debugging in case something goes wrong in your code)
        name: module.id,

        // If stream is set to false then a String will be provided. Otherwise, a readable stream will be provided
        stream: false,

        // Do the magic:
        transform: function(code, contentType, dependency, bundle) {
            return code.toUpperCase();
        }
    });
};

Below is the streaming version of the same transform:

var through = require('through');

module.exports = function (pageOptimizer, pluginConfig) {
    pageOptimizer.addTransform({

        // Only apply to JavaScript code
        contentType: 'js', //  'css' is the other option

        // Give your module a friendly name (helpful for debugging in case something goes wrong in your code)
        name: module.id,

        stream: true, // We want the code to be streamed to us

        // Do the magic:
        transform: function(inStream, contentType, dependency, bundle) {
            return inStream.pipe(through(
                function write(data) {
                    this.queue(data.toUpperCase());
                }));
        }
    });
};

Sample Projects

raptor-samples/optimizer-cli: Sample usage of the command-line interface.
raptor-samples/optimizer-config: Sample app that demonstrates the use of a JSON config file.
raptor-samples/optimizer-async: Sample app that demonstrates asynchronous/lazy dependency loading.
raptor-samples/optimizer-js-api: Sample app that demonstrates how to use JavaScript API to optimize a page and inject the resulting head and body markup into a page.
raptor-samples/optimizer-taglib: Sample app that demonstrates the use of the optimizer taglib for Raptor Templates.
raptor-samples/optimizer-templates: Sample app that demonstrates the use of rendering the same templates on both the server and the client.
raptor-samples/optimizer-express: Sample app that demonstrates using the Raptor Optimizer at runtime as part of an Express server app.

Discuss

Please post questions or comments on the RaptorJS Google Groups Discussion Forum.

Contributors

Patrick Steele-Idem (Twitter: @psteeleidem)
Phillip Gates-Idem (Twitter: @philidem)

Contribute

Pull Requests welcome. Please submit Github issues for any feature enhancements, bugs or documentation problems.

License

Apache License v2.0

Published

Vulnerabilities

Links

Maintainers

Keywords

Readme

Raptor Optimizer

Table of Contents

Example

Design Philosophy

Features

Another Client-side Bundler?

Tutorials

Tutorial: Command Line Interface

Tutorial: JSON Configuration File

Tutorial: Asynchronous/Lazy Loading

Tutorial: JavaScript API

Tutorial: Optimizer Taglib

Tutorial: Client/Server Template Rendering

Tutorial: Runtime Optimization with Express

Installation

Usage

Command Line Interface

Configuration

Default Configuration

Complete Configuration

JavaScript API

Configuring the Default Page Optimizer

Optimizing a Page

Creating a New Page Optimizer

Dependencies

Conditional Dependencies

Enabling Extensions

Node.js-style Module Support

Bundling

Bundling Example

Asynchronous Module Loading

Available Plugins

Optimizer Taglib

Using the Optimizer Taglib with Raptor Templates

Using the Optimizer Taglib with Dust

Extending the Raptor Optimizer

Custom Plugins

Custom Dependency Types

Custom JavaScript Dependency Type

Custom CSS Dependency Type

Custom Package Type

Custom Output Transforms

Sample Projects

Discuss

Contributors

Contribute

License