npm package discovery and stats viewer.

Discover Tips

  • General search

    [free text search, go nuts!]

  • Package details

    pkg:[package-name]

  • User packages

    @[username]

Sponsor

Optimize Toolset

I’ve always been into building performant and accessible sites, but lately I’ve been taking it extremely seriously. So much so that I’ve been building a tool to help me optimize and monitor the sites that I build to make sure that I’m making an attempt to offer the best experience to those who visit them. If you’re into performant, accessible and SEO friendly sites, you might like it too! You can check it out at Optimize Toolset.

About

Hi, 👋, I’m Ryan Hefner  and I built this site for me, and you! The goal of this site was to provide an easy way for me to check the stats on my npm packages, both for prioritizing issues and updates, and to give me a little kick in the pants to keep up on stuff.

As I was building it, I realized that I was actually using the tool to build the tool, and figured I might as well put this out there and hopefully others will find it to be a fast and useful way to search and browse npm packages as I have.

If you’re interested in other things I’m working on, follow me on Twitter or check out the open source projects I’ve been publishing on GitHub.

I am also working on a Twitter bot for this site to tweet the most popular, newest, random packages from npm. Please follow that account now and it will start sending out packages soon–ish.

Open Software & Tools

This site wouldn’t be possible without the immense generosity and tireless efforts from the people who make contributions to the world and share their work via open source initiatives. Thank you 🙏

© 2024 – Pkg Stats / Ryan Hefner

nodesi

v2.0.1

Published

ESI: the good parts in node.js

Downloads

30,352

Readme

Tests GitHub license

What is this?

It's a subset of Edge Side Include standard implemented with promise-based interface.

What problem does it solve?

Let's say you want to use ESI in your project, but also want to retain good developer experience. Rather than having to configure Varnish or Ngnix to take care of server-rendered ESI tags locally you can simply pass the server output through esi.process function right before pushing it out to the client.

    const response = obtainServerResponseWithEsiTags();
    return Promise.resolve()
        .then(function() {
            if(process.env.NODE_ENV !== 'production') {
                return esi.process(response);
            }
            return response;
        });

It also improves code mobility - if for whatever reason you decide to move from ESI-enabled environment into one that doesn't support it (yet?), all you have to do is to process the response directly on the server. This module should be performant enough for that use case.

Features

  • Support for <esi:include> tags
    • support for alt as fallback URL
  • Out of the box Express support
  • Custom logging
  • Lots of good stuff like caching, retires, request collapsing and such provided by Good Guy HTTP

…and more, take a look at test cases for complete list.

Advanced ESI features

nodesi does not support the entire ESI spec, but aims to provide a usable subset. This includes, of course <esi:include src="…">, but also some more advanced features like:

Fallback URL with alt

<esi:include src="http://example.com/1.html" alt="http://bak.example.com/2.html"/>

Will try to include http://example.com/1.html first, and if that fails, fall back to http://bak.example.com/2.html. If both requests fail, the standard error handling described below will kick in.

Installation

npm install nodesi

Usage

Basic:

    const ESI = require('nodesi');

    const esi = new ESI({
        allowedHosts: ['http://full-resource-path']
    });
    esi.process('<esi:include src="http://full-resource-path/stuff.html" />').then(function(result) {
        // result is a fetched html
    });

As Express middleware:

    const esiMiddleware = require('nodesi').middleware;
    const app = require('express')();

    // inject the middleware before your route handlers
    app.use(esiMiddleware());

All the ESI constructor options described below are also applicable for the middleware function. Just pass them like that: esiMiddleWare({baseUrl: ..., allowedHosts: [...]});

If you'd like to pass options like headers to ESI middleware, use req.esiOptions object:

...
    app.use(esiMiddleware());

    app.get('/example', function(req, res) {
        req.esiOptions = {
            headers: {
                'Authorization': 'Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ=='
            }
        };
        res.render('example');
    });

If you'd like to adjust the baseUrl dynamically, use req.esiOptions object:

...
    app.use(esiMiddleware());

    app.get('/example', function(req, res) {
        req.esiOptions = {
            baseUrl: req.url
        };
        res.render('example');
    });

With base URL for relative paths:

    const ESI = require('nodesi');

    const esi = new ESI({
        baseUrl: 'http://full-resource-path'
    });
    esi.process('<esi:include src="/stuff.html" />').then(function(result) {
        // result is a fetched html
    });

With headers:

    const ESI = require('nodesi');

    const esi = new ESI({
        baseUrl: 'http://full-resource-path'
    });
    esi.process('<esi:include src="/stuff.html" />', {
        headers: {
            'Authorization': 'Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ=='
        }
    }).then(function(result) {
        // result is a fetched html
    });

With a custom HTTP client

By default, nodesi uses a native fetch to fetch the data.

It uses a limited set of features of the fetch API, so it's easy to provide a custom fetch implementation.

The required subset of the fetch interface looks like this in TS types:

type Options = {
    headers: Record<string, string>; // HTTP request headers to be attached
    // plus any other custom options you set:
    //   * calling esi.process
    //   * adding req.esiOptions = { ... }
};

type Response = {
    status: number; // http status code: 200, 400, 500, etc
    statusText: string; // status description
    text: () => Promise<string>; // returns the fetched text content
};

// https://developer.mozilla.org/en-US/docs/Web/API/fetch
type FetchFunction = (resource: string, options: Options) => Promise<Response>;
    const ESI = require('nodesi');

    const esi = new ESI({
        httpClient: (url, options) => {
            return Promise.resolve({
                status: 200,
                statusText: 'ok', // this is used only when status >= 400
                text: () => Promise.resolve('custom static response'),
            });
        },
    });

Security

Since this module performs HTTP calls to external services, it is possible for a malicious agent to exploit that, especially if content of a esi:include tag can be provided by user.

In order to mitigate that risk you should use allowedHosts configuration option. It's supposed to be a list of trusted hosts (protocol + hostname + port), represented as strings or regular expressions.

Example:

const esi = new ESI({
    allowedHosts: ['https://exact-host:3000', /^http(s)?:\/\/other-host$/]
});

If you're using baseUrl option then it's host will automatically be added to allowedHosts.

In case some url gets blocked you'll receive an error in your onError handler (see below) with blocked property set to true.

Error handling

You can provide onError callback to a ESI constructor. It will recieve two arguments: source URL and error object.

It should return a string that will be put in place of errorous content.

Example

    const esi = new ESI({
        onError: function(src, error) {
            if(error.statusCode === 404) {
                return 'Not found';
            }
            return '';
        }
    });

Logging

It's a common anti-pattern that libraries write to stdout w/o users permission.

We want to be nice so you can provide your own logging output with logTo configuration option.

It's expected to be an object with "write" method on it that accepts a single string.

Examples

Logging to a custom object

    const esi = new ESI({
        logTo: {
            write: function(log) {
                // do some stuff with log string here
            }
        }
    });

Logging to a standard output (same as console.log):

    const esi = new ESI({
        logTo: process.stdout
    });

Logging to a file (possible, but please don't do that):

    const logFile = require('fs').createWriteStream('./log.txt');
    const esi = new ESI({
        logTo: logFile
    });

Decoding of the ESI url

By default url passed as an argument in ESI tag gets decoded.

You might want to not have it decoded from some purposes, so you can pass decodeUrl: false config item.

Example

    const ESI = require('nodesi');

    const esi = new ESI({
        baseUrl: 'https://example.com',
        decodeUrl: false,
    });
    esi.process('<esi:include src="/path?foo=bar&amp;baz=bat" />').then(function(result) {
        // result is a fetched content
        // when decodeUrl is set to false, https://example.com/path?foo=bar&amp;baz=bat will be fetched
        // when decodeUrl is set to true or not set, https://example.com/path?foo=bar&baz=bat will be fetched
    });

Performance testing

You can run performance tests with npm run perf /test and npm run perf /noop that will test the base performance of your system without nodesi.

License

nodesi is made available under the conditions of the ISC license