express-api-proxy

High performance streaming API reverse proxy for Express. Supports caching, environment variable substitution of sensitive keys, authentication, and response transformations.

Usage

var redis = require('redis');
var apiProxy = require('express-api-proxy');

require('redis-streams')(redis);

app.all('/proxy', apiProxy({
	cache: redis.createClient(),
	ensureAuthenticated: false, 
	endpoints: [
		{
			pattern: /\public/,
			maxCacheAge: 60 * 10 // cache responses for 10 minutes
		},
		{
			pattern: /\/secure/,
			ensureAuthenticated: true,
			cache: false
		}
	]
});

Client Code

var params = {
	api_key: '${SOMEAPI_API_KEY}',
	api_secret: '${SOMEAPI_API_SECRET}'
};

$.ajax({
	url: '/proxy',
	data: {
		url: 'http://someapi.com/api/secure?' + $.param(params);
	},
	statusCode: {
	   	200: function(data) {
	   		console.log(data);
	   	},
		401: function() {
		  console.log("unauthorized api call");
		}
	}
});

Options

cache

Cache object that conforms to the node_redis API. Can set to false at an endpoint level to explicitly disable caching for certain APIs. See Cache section below for more details.

cacheMaxAge

The duration to cache the API response. If an API response is returned from the cache, a max-age http header will be included with the remaining TTL.

envTokenStartSymbol

The start symbol that represents an environment variable token. Defaults to ${.

envTokenEndSymbol

The symbol representing the end of an environment variable token. Defaults to }.

ensureAuthenticated

Ensure that there is a valid logged in user in order to invoke the proxy. See the Ensure Authenticated section below for details.

userAgent

The user agent string passed as a header in the http call to the remote API. Defaults to "express-api-proxy".

cacheHttpHeader

Name of the http header returned in the proxy response with the value "hit" or "miss". Defaults to "Express-Api-Proxy-Cache".

envVariableFn

Function for providing custom logic to return the value of an environment variable. Useful for reading values from alternative stores to process.env. Function takes two arguments: req and key. If omitted, process.env[key] is used by default.

endpoints

Allows overriding any of the above options for a specific remote API endpoint based on a RegExp pattern match.

endpoints: [
	{
  		pattern: /api.instagram.com/,
  		cacheMaxAge: 4000
  	}
]

Environment Variables

In order to keep sensitive keys out of client JavaScript, the API proxy supports replacing environment variables in the URL, HTTP headers, and the JSON body of a POST or PUT request. Environment variables that are blank or undefined will result in a 400 (Bad Request) HTTP response.

URL

$.ajax({
	url: '/proxy',
	data: {
		url: 'https://someapi.com/api/endpoint?api_key=${SOME_API_API_KEY}'
	}
});

HTTP header

For HTTP Basic authentication, the value of the SOME_API_BASIC_AUTH_TOKEN environment variable would be "username:password" Base64 encoded. The X-Authorization header is used to avoid colliding with a Authorization header required to invoke the /proxy URL. The proxy will strip off the X- prefix when invoking the remote API.

$.ajax({
	url: '/proxy',
	data: {
		url: 'https://someapi.com/api/endpoint'
	},
	headers: {
		'X-Authorization': 'Basic ${SOME_API_BASIC_AUTH_TOKEN}'
	}
});

POST or PUT body

$.ajax({
	url: '/proxy?url=' + encodeURIComponent('https://someapi.com/api/endpoint'),
	type: 'POST',
	data: {
		api_key: '${SOME_API_API_KEY}',
		params: {}
	}
});

User Variables

For any environment variable with the prefix USER_, the proxy will replace it with a matching property on the req.user object. It does so by stripping off the USER_ prefix, then converting the remainder from underscore case to camel case. For example the variable USER_ACCESS_TOKEN would get substituted by req.user.accessToken. If there is req.user or req.user.accessToken is undefined, a 400 response is returned.

Custom Environment Variable Store

By default the proxy looks up environment variables using process.env[key], but in some cases it is desirable to provide a custom environment variable implementation. A envVariableFn option can be provided that accepts the req and key to perform custom logic:

options: {
	envVariableFn: function(req, key) {
		return ...;
	}
}

Caching

For APIs whose data does not frequently change, it is often desirable to cache responses at the proxy level. This avoids repeated network round-trip latency and can skirt rate limits imposed by the API provider. Caching can be set as a global option, but more commonly you'll want to control it for each individual endpoint.

The object provided to the cache option is expected to implement a subset of the node_redis interface, specifically the get, set, setex, exists, del, and ttl commands. The node_redis package can be used directly, other cache stores require a wrapper module that adapts to the redis interface.

As an optimization, two additional functions, readStream and writeThrough can be implemented on the cache object to allow direct piping of the API responses into and out of the cache. This avoids buffering the entire API response in memory. For node_redis, the redis-streams package augments the RedisClient with these two functions. Simply add the following line to your module before the proxy middleware is executed:

var redis = require('redis');

require('redis-streams')(redis);
// After line above, calls to redis.createClient will return enhanced
// object with readStream and writeThrough functions.

app.all('/proxy', apiProxy({
	cache: redis.createClient(),
	endpoints: [
		pattern: /blog\/posts/,
		maxCacheAge: 60 * 5 // 5 minutes
	]
});

HTTP Headers

If an API response is served from the cache, the max-age header will be set to the remaining TTL of the cached object. The proxy cache trumps any HTTP headers such as Last-Modified, Expires, or ETag, so these get discarded. Effectively the proxy takes over the caching behavior from the origin for the duration that it exists there.

Ensure Authenticated

It's possible restrict proxy calls to authenticated users via the ensureAuthenticated option property which can be specified at the top level, or on a specific object in the endpoints array.

app.all('/proxy', apiProxy({
	endpoints: [
		{
			pattern: /\/secure/,
			ensureAuthenticated: true
		}
	]
});

The proxy does not perform authentication itself, that task is delegated to other middleware that executes earlier in the request pipeline which sets the property req.ext.isAuthenticated. If the ensureAuthenticated is true and req.ext.isAuthenticated !== true, a 401 (Unauthorized) HTTP response is returned.

Note that this is different than authentication that might be enforced by the remote API itself. That's handled by passing environment variables as discussed above.

Transforms

The proxy supports transforming the API response before piping it back to the caller. Transforms are functions which return a Node.js transform stream. The through2 package provides a lightweight wrapper that makes transforms easier to implement.

Here's a trivial transform function that simply appends some text

module.exports = fn = function(options) {
	return through2(function(chunk, enc, cb) {
		this.push(chunk);
		cb();
	}, function(cb) {
		this.push(options.appendText);
		cb();
	});
};

If the transform needs to change the Content-Type of the response, a contentType property can be declared on the transform function that the proxy will recognize and set the header accordingly.

module.exports = function(options) {
	var transform = through2(...);
	transform.contentType = 'application/json';
	return transform;
};

See the markdown-transform for a real world example. For transforming HTML responses, the trumpet package, with it's streaming capabilities, is a natural fit.

License

Licensed under the Apache License, Version 2.0. See the top-level file LICENSE.txt and (http://www.apache.org/licenses/LICENSE-2.0).