exp-deploy

Simple deployment handling using the npm script feature.

Requirements

• Linux or OSX
• Applications are managed with pm2 and a config/pm2.json file must be present.
• Passwordless ssh access to web user is required to all servers.
• node version is managed via nvm, so a .nvmrc file must present at project root.

Installation

Just add exp-deploy to your devDependencies.

NOTE: make sure you don't use dependencies, or else shrinkwrap will block the package from being installed.

Configuration

All configuration of exp-deploy is done right inside your package.json.

Define environments

Add an "exp-deploy" configuration to your package.json, describing your different environments:

"config": {
  "exp-deploy": {
    "notifications": {"host": "somehost", "port": 2345},
    "environments": {
      "production": {
        "pm2conf": "config/production.json",
        "servers": ["prod-server-1", "prod-servgr-2"],
        "testEndpoints": ["http://localhost:1332/abc", "http://localhost:1332/def"]
      },
      "stage": {"servers": ["stage-server"]},
      "test": {"servers": ["test-server"]}
    }
  }
}

Valid options are

• name - name to use for the app (defaults to npm package name).
• revisionFile - file where the current revision is written before deploy. Defaults to config/revision
• notifications.host - host for deploy notifications, see "Notifications" below.
• notifications.port - port for ditto.
• environments - list of environments.
• [environment].pm2conf - Relative path to pm2 configuration file. Defaults to config/pm2.json.
• [environment].servers - list of servers to deploy to.
• [environment].testEndpoints - list of endpoints to call to ensure that the deploy was successful. Call is made from the server being deployed to.
• [environment].waitForLoadbalancer - Remove "alive" flag before deploying and wait for the loadbalancer to notice the server is up again before proceeeding with the next. (defaults to "false").

Add deploy tasks

Add entries to the scripts section to define your deployment tasks.

"scripts": {
  "deploy-production": "exp-deploy production",
  "deploy-staging": "exp-deploy staging",
  "deploy-test": "exp-deploy test"
}

Running

Invoke just like any other npm script

prompt> npm run deploy-staging

Server override

If you want to override the config and deploy to a specific server, set the EXP_SERVERS variable.

prompt> EXP_SERVERS="prod-server-3" npm run deploy-production

Hooks

To define deploy hooks, we utilze the pre/post feature built into the npm script task. You can define your own scripts and/or use the ones that come with exp-deploy described below.

Pre

• exp-ensure-unmodified - ensures that everything is commited to git
• exp-ensure-master - ensure that we deploy only from the master branch.
• exp-ensure-tests - ensure that all tests are running.
• exp-ensure-version-tag - ensures that the current version of the app has not been deployed before. Should be used in conjunction with exp-set-version-tag post hook described below

Post

• exp-set-tag - sets a "deployed" tag in git to keep track of what is running in production.
• exp-set-version-tag - sets a tag in git named after the version numnber in package.json.

Example

"scripts": {
  "deploy-test": "exp-deploy test",
  "deploy-staging": "exp-deploy staging",
  "deploy-production": "exp-deploy production",
  "predeploy-production": "exp-ensure-tests && exp-ensure-unmodified && exp-ensure-master"
  "postdeploy-production": "exp-set-tag && scripts/send-message-to-slack.sh"
}

For staging and test, just deploy without further actions. For production, ensure that tests run ok, everything is commited to git and that we are on the master branch; afterwards set deploy tag and notify slack using custom script.

Building project

The deploy will use npm pack to package the application, which in turn will trigger the npm prepublish command, which can be set in package.json. The deploy environment will be sent into the build process as NODE_ENV, allowing for environment specific builds.

Notifications

If you specify the notifications host/port, exp-deploy will send notifications to the given endpoint. This is useful for tracking deployments in the organisation.

The following HTTP POST requests will be made for each deploy:

• http://url/deplpoy-started - on script start
• http://url/deplpoy-ended - on script exit with zero status
• http://url/deplpoy-failed - on script exit with non-zero status

url above will be resolved from the notification host/port settings in your config

The following url parameter are added to each call.

• app - application name
• env - node env (development, test, production etc)
• commit - commmit sha
• ts - unix timestamp when deploy was started

If a notification call fails, exp-deploy will ignore this and proceed with the deploy as usual. Downtime for the notification endpoint won't disrupt the entire deployment procedure.

pm2/nvm hack

The node version specified via nvm does not survive survive a server reboot. To guarantee the correct node version is retained, use the following config for all applications in the pm2 config file.

 "exec_mode": "fork_mode",
 "exec_interpreter": "@NVM_NODE@",

Using dirty black magic, exp-deploy will replace the @NMV_NODE@ token with an absolute path to the correct node version for the app.

NOTE: if you add this to an already running application, you need to do pm2 delete for it before running exp-deploy.

BD/XPR overrides

Exp-deploy will soon retire and be replaced with exp-containership when the move to our new xpr environment is finished. During the transistion period it might be useful to deploy to both the old and the new servers at the same time. Since the config might differ between xpr and bd it is possible to place an env file in the "config" directory which will overiide the default config for the bd servers.

Example

Create a config/livedata.json file that suits the xpr environment:

{
  "rabbit" : {
    "host": "livedata.rabbitmq-vip.service.consul.xpr.dex.nu",
    "exchange": "kafka", 
    ...
  }
}

Then create a config/livedata-bd.env file containing only the entries that differ for the BD servers:

rabbit.host=rabbit.host=rabbitmq-stage.expressen.se

This file will be copied to .env in the working dir for the BD servers. Env-files is an exp-config feature: https://www.npmjs.com/package/exp-config#overriding-configuration-values

Environment variables

These variables can control the deploy process:

• EXP_SERVERS - list of alternative servers to deploy to
• EXP_IGNORE_MASTER - to deploy non-master branch
• EXP_IGNORE_TESTS - skip running tests
• EXP_IGNORE_UNMODIFIED - deploy uncommitted changes
• EXP_IGNORE_VTAG - force deploy of already set git version tag
• EXP_SKIP_NOTIFY - don't send deploy notifications