Thalassa Consul

Overview

Thalassa Consul is a reworking of the Thalassa system of components. In this updated version, what was the Thalassa Server and Client has been replaced by a Consul cluster and agent respectively. Thalassa is primarily geared to enable continuous deployment scenarios through dynamic configuration of HAProxy load balancers and seamless, no connection drop A/B deploys. Thalassa is the node.js service that manages and controls an HAProxy server.

Thalassa exposes a REST API for configuring HAProxy and can dyanmically set HAProxy backends members based on the name and version of Consul registered services. Thalassa leverages HAProxy's ability to gracefully reload config without any interruption to user, in other words, without dropping any connections.

Thalassa has been updated to use @3rd-Eden's haproxy module to manage and control the HAProxy process.

HAProxy Fundamentals

Thalassa does not try to obfuscate HAProxy, and it's important to know the fundamentals of how HAProxy works to understand Thalassa. The API mirrors HAProxy's semantics. The HAProxy documentation contains a wealth of detailed information.

1. Frontends - A "frontend" section describes a set of listening sockets accepting client connections.

2. Backends - A "backend" section describes a set of servers to which the proxy will connect to forward incoming connections.

3. Members/Servers - Thalassa calls the servers that backends route to "members". In other words, members of a backend pool of servers.

4. Config file - At startup, HAProxy loads a configuration file and never looks at that file again. Thalassa manages this by re-templating a new config file and gracefully restarting the HAProxy process.

5. Stats Socket - a UNIX socket in stream mode that will return various statistics outputs and even allows some basic configuration tweaking like enabling and disabling existing backend members, setting weights, etc. Thalassa connects to this socket and provides realtime streaming stats over a web socket stream.

Installation

npm install thalassa-consul

Running

The easiest way to run Thalassa at this point is with the bin script from the command line. Thalassa is exposed as a module and can be used as such in your own application but you should have a close look at how the Hapi server and web socket stream is configured.

./node_modules/.bin/thalassa-consul 

Options

 ./node_modules/.bin/thalassa-consul --help
Options:
  --host               host to bind to                           [default: "0.0.0.0"]
  --port               port to bind to                           [default: 10000]
  --label              logical label for this thalassa
  --serviceReg         host of the service registry
  --serviceRegPort     port of the service registry              [default: 8500]
  --datacenter         datacenter for this thalassa instance
  --location           cloud location for this thalassa instance
  --haproxySocketPath  path to Haproxy socket file               [default: "/tmp/haproxy.status.sock"]
  --haproxyPidPath     path to  Haproxy pid file                 [default: "/var/run/haproxy.pid"]
  --haproxyCfgPath     generated Haproxy config location         [default: "/etc/haproxy/haproxy.cfg"]
  --templateFile       template used to generate Haproxy config  [default: "default.haproxycfg.tmpl"]
  --persistence        directory to save configuration
  --sudo               use sudo when starting haproxy
  --debug              enabled debug logging
  --dbPath             filesystem path for leveldb               [default: "./node_modules/thalassa-crowsnest/bin/db"] 

For example the command to run might look something like this (typically how I run locally):

./node_modules/.bin/thalassa-consul --haproxyCfgPath /tmp/haproxy.cfg --debug --persistence
'/tmp/thalassa.json' --templateFile dev.haproxycfg.tmpl --haproxyPidPath /tmp/haproxy.pid
--label 'myapp-dev'

Web UI

Thalassa provides a web UI that allows users to get a visual representation of the frontends/backends/member data related to their haproxy instance (via the 'overview' page), as well as some insight into the activity (haproxy config changes, online/offline events) occuring within Thalassa (via the 'activity' page).

The UI can be accessed on the port specified by the --port parameter (by default port 10000).

e.g. http://127.0.0.1:10000 would access the web UI on localhost at the default port of 10000

HTTP API

GET /frontends

Returns Array of frontend objects for all of the frontends configured for this Thalassa server.

For example:

[{
    "id": "frontend/myapp",
    "_type": "frontend",
    "name": "myapp",
    "bind": "*:8080,*:80",
    "backend": "live",
    "mode": "http",
    "keepalive": "default",
    "rules": [{
        "type": "header",
        "header": "host",
        "operation": "hdr_dom",
        "value": "staged.myapp.com",
        "backend": "staged"
    }],
    "natives": []
}]

GET /frontends/{name}

Gets a specific frontend by name. Expect a response status code of 200 otherwise 404.

PUT /frontends/{name}

Create or update a frontend by name. PUT with a Content-Type of application/json and a body like:

{
    "bind": "10.2.2.2:80,*:8080" // IP and ports to bind to, comma separated, host may be *
  , "backend": "foo"      // the default backend to route to, it must be defined already
  , "mode": "tcp"         // default: http, expects tcp|http
  , "keepalive": "close"  // default: "default", expects default|close|server-close
  , "rules": []           // array of rules, see next section
  , "natives": []         // array of strings of raw config USE SPARINGLY!!
};

Routing Rules

There are currently 3 types of rules that can be applied to frontends: path, url, and header.

Path rules support path, path_beg, and path_reg HAProxy operations

{ "type": "path" , "operation": "path|path_beg|path_reg" , "value": "favicon.ico|/ecxd/|^/article/[^/]*$" , "backend": "foo" // if rule is met, the backend to route the request to }

Url rules support url, url_beg, and url_reg HAProxy operations

{ "type": "url" , "operation": "url|url_beg|url_reg" , "value": "/bar" // value for the operation , "backend": "bar" // if rule is met, the backend to route the request to }

Header rules support hdr_dom with a entire value at this point

{ "type": "header" , "header": "host" // the name of the HTTP header , "operation": "hdr_dom" , "value": "baz.com" , "backend": "baz" // if rule is met, the backend to route the request to }

Natives

The natives property is an end around way to insert raw lines of config for front ends and backends. Use them sparingly but use them if you need them.

DELETE /frontends/{name}

Delete a specific frontend by name. Expect 200 or 404

GET /backends

Returns Array of backend objects for all of the backends configured for this Thalassa server.

For example:

[{
  "id": "backend/live",
  "_type": "backend",
  "key": "live",
  "type": "dynamic",
  "name": "classroom-ui",
  "version": "1.0.0",
  "balance": "roundrobin",
  "host": null,
  "mode": "http",
  "members": [{
    "name": "myapp",
    "version": "1.0.0",
    "host": "10.10.240.121",
    "port": 8080,
    "lastKnown": 1378762056885,
    "meta": {
      "hostname": "dev-use1b-pr-01-myapp-01x00x00-01",
      "pid": 17941,
      "registered": 1378740834616
    },
    "id": "/myapp/1.0.0/10.10.240.121/8080"
  },
  {
    "name": "myapp",
    "version": "1.0.0",
    "host": "10.10.240.80",
    "port": 8080,
    "lastKnown": 1378762060226,
    "meta": {
      "hostname": "dev-use1b-pr-01-myapp-01x00x00-02",
      "pid": 18020,
      "registered": 1378762079883
    },
    "id": "/myapp/1.0.0/10.10.240.80/8080"
  }],
  "natives": []
}]

PUT /backends/{name}

Create or update a backend by name. PUT with a Content-Type of application/json and a body like:

{
    "type" : "dynamic|static" 
  , "name" : "foo" // only required if type = dynamic
  , "version" : "1.0.0" // only required if type = dynamic
  , "balance" : "roundrobin|source" // defaults to roundrobin
  , "host" : "myapp.com"  // default: undefined, if specified request to member will contain this host header
  , "health" : {                 // optional health check
      "method": "GET"            // HTTP method
    , "uri": "/checkity-check"   // URI to call
    , "httpVersion": "HTTP/1.1"  // HTTP/1.0 or HTTP/1.1 `host` required if HTTP/1.1
    , "interval": 5000           // period to check, milliseconds
  }
  , "mode" : "http|tcp" // default: http
  , "natives": []  // array of strings of raw config USE SPARINGLY!!
  , "members" : [] // if type = dynamic this is dynamically populated based on role/version subscription
                   // otherwise expects { host: '10.10.10.10', port: 8080}
}

GET /backends/{name}

Gets a specific backend by name. Expect 200 else 404.

DELETE /backends/{name}

Delete a specific backend by name. Expect 200 or 404

POST /backends/{name}

Update a backends role and version Subscription with a Content-Type of application/json and a body like:

{
    "name": "myapp"   // app name registered to thalassa
  , "version": "1.1.0" // version to route to
}

name is actually optional. You may also just send the version:

{
    "version": "1.1.0"
}

GET /haproxy/config

Return the last know generated HAProxy config file contents that were written to the location of opts.haproxyCfgPath.

global
log 127.0.0.1 local0
log 127.0.0.1 local1 notice
daemon
maxconn 4096
user haproxy 
group haproxy 
stats socket /tmp/haproxy.status.sock user appuser level admin

defaults
  log global
  option dontlognull
  option redispatch
  retries 3
  maxconn 2000
  timeout connect 5000ms
  timeout client 50000ms
  timeout server 50000ms

listen stats :1988
  mode http
  stats enable
  stats uri /
  stats refresh 2s
  stats realm Haproxy\ Stats
  stats auth showme:showme


frontend myapp
  bind *:8080,*:80
mode http
default_backend live
option httplog
option http-server-close
option http-pretend-keepalive
acl header_uv7vi hdr_dom(host) myapp-staged.com
use_backend staged if header_uv7vi



backend live
  mode http
balance roundrobin
server live_10.10.240.121:8080 10.10.240.121:8080 check inter 2000
server live_10.10.240.80:8080 10.10.240.80:8080 check inter 2000

backend staged
  mode http
balance roundrobin
server staged_10.10.240.174:8080 10.10.240.174:8080 check inter 2000
server staged_10.10.240.206:8080 10.10.240.206:8080 check inter 2000

Known Limitations and Roadmap

Thalassa currently doesn't implement any type of authentication or authorization and at this point expects to be running on a trusted private network. This will be addressed in the future. Ultimately auth should be extensible and customizable. Suggestions and pull requests welcome!

License

Licensed under Apache 2.0. See LICENSE file.