Project Venice

Venice encapsulates a series of tools, practices and systems to make software development simpler, easier and more standardized.

It was born of questions such as: "Wouldn't it be great if we could easily deploy this thing for QA?" "How can we version these deliverables?" "Why is it so hard to test each PR?"

These are the 10 governing principles behind venice:

1. A build always start from a clean slate - dependent packages changed? New things need to be installed? All of these will stand up when starting each build from scratch.
2. Building and testing is a locally-reproducible step - wondering why a build works locally but fails on the CI? No problem: just run the very same environment locally and debug it.
3. Every PR is built and tested - get immediate feedback at the PR level. If a regression is introduced, it will show on the PR itself. No more guessing.
4. Functional branches are deployment-ready - when pull requests are merged into main branches, these branches’ builds become potentially deployable packages.
5. Multiple functional branches - in some cases, there are several work streams that may need to be deployed independently (i.e. maintenance vs. new features). Each functional branch represents a deployment-ready build.
6. Versioning is flexible but its absence is non-negotiable - each deployable build is properly versioned, tagged, and uploaded to github accordingly. The absence of versioning is not an option.
7. Deployment pipeline is a necessity - a build can be automatically deployed to a constant deployment environment or manually deployed in cases of QA or production environments. Need to roll some version back? Totally possible.
8. Multiple deployment environments per functional branch - besides having an automatically deployable environment, each functional branch can have many other environments making your system deployable on a cross-functional/cross-environment way.
9. Reuse of good practices and tools - the practices employed on venice are tested and proven. Build steps and deployment systems can be reused and adapted over and over across projects.
10. You do the creative work, bots do the heavy lifting - focus on creating the best code out there and leave the heavy lifting of building, reporting and deploying to bots.

Installing the Venice CLI (Basic)

Venice relies on other opens source softwares to run. If you are a developer you probably already have those installed, if you not you will need (in order):

Basic installation:

• Download node v6.x.x from the official website and install it. This installation will also install npm for you.
• Download git (latest) from the official website and install it.

Advanced installation:

• brew(latest) Installation Instructions
• nvm (latest) Install running: brew install nvm
• node (v6.x.x) and npm (v3.x.x) Install running: E.g nvm install 6.7.0
• git (v2.5.0) Install running brew install git

Finally, after following either the basic or advanced installation guide, install venice running:

npm install -g @workco/venice

Venice relies heavily on docker instances, you may need to install docker (version 1.12.1) and docker-compose (version 1.8.0) for some features. You can download and install here. If you are interested in development of Venice CLI, read the section Other venice dependencies (developers) at the end of this document.

Overview of Venice commands

Venice help

If you need help, you can print instructions of the available commands using:

venice -h

ProTip: You can print the help of venice subcommands as well (e.g. venice login -h, venice create -h, etc.)

Login

Venice requires you to use your github user. As such, it will need access to your github account. Check at how to create an access token for command-line usage on github's documentation. We recommend creating a token with all access rights.

In order to login to Venice you simply run:

venice login --server <server> --user <user> --token <token>

Where:

• <server> is your Venice server (i.e. venice.mydomain.com)
• <user> is your Github users (ie. masterdeveloper)
• <token> is your Github access token (ie. 974bq13448d70ca3fae0e9cd8d02d9d1d68b3ab5)

Once logged in, all your subsequent commands will be sent to your Venice server.

DevTip: The credentials are saved in a dotfile .venice in the user home directory.

Logout

In order to log out, simply:

venice logout

Your credentials will be deleted and you will need to login before using other venice commands.

Builds

Venice call "builds" published tags on GitHub. A build is created automatically when a push is done in one of the monitored branches (explained below). In order to see all published tags/releases use the following git command, inside the project folder.

git log -n 10 --oneline --decorate --tags --no-walk

Deploy

The command to deploy a project is:

venice deploy --build <build> --env <env> --branch <branch>

Where:

• <build> is a tag (i.e. feature-x-1.0.0-build.3)
• <env> is one of the environments defined in venice.json (i.e. qa)
• <branch> (Optional) is the branch that should be deployed. If not provided, it will be inferred from the build name

Upon success you will receive a message that the build process has started.

Create

Once logged in, you can create Venice-based project with:

$ venice create --name <name>

Where:

• <name> is the name of the project you want to create

This should be executed only once, during the creation of each project.

Example:

$ venice create my-proj
created my-proj
https://github.com/companyorg/my-proj

$ cd my-proj/

$ ls -la
total 8
drwxr-xr-x  3 tiagoluchini  staff   102 Sep  2 22:59 .
drwxr-xr-x  8 tiagoluchini  staff   272 Sep  2 22:59 ..
-rw-r--r--  1 tiagoluchini  staff  1184 Sep  2 22:59 venice.json
-rw-r--r--  1 tiagoluchini  staff   474 Sep  2 22:59 docker-compose.yml
-rw-r--r--  1 tiagoluchini  staff   975 Sep  2 22:59 Dockerfile

Fine tuning your project

As you can see, the my-proj folder was created alongside a GitHub repo. Inside the project folder a few configuration files were also created.

The file called venice.json is the main touch of point between your project and the Venice system. The full documentation for venice.json can be found here.

The other two files (Dockerfile and docker-compose.yml) are respectively your build system and your services description. We'll get to those soon.

For now take a look at the basic document created by venice create:

$ cat venice.json
{
  "description": {
    "project": "my-proj",
    "type": "smart"
  },
  "build": {
    "steps": [
      "builder"
    ],
    "artifacts": {
      "path": "dist/"
    },
    "versioning": {
      "type": "npm"
    },
    "branches": {
      "master": {
        "version": "1.0.0"
      }
    }
  },
  "deploy": {
    "service": "web-server",
    "environments": {
      "cd": "auto",
      "qa": "manual"
    }
  },
  "reporter": {
    "type": "slack",
    "channel": "venice-test-bot"
  }

The file is divided in four major sections:

• description: generic details about the project
• build: instructions on how to build, branches being monitored, test and where artifacts are
• deploy: how to version, where to deploy and which branches need to be monitored for deployment
• reporter: how does the bots communicate back with users

Log

All the following commands will be executed based on the project folder that you are calling them. There are three options to see the build output:

• venice log (will run venice log --live by default)
• venice log --live

Live log for the current build. Running this command will block your terminal and print the log messages as they are appear during the build step. When you are done, press ctrl + c to have your terminal back.

• venice log --pr (will return the log of the last PR as default)
• venice log --pr <PR#>
• venice log --build (will return the log of the last push as default)
• venice log --build <tag>

Constant build/test settings

The build section of the venice.json file specifies the steps the build bot will follow to check, build, and test your project. Each step is the name of a docker-compose.yml service. Each step is run in sequence and only if the previous step ran successfully.

The default docker-compose.yml file looks like this:

$ cat docker-compose.yml
workco-commit-checker:
  image: workco-commit-checker
builder:
  build: .
  command: ['/tmp/run.sh']
  dockerfile: ./docker/Dockerfile-builder
  volumes:
    - .:/workdir
unit-tester:
  image: busybox
  volumes:
    - /disk
web-server:
  build: .
  dockerfile: ./docker/Dockerfile-web-server

TBD: validate this file

TBD: describe better some of the sessions/settings

TBD: show different possibilities

Other venice dependencies (developers)

In order to develop for venice-cli you will need:

• All dependencies of section Installing the Venice CLI (Basic)
• yarn (latest) Install with npm install -g yarn

Development instructions:

• Clone the Venice CLI repository
• To install the dependencies run: yarn
• Venice CLI is written in ES6, so in order to run its code to test locally during development, you need to run npm run build. This will create a dist folder with ES5 JS and then you can node ./dist/venice.js

ProTip: alias venice="npm run build && node dist/venice.js" This creates a temporary venice alias that will build and run the current code without cluttering your command line of commands. Very helpful when developing a CLI.

The Venice Server

Venice CLI relies on a running server. More documentation about setting up a server is found at the Venice core documentation.

Deployment settings

TBD: explain versioning

TBD: explain target

TBD: exaplin branch monitoring

TBD: explain manual releasing

All project repos

https://github.com/workco/venice-cli https://github.com/workco/venice-core