Devops Tools

Provides CLI tools and a container runtime for use in our pipelines.

Devops CLI

Installation

npm i -g @viewdo/devops-cli

Commands

Usage: opsctl [options] [command]

Options:
  -h, --help                                          display help for command

Commands:
  docker-publish|dp [options]                         Push a docker image to docker hub.
  gcp-auth|gcr-auth [options]                         Authenticate gcloud with using a keyfile.
  gcr-publish|gp [options]                            Push a docker image to gcr.
  get-namespace|ns <branch>                           Translate a namespace from the passed in branch.
  git-tag|tag [options] [version]                     Tags git with the version and build info
  git-version|version [options]                       Uses gitverion to write version.json to the clone directory.
  jira-release|release [options] <project> [version]  Create a release in Jira with the specified version, and release all done issues.
  kube-auth|auth [options]                            Runs the steps to authenticate with k8s in bitbucket.
  kube-build|kb [options] [source] [output]           Copies all files from .kube to ./kube and replaces all variables.
  run-pipeline|pipeline [options] [pipeline]          Run any pipeline within this organization.
  run-configure-environments|dns [options]            Run the DNS/Environments control pipeline on devops-dns-control main branch.
  variable-set|set-var [options] <key> <value>        Create or update a repo-level variable in the given repository
  variables-get|vars-get [options]                    Creates vars: VERSION, ASSEMBLY_VERSION, NAMESPACE & BUILD_ID (Requires: version.json).
  variables-replace|rv [options] [path]               Replaces environment $VARIABLES in text files at a given path (default: /kube)
  send-slack|slack [options] <text>                   Send a slack message to the #delivery-log channel.
  help|h

|command|description| |-|-| |get-vars|...| |set-var|...| |todo|...|

Configuration

Bitbucket Credentials (set-var)

Uses BitBucket Credentials as configured via environment variables.

From within bitbucket pipelines, we use two Bitbucket Workspace Variables to make these credentials globally accessible:

• BITBUCKET_APP_USER - the user who configured the their BitBucket App Password.
• BITBUCKET_APP_PASSWORD as generated by the user. It should be secured.

To create a new app password, you will need to grant the following permissions:

Kubernetes Deployments Credentials (kube-auth)

Currently we have a public control plane and deploy to it via Kubernetes Service Account (KSA).

The kube-auth command expects five values to be set and exposed in the pipeline as ENV vars.

From within bitbucket pipelines, we config these five ENV vars as Bitbucket Workspace Variables to make these credentials globally accessible:

• GCP_USER - the KSA user in the cluster with sufficient RBAC to deploy to destination namespaces, ex: bitbucket
• GCP_TOKEN_PROD - KSA token/secret, should be base64 encoded
• GCP_CLUSTER_PROD - name of the cluster in gcp, ex dxp-cluster (this might just be the name of the kube-context, and may not matter)
• GCP_CA_PROD = certificate authority, should be base64 encoded, must be retrieved from Google Console
• GCP_SERVER_PROD - control plane url like https://35.226.104.13, ip address can be found in Google Console

If you need to reconfigure these, you will need to get these values from the cluster.

Values from Inside Kubernetes Cluster

Ensure tha the KSA has been created in the repository tracking the the state of the cluster to be deployed to. This is the value that should be set as GCP_USER (it's a bit of a misnomer, it does NOT refer to a Google Service Account / GSA) At the time of writing this KSA exists and is a called bitbucket and lives in the kube-system namespace.

Export KSA token for GCP_TOKEN_PROD, it will be exported already base64 encoded

export KSA_USER=bitbucket
export TOKEN_NAME=`kubectl -n kube-system get serviceaccount/$KSA_USER -o jsonpath='{.secrets[0].name}'`
kubectl -n kube-system get secret $TOKEN_NAME -o jsonpath='{.data.token}'

Values from Google Console

The rest of the settings (GCP_CA_PROD, GCP_CLUSTER_PROD,GCP_SERVER_PROD) need to be retrievied from Google Console. First navigate to the Kubernetes area, then find the cluster and view the cluster details. It should look like the following:

For the Certifcate Authority (CA), you will need to retrieve this via Gooogle Console AND THEN base64 encode it.

Jira Integration (jira-release)

Webhook Overrides via ENV VARs:

• JIRA_RELEASE_HOOK_COMPONENT - Jira Automation hook url to call when using -c <component> argument
• JIRA_RELEASE_HOOK_PROJECT - Jira Automation hook url when no component argument passed

# note the -n flag ensures we don't encode a trailing slash
echo -n "<paste-certificate>" | base64

Devops Container