Devtools CLI

A CLI to make developing the ScalingFunds platform easier

Why? What can it do?

There are many repetitive activities when developing locally, i.e.:

• Changing your gcloud environment from staging to demo
• Switching between the different Kubernetes clusters for staging, demo, etc.
• Accessing the staging database from your local machine
• Looking at staging logs right from within your VS Code terminal to debug problems

For all of these activities, using just the raw gcloud or kubectl commands would be quite cumbersome and repetitive. For example, the staging environment isn't really called exactly 'staging' on gcloud but has a more random ID such as 'staging-1827369'. So every time you wanted to run a command against staging, you would first have to get the full staging ID, then attach it via a --project flag to your respective gcloud command etc.

The main purpose of these devtools is to save you time and allowing you to develop in a more straightforward way.

A good example: Connecting to a remote database

Instead of having to remember this monster of a command:

docker run --volume /tmp/cloudsql:/cloudsql \
           --volume ~/.config:/root/.config \
           --publish 127.0.0.1:5432:5432 \
           gcr.io/cloudsql-docker/gce-proxy:1.12 /cloud_sql_proxy \
           -instances=ace-sandbox-341612:europe-west3:sandbox-europe-west3-ace-core-core-jpdo=tcp:0.0.0.0:5432

The devtools make this as straightforward as:

sf db:proxy start

And boom, you're done 🚀

Concepts

What's a Context?

The sf ctx command shows your current context.

A 'context' refers to one of the runtime environments of our platform. For example, development refers to a developer's local machine, whereas staging or production refer to a Google Cloud Project in which the platform runs.

Technically, 'context' refers to a Google Cloud Project ID and a Kubernetes Cluster running inside that project.

How to add a new context?

If we add a new environment in the future (i.e. 'pre-production'), here's how you add devtools support for it:

1. Add the name of the environment (i.e. 'staging') to the config.environments array in ./src/config.js
2. Add the name of the environment (i.e. 'staging') to the ContextT type in ./src/types.js
3. Map the environment name (i.e. 'staging') to the full google cloud project ID (i.e. 'sf-staging-1234') in ./src/commands/ctx.js

Group and Project

The Gitlab group and project to be used for project-specific commands such as sf secrets:decrypt or sf logs are determined by the git origin url of the current directory.

Please make sure to cd into the correct directory before running commands

Setup

$ npm install -g @scalingfunds/devtools
$ sf COMMAND
running command...
$ sf (-v|--version|version)
@scalingfunds/devtools/3.0.0 linux-x64 node-v16.19.1
$ sf --help [COMMAND]
USAGE
  $ sf COMMAND
...

Troubleshooting

When checking out this repo for the first time and trying to run yarn report:health, you might run into a known issue with commitlint: https://github.com/conventional-changelog/commitlint/issues/589

The workaround for this is to touch .git/COMMIT_EDITMSG, and then the commitlint check should work 🔧

Development

Follow these steps to work locally on our devtools

1. Run yarn start to start a nodemon watching the src folder for changes
2. Make some changes to the code
3. Try out your changed commands by running ./bin/run <command-name> (for example: ./bin/run ctx)

If you'd like to run commands outside of the devtools folder:

1. Uninstall the devtools as a global dependency - yarn global remove @scalingfunds/devtools
2. Run yarn link which will link a local sf binary from your latest local build
3. Try out your changes by running the sf command that you modified

Commands

• sf autocomplete [SHELL]
• sf ctx [ENVIRONMENT]
• sf db:get
• sf db:proxy [OPERATION]
• sf doctor
• sf help [COMMAND]
• sf install
• sf logs PROJECT
• sf secrets:check CONFIGPATH
• sf secrets:decrypt [CONTEXT]
• sf secrets:encrypt [CONTEXT]
• sf secrets:set CONTEXT

sf autocomplete [SHELL]

display autocomplete installation instructions

USAGE
  $ sf autocomplete [SHELL]

ARGUMENTS
  SHELL  shell type

OPTIONS
  -r, --refresh-cache  Refresh cache (ignores displaying instructions)

EXAMPLES
  $ sf autocomplete
  $ sf autocomplete bash
  $ sf autocomplete zsh
  $ sf autocomplete --refresh-cache

See code: @oclif/plugin-autocomplete

sf ctx [ENVIRONMENT]

get or set the current context

USAGE
  $ sf ctx [ENVIRONMENT]

OPTIONS
  -k, --optimisticallyPickFirstK8sCluster  When no Kubernetes cluster is found that adheres to our naming convention,
                                           just pick the first one (assuming there is only one cluster in the project)

ALIASES
  $ sf ctx
  $ sf context

sf db:get

get information about the database instance, incl. credentials to connect

USAGE
  $ sf db:get

OPTIONS
  -s, --silent  Suppress terminal output (useful when using this command programmatically from other commands)

sf db:proxy [OPERATION]

start a cloudsql proxy to connect to an instance

USAGE
  $ sf db:proxy [OPERATION]

ARGUMENTS
  OPERATION  (start|stop) [default: start] start or stop the proxy

OPTIONS
  -p, --port=port  [default: 5432] port to run the proxy on

sf doctor

check if dependencies are installed

USAGE
  $ sf doctor

sf help [COMMAND]

Display help for sf.

USAGE
  $ sf help [COMMAND]

ARGUMENTS
  COMMAND  Command to show help for.

OPTIONS
  -n, --nested-commands  Include all nested commands in the output.

See code: @oclif/plugin-help

sf install

install system-level dependencies required for local development

USAGE
  $ sf install

sf logs PROJECT

get logs from services running on Kubernetes

USAGE
  $ sf logs PROJECT

ARGUMENTS
  PROJECT  [default: https:/] The project to get logs from

OPTIONS
  -a, --all                Show logs from all pods
  -i, --interval=interval  [default: 3000] Show logs polling interval
  -s, --since=since        [default: 15] Show logs since this many minutes ago

EXAMPLES
  sf logs
  sf logs platform/investor-service --since 180

sf secrets:check CONFIGPATH

Checks if the config are encrypted correctly

USAGE
  $ sf secrets:check CONFIGPATH

ARGUMENTS
  CONFIGPATH  [default: /config/] path to the directory containing the configs

ALIASES
  $ sf check

sf secrets:decrypt [CONTEXT]

decrypt a config file

USAGE
  $ sf secrets:decrypt [CONTEXT]

EXAMPLE
  sf secrets:decrypt development

sf secrets:encrypt [CONTEXT]

encrypt a config file

USAGE
  $ sf secrets:encrypt [CONTEXT]

EXAMPLE
  sf secrets:encrypt development

sf secrets:set CONTEXT

Set a value in an encrypted file

USAGE
  $ sf secrets:set CONTEXT

OPTIONS
  -k, --key=key      (required) Key to set (in Python dict lookup format)
  -v, --value=value  (required) Value to set (JSON)

DESCRIPTION
  Set a value in an encrypted file
  NOTE: the value must be JSON encoded (ie. JSON.stringify(value))

EXAMPLES
  setting a simple value:
  sf secrets:set production --key '["emailService"]["url"]' --value https://email.staging.brickblock.sh

  setting a complex value:
  sf secrets:set staging --key '["apis"][2]' --value '{"isGraphQL": false, "target": 
  "http://platform-kyc-provider-api.platform:3000/", "path": "/api/id-now-install-sms", "changeOrigin": true, "rewrite":
   {"regex": "/api/", "replacement": "/"} }'