sov-fund-mover service

Overview

The fund mover service is responsible to the money transfers in the SOV project. It executes both Money-In and Money-Out operations over the wallet.

Prerequisites

1. Install yarn using npm install -g yarn. Use yarn to download, build and run local commands
2. forge, opctl and other common Remitly tools.
3. pritunl
4. Install sequelize using npm i -g sequelize-cli.
5. Install nodemon using npm i -g nodemon.

Environments

preprod

First make sure you have sequelize cli - if not run npm i -g sequelize-cli. https://sov-fund-mover-preprod.us-west-2.remitly.com/

forge-system

docs (swagger)

1. opctl run db-plain
2. yarn run init

prod

TBD

Getting Started

Running the service

We recommend that you run the service locally using:

opctl run dev

The op will run a database and migrations for you and will also initialize temporal workers to running workflows of both money in and money out. If you prefer to run the service without opctl you can always run yarn dev but don't forget that you will need to run the database (opctl run mysql), migrations (yarn migrate-up), and temporal (yarn temporal server start dev.

Before you are running it:

1. Make Sure you are connecting to the remitly's vpn and make sure you have docker desktop running on your machine.
2. Only for the first time ever running - run the following command on the terminal:

echo -e "[default]\nregion=us-west-2" > ~/.aws/config

3. Run the following commands.

forge auth github
export github='{"username": "ENTER YOUR GITHUB USERNAME", "accessToken": "'$(forge auth github)'"}'
forge auth aws --account "Remitly Product" --role "DeveloperAccess" --role "pb-dev-role"

(The first two commands required only on the first time you open the terminal)

Tips:

• If you run into an unauthorized error when you run opctl run dev, you may need to opctl self-update

Local endpoints:

1. api: http://localhost:3000
2. healthcheck: http://localhost:9080/health
3. docs (swagger): http://localhost:3000/api-docs

Debugging the service locally

1. Run

opctl run debug

2. Open VsCode
3. Attach the remote debugger, by opening "Run and Debug" in the side-panel and selecting "Attach to Remote" in the drop-down.
4. Press the Play button to attach the debugger
5. Observe the service initializing in the termina logs or VsCode Debug Console
6. Set breakpoints, run through your scenario, and observe your breakpoints getting hit.

Running tests

Run All tests

For running all the tests:

opctl run -a dockerSocket=/var/run/docker.sock test

The op will init a plain mysql DB for the tests called 'sov_fund_mover_test' with migration and then will run all the tests in the service.

Run a test manually

For running specific tests manually - you should init a DB once and then you can run every test. you can tdo this with the following commands:

1. Run:

opctl run -a DB_NAME="sov_fund_mover_test" mysql

2. Then on another terminal for migrations:

yarn run init-test

3. Run your test:

yarn coverage -- {PATH_TO_THE_TEST_FILE}

Debugging tests

We've enabled running tests within an IDE or yarn tests (without opctl) by spinning up a MySQL test container. This is setup by:

1. Leveraging the TestContainers SDK, which spins up a relevant container when needed, and reaps a container when we're done. MySqlTestContainer.ts contains the logic on how this works.
2. Adding the following test hook to UTs that depend on a database: enableIDEDebuggingForTestsDependentOnDB(). This call will setup beforeAll(...) scaffolding in your Jest unit tests to initialize the database before a test run.
3. A global teardown hook after tests finish running
   1. Global Teardown
4. Setting up the ARE_TEST_CONTAINERS_DISABLED environment variable to false in opctl runs so that it doesn't conflict with the pre-existing opctl database setup.

Known Issue - If you're getting reason: An error occurred listing credentials while running the tests with yarn test or an IDE

You might get an error that looks like the following after trying to run the test: Error: Jest: Got error running globalSetup - /.../globalTestSetup.ts, reason: An error occurred listing credentials

If this is occurring, this is because docker-credential-desktop might be missing in your shell $PATH or the symlink has been corrupted by a docker update. You can confirm this is the case by seeing if you can execute docker-credential-desktop in the terminal; if unsuccessful, then the following fix below can help.

You can fix this issue by:

1. Reinstalling Docker (recommended) - Doing a clean install of Docker by uninstalling and reinstalling Docker Desktop. This should fix the symbolic links to the Docker binaries and docker-credential-desktop should be callable again.

OR

2. Add location of your binaries to $PATH - Confirm the known location of the docker binaries (e.g. /Applications/Docker.app/Contents/Resources/bin), and add the path to your shell environment's $PATH variable (via .bashrc, .zshrc). For example, you can add the following line to your .bashrc file - export PATH=$PATH:/Applications/Docker.app/Contents/Resources/bin. Reload the terminal ( e.g. . ~/.bashrc, or close and re-open your terminal) for the environment variable update to take effect.

How to add a secret

1. Add the secret in this format: SECRET_NAME: $secret to the config part at the forge.yml to the right domain ( local/preprod/prod) or at default to add to all of them.
2. Deploy to the domain with forge with the command: forge deploy -d {domain} -v {version} - the deploy will failed (it is fine).
3. Enter to the service at forge: forge and go to the domain.
4. Go to secrets, find the secret and press on the aws link.
5. Make sure to switch role to the pb-dev-role. If you don't have it configure it using this link
6. Go to secret value - press retrive secret value and then edit.
7. Add the value to the plain text window.
8. deploy again.

Temporal

To watch the workflows executed on temporal UI - use temporal cloud (or via temporal cloud on Okta). If you don't have access, ask here(search for temporal and post the ticket).

When running locally (yarn temporal server start dev) - use this link

Documentation

Documentation is in openapi v3 (formerly Swagger) format. The documentation is generated at runtime. It is saved as a file src/swagger-output.json and can also be accessed though the API at http://localhost:3000/api-docs. Using the library swagger-jsdoc the documentation can be split across the API folder (src/api/**) as comments:

CI/CD

The CI/CD pipline executes on Jenkins and is configured on the Jenkinsfile.

• Every push or PR will run opctl run build, forge pkg and opctl run -a dockerSocket=/var/run/docker.sock test.
• Merging into dev branch releases new version and deploy to pre-prod.
• Merging into main branch releases new version and deploy to both pre-prod and prod.

You can watch the jobs on Jenkins here

• to run a specific test, use jest -t "....", notice you need mysql running for db related tests to work

Linter

add to setting.json in vscode so it will be nice to work on

"editor.codeActionsOnSave": {
    "source.fixAll.eslint": true
},

before pushing run:

yarn run lint

or, if you want autofixes run

yarn lint-fix

please don't push to master with lint errors