microvolt-lib

NPM library for MicroVolt webservice architecture

Building web applications can be hard. No longer can an application survive in creating a monolithe to perform everything. They are not suited for quick development, expanding features easily and for multiple teams to easily collaborate and test on a single code base. Additionally, every web application require very similar basic functionality, including, but not limited to: database, database schema evolution, web requests, intra-service communication, cache, session management (register, login, tokens, SSO, etc.). To higher level services that are found common in web applications, including: Email, SMS, reporting, activity, in/outjest from/to external services, e-commerce and more.

The concept is to have a common set of capabilities that work out of the box and that can be easily extended to meeting the business needs of the web application.

This is a project to create a microservice framework based on Node.js. It is currently being tested on Google Cloud but other hosting environments are planned.

The code base is entirely TypeScript. No Kubernetes required.

Status:

Complete:

1. Base microservice class
2. Sample microservice
3. Database and query abstraction for Postgres
4. Dynamic database table setup and seeding
5. Session microservice implementation (register, login, auth)

In Work:

1. Hub microservice implementation
2. Web service implementation
3. Actions/events registration and notification
4. Email microservice implementation

Planned:

1. Data cache layer
2. AI microservice implementation
3. Activity microservice implementation
4. Database abstraction for MySQL
5. Database abstraction for NoSQL (e.g. MongoDB)
6. ZIMM implementation

Goals:

1. Object Oriented Typescript with good design patterns
2. Ease of adding new services
3. Fast horizontal scaling
4. Easy deployment and versioning

It shall include:

1. Easy to create new sertvices
2. Includes "hub" service to manage:
   1. Service registration and lifecycle
   2. Service configuration
   3. Events/actions between services
   4. Service discovery
3. Includes "webserver" to host static files and act as a gatway
4. The hosting environment(s) selected will manage horizontal scaling of services
5. Other canned services shall included
   1. Session management (login, registration, authentication, throttle limits)
   2. ETL to reporting database
   3. Activity Service
   4. Email Service
   5. Input/Output (web hook) Service
   6. File/CDN Service
   7. Report Service
   8. Import Service
   9. Zapier Service
   10. Commerce Service (cart, multi-currency, payment, zipcode/address verification; factory:stripe)
   11. SMS service (factory: twilio)
   12. Product Service
   13. Order Service
   14. Customer Service

Notes:

Google Deployment Notes:

1. For each project this is being deployed to, you will need to:
   1. Add IAM Role to include Secrets Manager + Secrets Accessor or you will not be able to access saved secrets in the app. You will get a permissions denied even though the project has Edit access to do everything.

CONFIGURATION:

=================== For configurations that contains sensitive information (e.g. passwords) should never be stored in code (hard coded) or in coniguration files that would get checked-in to GIT. This is a major security violation.

Configuration variables being used for secrets or environment variables are case sensitive and should only include letters and underscoress (_). Google Cloud does not allow periods (.) and environment variables do not allow for hypans (-). This will be enforced to avoid needless errors in the different environments.

In Google Cloud, those kinds of configuration variables shold be stored in the Secrets Manager. In the service configuration file, you may set:

"secrets" : { "root" : "projects/<PROJECTID>/secrets" }

Where <PROJECTID> is the project id for the project of the app engine. Each environment (e.g. development, staging, product) have unique projects so each environment will have a unique configuration file (e.g. local.json, development.json, production.json).

In the configuration file, all will need to do is set that variable to indicate that it is a secret. For example:

"database" : { "host" : "secret:hub_db_host/versions/latest", "port" : "secret:hub_db_port/versions/latest", "name" : "hub", "username" : "secret:hub_db_username/versions/latest", "password" : "secret:hub_db_password/versions/latest" }

Anything prefiexed by "secret:" will assume that it is to be accessed in the Secrets Manager. Note for Google, you can have different versions of the variable to not to break currently deployed versions of your microservce.

For Local developent, you can store sensitive information as an environment variable. So a local.json file may look like:

"database" : { "host" : "env:hub_db_host", "port" : "env:hub_db_port", "name" : "hub", "username" : "env:hub_db_username", "password" : "env:hub_db_password" }

The prefix of "env:" will look for the locally set environment variable.

For Windows, you can set the environment varialbe(s) from the Windows GUI or from the terminal prompt by:

$env:hub_db_password = 'passwsord1'

To verify that it is set:

dir env:hub_db_password

Unless set locally, you may need to restart your Windows computer to see the environment variables.

This is a work in progress...