COW: Library

This library is intended to be a toolkit for developing different applications with more ease.

There are some parts of this library which are still a work in progress and may be subject to change, these have (WIP) next to them.

Basic concepts

• Lists

  While developing COW, there was a repeating pattern of creating lists of entities that would abstract away some MongoDB details, which resulted with repetition of code. Therefore this concept is moved to the library for better utilization. This class also handles authorization for all the entities it has. A list is an API for an entity existing in MongoDB and it's API is as follows:

  getCollectionName(): string 
  async getEntities(selection?: object, projection?: object): Promise<Interface[]> 
  async getById(identifier: string): Promise<Interface>
  async getRawEntity(identifier: string): Promise<Document> 
  async addEntity(entity: Interface): Promise<Interface & {_id}> 
  async addEntities(entity: Interface[]) 
  async deleteEntity(identifier: string) 
  async updateEntity(identifier: string, update: any) 

  In this API Interface is the interface provided for the entity where Document is the Mongoose document given. identifier is the given indexedBy attribute of the document, if it is not provided _id is used instead.

• Injections Injections are manual configs that other apps inject to your app to alter your app's behaviour which usually involves pointing to themselves. These are defined in cow.yml in the service that they are created from. You are free to define what your injections are made from and other apps can create custom ways to add functionality to your app. The more the better for customizability. They include a field cow_fromService to indicate what service has injected the config to your app. Your app can have different types of injections and you can request a type of injection.\

• Message passing Some components might want to be aware of some changes in the cluster, something like entities being created, updated, deleted or a critical action being performed. In order to make an app to listen an event performed by another app, we have a Pub/Sub mechanism for all apps to listen to different messages and publish them. To ensure reliability all components that receive a message are expected to acknowledge (ack) or not acknowledge (nack) the message. Here is the API for Pub/Sub:

  send(eventKey: string, body: any): Promise<void> // Send a message with the event key
  listen(listenerName: string, eventKey: string, onMessage: (message: IMessage) => Promise<void>): Promise<void> // Add a listener for a specific event, listenerName shall be unique for the specific listener, otherwise messages will be splitted among listeners.
  stopListener(listenerName: string): Promise<void>; // Stops an already created listener

  The object that you retrieve when registering the listener is a special type of IMessage which consists of:

  content: any; // The content of the message, this is set by the sender.
  ack: () => void; // You should call this function to acknowledge the retrieved message
  nack: () => void; // You should call this function to nack the retrieved message
  from: string; // The name of the service that sent this message.

Application

This utility allows you to create a full-fledged COW application, that injects some services for you, and allows you to create some more services before running your app.

Your app has 2 stages:

• Initialization: This is the stage where all the database connections, etc. are set up. Most of the core stuff are already prepared for your application to use, however you can chain some functions to app to initialize some more services for yourself.
• Running: At this stage your application will be bootstrapped and ready to run. You will receive a huge services list, from which you can get any service you want to use in your main code.

So basic structure of an app looks like:

import {Application} from "@coweb/cow";
(new Application())
    // Initialization part
    .addDbCollection("Test", someMongooseSchema)
    .run(async (services) => {
        // Running part
        services.logger.info(`Test is alive with component UUID: ${services.envs.COW_APP_UUID}`);
    })

Initialization

In initialization stage there are some functions that allow you to create some customizations, these are:

addService<T>(serviceName: string, service: T) 

Adds a service that can be accessed via services interface

addDbCollection<T>(name: string, schema: Schema, indexedBy?: string, populations?: IDbPopulation[])

Adds a new DB collection that doesn't already exist. name is collection name, schema is the Mongoose schema of the entity. You may pass an optional indexedBy parameter which will determine the identifier to be used as the id in all operations (If not provided built-in _id will be used). Also you can provide populations for entities that needs to be populated when queries are executed.

setRightsForRoleOnEntity(name: string, role: Role, grantedAccesses: Access[])

After you create an entity you can call this function to create roleBindings per role for your created entity.

initFileService(basePathEnv: string)

You have to call this function before initializing your app, when you need to use file system. You have to pass the data path from an env value and you should give the env key for the path here.

In the future we might have the capability to register multiple file services.

Running

You shall call run with an async function that accepts services of type IInjection. In this function you shall run whatever your app is supposed to do. The API for the services object is given below

services: {
    [key: string]: any; // This is for the services that you inject

    logger: winston.Logger;
    getListOf: <Doc extends Document, Interface>(name: string) => (username: string) => Promise<CollectionList<Doc, Interface>>;
    getInjectionsOf: (type: string) => Promise<any[]>;
    envs: (envKey: string) => string;
    files: {
        find: (id: string) => Promise<IFile>
        remove: (id: string) => Promise<void>
        save: (file: IFile) => Promise<IFile>,
    };
    sessions: {
        getUser: (sessid: string) => Promise<string>
        create: (username: string) => Promise<string>
        delete: (sessid: string) => Promise<void>,
    };
    pubSub: PubSub;
    createEndpoint: () => Endpoint;
    url: {
        build: (path: string) => string,
        buildForService: (path: string, service: string) => string,
    };
    users: UserCollection;
}

• logger

  This is the logger service, you are expected to use it for logging instead of console.log, console.debug, etc. Do not blindly console log, this logger adjusts itself according to the log_level given.

• getListOf

  This service encapsulates some manual work that needs to be done in order to create a CollectionList and manages authorization. You have to first pass it the name of the collection. This name can be a Collection's name that you have created in initialization phase or a core component (all core components are registered automatically by Application class). Then, considering that people will be accessing the resource that you want to use, you have to pass the username of that user to retrieve a fully working CollectionList instance.

• getInjectionsFor

  This service allows to read injections of a specific type.

• envs

  Retrieve an env value of the process, this throws an exception if an environmental value doesn't exist, instead of returning undefined. We are using environmental variables to pass run-time configuration according to 3rd of 12 Factor App Principles. Thus some configurations of applications are passed by COW infrastructure to your project, of which you can freely use. These envs include:

  COW_LOG_LEVEL: string; // The log level of the application, this is usually INFO for production
  COW_MONGODB_URL: string; // MONGODB_URL for the database to be accessed.
  COW_APP_UUID: string; // UUID of the app, which is basically `service`-`component`
  COW_APP_SERVICE: string; // Service name
  COW_APP_COMPONENT: string; // Component name
  COW_APP_BASE_URL: string; // Base URL of the Ingress host, all cow applications retrieve URLs based from BASE_URL/COMPONENT_UUID, all middlewares and endpoints handle this automatically
  COW_APP_DEBUG: string; // Is app being debugged, in production this is false
  COW_INJECTION_PATH: string; // The place of the injections in the file system to be read, used by injection service.
  COW_RABBITMQ_URL: string; // The URL for app to access RabbitMQ for passing messages.

• files

  Find, save and delete files to permanent storage, this one requires you to call initFileService function above to function

• sessions

  Anything you might need to interface with user sessions

• pubSub

  Exposes a pubSub service that you can use to createListeners, stopListening or publish messages.

• createEndpoint

  Returns an Express endpoint that you can inject routers, add middlewares, await serving from that retrieves port from an env value. All middlewares used by this library are included in the endpoint. Furthermore, all errors thrown from routes will be catched by error handling middleware automatically.

• users

  Exposes UserCollection interface which includes varying functions to be used for modifying users. (Hopefully the use will be minimal)

Express Middlewares

The express middlewares provided are designed to help you abstract away some more concepts using express. The AppMiddlewares constructor takes the services and provides helpful middlewares to abstract some concepts even further.

• getListOf

  Similar to getListOf method provided in the services part, but instead of providing the username, the middleware handles it based on current user (req.user has to be string for this to work) and adds the CollectionList to your request.

  You have to extend the basic express request to match the API result if you are using this middleware like:

  interface ITestRequest extends express.Request {
      TestCollection: CollectionList<ITest, ITestDocument>;
  }
  ...
      const middlewares = AppMiddlewares(services);
      app.use("/test", middlewares.getListOf("Test"));
      app.get("/test/", async (req: ITestRequest, res) => {
          res.send(await req.TestCollection.getById("test"));
      });

• fileHandler

  This middleware looks for files that are in the request and saves them using the files API in services, adds savedFiles as an array to request. You can extend the IFileRequest interface to have an interface on the request object.

  You need express-busboy middleware added with

  import * as bb from "express-busboy";
      bb.extend(app, { upload: true as any });

  for this middleware to function correctly.

• userHandler

  This middleware looks in the cookie for the sessionId and sets the req.user to user's username for the request. This middleware requires cookieParser middleware to function.

• handleCowErrors

  This middleware is used to handle varying errors raised by different components of this library. Since this is an error handling middleware this needs to be specified after all routes being set (see the docs)

Extending the library

This library is intended to be a common place for recurring practices in code and shared interfaces and tools. If you want to make a PR for adding new entities and/or tools, you should have an argument about what the abstraction is and how many different places this will be used.

Testing

You can run rabbitmq locally via Docker (the simplest way) with:

sudo docker run -d --hostname test --name a-rabbit -p 15672:15672 -p 5672:5672 rabbitmq:3-management