@dataquiver/mongo

Change Log

This is the Mongo infrastructure for our servers. There are currently 4 main classes which this package provides.

• Mongo
  • Once you have the instance you should call await instance.initSafe()
    • This will set the public variables of db and client.
    • You need to pass the db variable to MongoController
• MongoRouter
  • This class may be used or extended depending on your use case.
  • It sets up CRUD endpoints for your RESTful API. You should have 1 MongoRouter class per collection.
  • You do not always need to create a MongoRouter, unless you will hit the collection's API via HTTP
• MongoSocketRouter
  • This class may be used or extended depending on your use case.
  • The UI should almost always use this websocket endpoint to get data.
  • This sets up CRUD websocket endpoints, and can send data to all/any client when a db operation happens.
• MongoController
  • This class may be used or extended depending on your use case.
  • This class is called from MongoRouter and performs the db operations.

Table of Contents

1. Table of Contents
2. Required Technologies
3. Installation
4. Usage
5. API

Required Technologies

• Node / NPM: Node version 16 or greater is recommended.
• Express: You should use express for your webserver routing.
• Database connection: The database connection information is passed via the Node Env. The Database should be hosted on Mongo Atlas. The information from the Node ENV is used to build the database connection string.
• Middleware: Our generic Router class takes in a middleware. This middleware, at a minimum, should add user information to each CRUD request. This allows you to easily add permission limiting to the REST endpoints. It also allows you to keep track of who is creating or updating the documents.

Installation

To install, run:

npm install @dataquiver/mongo --save

Usage

The intent of this package is to make an easy to use, Mongo backed, CRUD api.

1. First ensure you run node with the ENV parameters you need. We highly recommend using the dotenv package.
   1. You need a username, password, db, and host.
   2. You may use either DEV, STAGING, or PROD for each of these (or just set one if that is all you need)
   3. Look at mongo.ts for all possible parameters you may set.
2. In your entrypoint for your webserver you should connect to the database.

   // this will configure mongo to connect to hosted atlas services
   const mongo = new Mongo(MONGO_DB, MONGO_HOST, MONGO_USER, MONGO_PASSWORD);
     
   // you can also pass a full connection string:
   // const mongo = new Mongo(MONGO_DB, MONGO_CONNECTION_STRING);
     
   // initialize the connection. The `Safe` method will throw an error if the connection can't be made
   await mongo.initSafe();
   const {db} = mongo;

3. You now have a reference to the mongo database. This reference db should be passed to all database controller classes. Each database controller class should only control one mongo collection

   const { MongoController } = require('@dataquiver/mongo');
   
   const myCollectionController = new MongoController({
       db,
       collectionName: COLLECTION_NAME,
       options: { trackUsers: true, trackDates: true }, //Will save user and time during creates and updates
     })

   Note: You may also extend MongoController and override to add what you need. The template class will provide a default API which is explained later in this doc.
4. You may now create a mongo router. You must pass the controller to the router.

   const { MongoRouter } = require('@dataquiver/mongo');
   
   this.router = new MongoRouter({
       app,
       collectionName: COLLECTION_NAME,
       controller: myCollectionController,
       middleware,
       accessControlFunction: function,
     });

5. You now will have a working CRUD endpoint for data storage.

API

Controller

Constructor

new MongoController({ db, collectionName, options = {} })

• db is the mongo db instance from the mongo class.
• collectionName is your collectionName, duh.
• options
  • trackDates
  • trackUsers

Functions

• create(obj, meta = {user ...})
  • The router may use the meta param to pass extra info (often the user performing the action)
  • if trackDates, the entry will be added with a createdAt: Date.now() field
  • if trackUsers, the entry will be added with a createdBy: meta.user._id field
  • if !obj._id, we will create a UUID for you
• update(id, update, options = {}, meta = {user ...})
  • The router may use the meta param to pass extra info (often the user performing the action)
  • update should almost always be of the form {$set: {...}}
  • options are Mongo options that will be passed through to findOneAndUpdate
  • if trackDates, the entry will be updated with a updatedAt: Date.now() field
  • if trackUsers, the entry will be added with a updatedBy: meta.user._id field
• get(id, meta = {})
  • The router may use the meta param to pass extra info (often the user performing the action)
  • Will query the db for the object with {_id: id}
• list(query = {}, options = {}, meta = {})
  • The router may use the meta param to pass extra info (often the user performing the action)
  • options are the Mongo options to pass through to find
• delete(id, meta = {})
  • The router may use the meta param to pass extra info (often the user performing the action)
  • Will delete the object found matching the query of {_id: id}

Router

Constructor

new MongoRouter({ app, collectionName, controller, middleware })

• app is the express routing instance.
• collectionName is your collectionName, duh.
• controller is the MongoController with the same collectionName
• middleware is your middleware. It must add a user object to the req. We generally use @dataquiver/cognito-middleware for this

Default Endpoints

Feel free to add any additional endpoints you need. For example, users may want a /invite endpoint etc.

• /api/collectionName/create
  • POST
  • insert req.body into the database
• /api/collectionName/update
  • POST
  • Will update a document following the API below.

    const { _id, update, options } = req.body;

• /api/collectionName/get
  • GET
  • get the document with _id of req.query._id from the database
• /api/collectionName/list
  • GET
  • list the documents following the api below

    const { query, options } = req.query;

• /api/collectionName/delete
  • POST
  • delete the document with _id of req.body._id from the database

Overrideable functions

• create(req, res)
• update(req, res)
• get(req, res)
• list(req, res)
• delete(req, res)

SocketRouter

Constructor

new MongoSocketRouter({socketManager, collectionName, controller })

• socketManager is a @dataquiver/websocket-manager instance.
• collectionName is your collectionName, duh.
• controller is the MongoController with the same collectionName

• We then add our event listeners for client requests.
• We then add our event listeners from the controller.

Default Endpoints

Feel free to add any additional endpoints you need. For example users may want a /invite endpoint etc.

• collectionName create
  • insert data into the database
• collectionName update
  • Will update a document following the API below.

    const { _id, update, options } = data;

• collectionName get
  • get the document with _id of data._id from the database
• collectionName list
  • list the documents following the api below

    const { query, options } = data;

• collectionName delete
  • delete the document with _id of data._id from the database

Overrideable functions

• create(data, user)
• update(data, user)
• get(data, user)
• list(data, user)
• delete(data, user)

Asynchronous Events

The coolest thing about websockets is that it can send data to the user whenever it wants. By default we utilize this for CREATE UPDATE and DELETE, but you can add more events if you need!

How it works (we will use CREATE as an example):

1. In the SocketRouter constructor we added a listener this.controller.on('created', this.created.bind(this));
2. The controller does a CREATE, at the end it emits this.emit('created', newItem)
3. SocketRouter created callback function (this.created) is hit. It calls this._emitEvent
4. This function iterates over all the connected clients, calls the filterFunction, and if that passes sends the data to the client!

Add a new event

1. call this.emit('newEventName', data) in your controller
2. add a listener and call back in your SocketRouter ex: this.controller.on('newEventName', this.onNewEvent.bind(this));
3. make sure your filter function properly handles this new event
4. in your callback (onNewEvent) call this._emitEvent('actionForFilterFunction', data)
5. DONE

Middleware

The router expects middleware which will be added to all routes (/api/collectionName) via the express app instance. It is expected that this middleware add a user to the req. If you do not want/need this, mock a user object in your middleware. Sorry :\ . If you are using cognito for users, you can use @dataquiver/middleware

Contributing

1. Clone this repo
2. Run npm install to download dependencies
3. Run npm run build to compile typescript into javascript for distribution
4. Run npm run lint to run lint checks. This is also in a pre-commit hook.

Publishing

1. Run npm publish to publish to npm. You'll need to be authorized. This ought to run the build automatically.

Testing in an app

1. In this repo, run npm link to register it for overriding
2. In the app, run npm link @dataquiver/mongo to override the package from NPM with a symlink to this local copy
3. Run your app

If you make changes, you'll need to run npm run build in this repo and likely restart your app to pick them up.

When you're done, in the app, run npm unlink @dataquiver/mongo --no-save to remove the symlink and go back to using the NPM package. Use npm install then to download from the internet.

Logging

This package can emit logs. Import and call setLoggerFactory with your winston-compatible logger. If you don't provide one, logs will be dropped.