jarvis-node

A multipurpose helper package for our node apps. This package contains helpers for the following:

• Authentication
• Feature flag control
• Logging & Monitoring
• Environment variable management
• Request Query processing

Installation

yarn install @go-mailer/jarvis

Authentication

In Go-Mailer, there are two major types of authentication:

1. JWT authentication: is employed in simple username/password log-ins and is the primary method of user authentication in the system. We use the jsonwetoken package for JWT authentication.

    const { Authenticator } = require('@go-mailer/jarvis');
    const { JWTAuth } = Authenticator;
         
    app.use('/route', JWTAuth, route_handler);

2. API authentication: is done via in-house authentication methods. The IAM app is responsible for managing user identities, authorizations and API keys. This package makes simple API calls to the IAM app to verify the provided API keys. There are two ways API keys can be authenticated:

   • As a Query Parameter: Here the API key is passed as a request query parameter - typically for applications which intend to authenticate via GET HTTP method.

   • As an Authentication Header: This is default way to pass API keys and security tokens in HTTP requests.

     const { Authenticator } = require('@go-mailer/jarvis');
     const { APIAuth } = Authenticator;
     const { defaultAuth, paramAuth } = APIAuth();
             
     app.use('/api-route', defaultAuth, handler); //API auth
     app.use('/api-param-route', paramAuth, handler); //Query param auth

Feature Flag Control

Go-Mailer uses Go-Flags to control application feature visibility. This gives us the ability to build our application's feature continuosly without worry about the customers seeing or using them before release.

async FeatureFlag.verify(<flage_name>, <criteria>): used to verify whether or not a feature is turned ON or OFF. Returns true if feature is enabled and false otherwise.

• flag_name string: required

  This is the name of the feature to be checked as defined on the Go-Flags app.

• criteria Object: required This is a key -> value pair of criteria with which the status of the flag should be checked. These criteria are matched against those configure on the Go-Flags app.

  const { FeatureFlag } = require('@go-mailer/jarvis');
  FeatureFlag.verify(<flag_name>: String, <criteria>: Object ).then((result) => {
    // do anything with result.
  });

Logging

Logging is essential to the success of any application. The ability to collect information about the application's processes is crucial to making engineering and business decisions as well as resolving any issues that may arise in the system. Jarvis provides logging capabilities. Currently, the logging application we utilize is Logtail.

There are two kinds of Loggers provided by Jarvis:

1. Request Logger
2. Process Logger

Request Logger

This middleware generates logs each time a request passes through the system. It should be placed before any routes are defined.

const { RequestLogger } = require('@go-mailer/jarvis'); 
app.use('/', RequestLogger);

Process Logger

This logger is used to keep various kinds of logs from different parts of the system. There are two kinds of process logs recorded: error & info.

1. error logs are recorded whenever an exception is thrown or some anormally is detected or encountered.
2. info logs are strictly informational. They are typically used to provide visibility through the system for things like system health, process paths, etc.

  const { ProcessLogger } = require('@go-mailer/jarvis');
  const logger = new ProcessLogger(<service_name>: String);
  
  // error logging
  logger.error(<error_object>: Error, <method_name>: String);
  
  // info logging
  logger.error(<message_to_be_logged>: String, <method_name>: String);

1. <service_name>: This is the name of the service or part of the system where the log is generated. For example, if the log was generated in the Mailing controller, this would have the value MailingController.
2. <error_object>: This is the instance of the Error that was thrown and caught.
3. <message_to_be_logged>: For informational logs, this is the information to be logged.
4. <method_name>: This is the method/function within which the log was generated.

.

Environment variable management

Jarvis also provides environment variable management. This enables applications to set and retrieve environment variables more seamlessly.

1. set(config: Object): allows for the configuration of new variables. This does not override environment variables.

   • config: Object: a key-value set that specifies application or instance specific criteria to the environment namespace.

   .

2. fetch(variable_name: String, is_required: Boolean): retrieves the specified variable.

   • variable_name: String: The name of the variable to retrieve.
   • is_required: Boolean: Specifies whether or not the variable to be retrieved is required. If set to true, an Exception would be thrown if no value exists. default: false

const { EnvVar } = require('@go-mailer/jarvis');

// set environment variable(s)
EnvVar.set({
  APP_NAME: 'Mailing'
});

// retrieve environment variable
EnvVar.fetch('APP_NAME', true); // required
EnvVar.fetch('APP_NAME'); // not required

Request Query processing

Jarvis provides a query processor that converts request queries into MongoDB compatible queries. Request Queries are processed based on keywords and the use of special characters. A request query is the part of the request path that comes after the question mark (?). The QueryBuilder.buildQuery() is used for request processing.

GET http://go-mailer.com?<query_part>

QueryBuilder.buildQuery(options: Object) : {
  count: boolean
  fields_to_return: string
  limit: number
  seek_conditions: MongooseQueryObject
  skip: number
  sort_condition: string  
}

Keywords:

Keywords in request queries are used to prepare the following queries. Given the sample records in the database:

[{
  name: "Nathan",
  age: 21,
  sex: 'M'
}, {
  name: "Anita",
  age: 15,
  sex: 'F'
}, {
  name: "Manchang",
  age: 25,
  sex: 'M'
}]

1. Field inclusion queries: when the keyword return_only is used the query, ONLY the properties that are contained in the value that is assigned to this keyword WILL be returned. For example:

   // given the request
   GET https://go-mailer.com?return_only=name,age
      
   // sample response will return only the `age` and `name` properties on the records
   [{
     name: "Nathan",
     age: 21,
   }, {
     name: "Anita",
     age: 15,
   }, {
     name: "Manchang",
     age: 25,
   }]

2. Field exclusion queries: when the keyword exclude_only is used the query, ONLY the properties that are NOT contained in the value that is assigned to this keyword WILL be returned. For example:

   // given the request
   GET https://go-mailer.com?exclude_only=name,age
      
   // sample response will return only the `sex` property on the records.
   [{
     sex: 'M'
   }, {
     sex: 'F'
   }, {
     sex: 'M'
   }]

3. Pagination queries: when the keywords page & population are specified in the query, the result set would be paginated. NOTE that the first page number is 0. For example:

   // given the request
   GET https://go-mailer.com?page=0&population=2
    
   // sample response. First two matching objects 
   [{
     name: "Nathan",
     age: 21,
     sex: 'M'
   }, {
     name: "Anita",
     age: 15,
     sex: 'F'
   }]

4. Sort queries: we can specify sort conditions by using the sort_by keyword in the query. The value would contain the property with which we would like to sort. Default sort mode is ASC. If you want DESC, specify the value with a minus sign (-):

   // ASCending (age)
   GET https://go-mailer.com?sort_by=age
      
   // DESCending (-age)
   GET https://go-mailer.com?sort_by=-age

5. count: To return a count of items that match the query, the count keyword is used.

   GET https://go-mailer.com?count=1

6. group: To group items that match the query by a certain field, the group_by keyword is used.

   GET https://go-mailer.com?group_by=age

7. boolean: To return items that match the query for boolean fields, the bool keyword is used. pre-pending the minus (-) signto the name of the field would evaluate that field to false and vice-versa.

   GET https://go-mailer.com?bool=age,-name
      
   // evaluates to:
   { age: true, name: false }

8. exists: To return items based on the existence of certain fields, the exists keyword is used. Prepend the minus (-) sign to achieve the opposite.

   GET https://go-mailer.com?exists=age,-state
      
   // evaluates to
   { age: { $exists: true }, state: { $exists: false } }

Special Characters

The use of special characters in query values indicate the need to prepare the following queries:

1. OR queries: Whenever a comma separated list is used as a keyword value. This is hand

   // return records where 'name' is 'Nathan' OR 'Michael' 
   GET https://go-mailer.com?name=Nathan,Michael

2. NOR queries: Whenever the values of a query field are separated by an exclamation mark (!), a NOR query is built.

   // return records where 'name' is neither 'Nathan' NOR 'Michael' 
   GET https://go-mailer.com?name=Nathan!Michael

3. Range selection queries: To support range queries, the tilde character (~) is used in between values. currently, ranges only work with integer values

   // get all records where age is between 12 and 34 (inclusive)
   GET https://go-mailer.com?age=12~34
      
   // get all records less than 34
   GET https://go-mailer.com?age=~34
      
   // get all records greater than 34
   GET https://go-mailer.com?age=34~

Wildcard

The QueryBuilder closure also provides a buildWildcardOptions() function that prepares a mongoose regular expression query object for text matching.

QueryBuilder.buildWildcardOptions(key_list: string, value) : MongooseQueryObject

• key_list: string: is a comma-separated list of properties against which value is sought
• value: any: is the value being sought for.