Data Model Querying

Datastore abstraction for advanced query support.

Install: @travetto/model-query

npm install @travetto/model-query

# or

yarn add @travetto/model-query

This module provides an enhanced query contract for Data Modeling Support implementations. This contract has been externalized due to it being more complex than many implementations can natively support.

Contracts

Query

This contract provides the ability to apply the query support to return one or many items, as well as providing counts against a specific query.

Code: Query

export interface ModelQuerySupport {
  /**
   * Executes a query against the model space
   * @param cls The model class
   * @param query The query to execute
   */
  query<T extends ModelType>(cls: Class<T>, query: PageableModelQuery<T>): Promise<T[]>;
  /**
   * Find one by query, fail if not found
   * @param cls The model class
   * @param query The query to search for
   * @param failOnMany Should the query fail on more than one result found, defaults to true
   */
  queryOne<T extends ModelType>(cls: Class<T>, query: ModelQuery<T>, failOnMany?: boolean): Promise<T>;
  /**
   * Find the count of matching documents by query.
   * @param cls The model class
   * @param query The query to count for
   */
  queryCount<T extends ModelType>(cls: Class<T>, query: ModelQuery<T>): Promise<number>;
}

Crud

Reinforcing the complexity provided in these contracts, the Query Crud contract allows for bulk update/deletion by query. This requires the underlying implementation to support these operations.

Code: Query Crud

export interface ModelQueryCrudSupport extends ModelCrudSupport, ModelQuerySupport {
  /**
   * A standard update operation, but ensures the data matches the query before the update is finalized
   * @param cls The model class
   * @param data The data
   * @param query The additional query to validate
  */
  updateByQuery<T extends ModelType>(cls: Class<T>, data: T, query: ModelQuery<T>): Promise<T>;
  /**
   * Update all with partial data, by query
   * @param cls The model class
   * @param query The query to search for
   * @param data The partial data
   */
  updatePartialByQuery<T extends ModelType>(cls: Class<T>, query: ModelQuery<T>, data: Partial<T>): Promise<number>;
  /**
   * Delete all by query
   * @param cls The model class
   * @param query Query to search for deletable elements
   */
  deleteByQuery<T extends ModelType>(cls: Class<T>, query: ModelQuery<T>): Promise<number>;
}

Facet

With the complex nature of the query support, the ability to find counts by groups is a common and desirable pattern. This contract allows for faceting on a given field, with query filtering.

Code: Query Facet

export interface ModelQueryFacetSupport extends ModelQuerySupport {
  /**
   * Facet a given class on a specific field, limited by an optional query
   * @param cls The model class to facet on
   * @param field The field to facet on
   * @param query Additional query filtering
   */
  facet<T extends ModelType>(cls: Class<T>, field: ValidStringFields<T>, query?: ModelQuery<T>): Promise<{ key: string, count: number }[]>;
}

Suggest

Additionally, this same pattern avails it self in a set of suggestion methods that allow for powering auto completion and type-ahead functionalities.

Code: Query Suggest

export interface ModelQuerySuggestSupport extends ModelQuerySupport {
  /**
   * Suggest instances for a given cls and a given field (allows for duplicates with as long as they have different ids)
   *
   * @param cls The model class to suggest on
   * @param field The field to suggest on
   * @param prefix The search prefix for the given field
   * @param query A query to filter the search on, in addition to the prefix
   */
  suggest<T extends ModelType>(cls: Class<T>, field: ValidStringFields<T>, prefix?: string, query?: PageableModelQuery<T>): Promise<T[]>;
  /**
   * Suggest distinct values for a given cls and a given field
   *
   * @param cls The model class to suggest on
   * @param field The field to suggest on
   * @param prefix The search prefix for the given field
   * @param query A query to filter the search on, in addition to the prefix
   */
  suggestValues<T extends ModelType>(cls: Class<T>, field: ValidStringFields<T>, prefix?: string, query?: PageableModelQuery<T>): Promise<string[]>;
}

Implementations

|Service|Query|QueryCrud|QueryFacet| |-------|-----|---------|----------| |Elasticsearch Model Source|X|X|X| |MongoDB Model Support|X'|X'|X'| |SQL Model Service|X'|X'|X'|

Querying

One of the complexities of abstracting multiple storage mechanisms, is providing a consistent query language. The query language the module uses is a derivation of mongodb's query language, with some restrictions, additions, and caveats. Additionally, given the nature of typescript, all queries are statically typed, and will catch type errors at compile time.

General Fields

• field: { $eq: T } to determine if a field is equal to a value
• field: { $ne: T } to determine if a field is not equal to a value
• field: { $exists: boolean } to determine if a field exists or not, or for arrays, if its empty or not
• field: T to see if the field is equal to whatever value is passed in`

General Single Valued Fields

• field: { $in: T[] } to see if a record's value appears in the array provided to $in
• field: { $nin: T[] } to see if a record's value does not appear in the array provided to $in

Ordered Numeric Fields

• field: { $lt: number } checks if value is less than
• field: { $lte: number } checks if value is less than or equal to
• field: { $gt: number } checks if value is greater than
• field: { $gte: number } checks if value is greater than or equal to

Ordered Date Fields

• field: { $lt: Date | RelativeTime } checks if value is less than
• field: { $lte: Date | RelativeTime } checks if value is less than or equal to
• field: { $gt: Date | RelativeTime } checks if value is greater than
• field: { $gte: Date | RelativeTime } checks if value is greater than or equal to

Note: Relative times are strings consisting of a number and a unit. e.g. -1w or 30d. These times are always relative to Date.now, but should make building queries more natural.

Array Fields

• field: { $all: T[]] } checks to see if the records value contains everything within $all

String Fields

• field: { $regex: RegExp | string; } checks the field against the regular expression

Geo Point Fields

• field: { $geoWithin: Point[] } determines if the value is within the bounding region of the points
• field: { $near: Point, $maxDistance: number, $unit: 'km' | 'm' | 'mi' | 'ft' } searches at a point, and looks out radially

Groupings

• { $and: [] } provides a grouping in which all sub clauses are require,
• { $or: [] } provides a grouping in which at least one of the sub clauses is require,
• { $not: { } } negates a clause A sample query for User's might be:

Code: Using the query structure for specific queries

import { ModelQuerySupport } from '@travetto/model-query';
import { User } from './user';

export class UserSearch {
  service: ModelQuerySupport;

  find() {
    return this.service.query(User, {
      where: {
        $and: [
          {
            $not: {
              age: {
                $lt: 35
              }
            }
          },
          {
            contact: {
              $exists: true
            }
          }
        ]
      }
    });
  }
}

This would find all users who are over 35 and that have the contact field specified.

Custom Model Query Service

In addition to the provided contracts, the module also provides common utilities and shared test suites.The common utilities are useful for repetitive functionality, that is unable to be shared due to not relying upon inheritance(this was an intentional design decision).This allows for all the Data Model Querying implementations to completely own the functionality and also to be able to provide additional / unique functionality that goes beyond the interface. To enforce that these contracts are honored, the module provides shared test suites to allow for custom implementations to ensure they are adhering to the contract's expected behavior.

Code: MongoDB Service Test Configuration

import { Suite } from '@travetto/test';

import { ModelQuerySuite } from '@travetto/model-query/support/test/query';
import { ModelQueryCrudSuite } from '@travetto/model-query/support/test/crud';
import { ModelQueryFacetSuite } from '@travetto/model-query/support/test/facet';
import { ModelQueryPolymorphismSuite } from '@travetto/model-query/support/test/polymorphism';
import { ModelQuerySuggestSuite } from '@travetto/model-query/support/test/suggest';
import { ModelQueryCrudSupport } from '@travetto/model-query/src/service/crud';
import { ModelQuerySuggestSupport } from '@travetto/model-query/src/service/suggest';
import { ModelQueryFacetSupport } from '@travetto/model-query/src/service/facet';
import { Config } from '@travetto/config';
import { Injectable } from '@travetto/di';

import { QueryModelService } from './query-service';

@Config('model.custom')
class CustomModelConfig { }

@Injectable()
class CustomModelService extends QueryModelService implements ModelQueryCrudSupport, ModelQueryFacetSupport, ModelQuerySuggestSupport {
}

@Suite()
export class CustomQuerySuite extends ModelQuerySuite {
  serviceClass = CustomModelService;
  configClass = CustomModelConfig;
}

@Suite()
export class CustomQueryCrudSuite extends ModelQueryCrudSuite {
  serviceClass = CustomModelService;
  configClass = CustomModelConfig;
}

@Suite()
export class CustomQueryFacetSuite extends ModelQueryFacetSuite {
  serviceClass = CustomModelService;
  configClass = CustomModelConfig;
}

@Suite()
export class CustomQueryPolymorphismSuite extends ModelQueryPolymorphismSuite {
  serviceClass = CustomModelService;
  configClass = CustomModelConfig;
}

@Suite()
export class CustomQuerySuggestSuite extends ModelQuerySuggestSuite {
  serviceClass = CustomModelService;
  configClass = CustomModelConfig;
}