GLUE Search Service

Overview

GLUE Search Service (GSS) allows desktop applications to search Entities like Clients, Instruments, Accounts, etc.

Each Entity is associated with a set of fields, where each field has:

• name, e.g. firstName, lastName, id, etc.
• type:
  • scalar types: Boolean, Int, Long, Double, String, DateTime; or
  • Composite, which means that the field defines an object (defined again as a sequence of fields)
• search type - None, Partial, Exact or Both (supports Exact and Partial matching)
• isArray - flag indicating that the field is an array
• display name
• description
• metadata (arbitrary key/value pairs)

Composite combined with the isArray flag allows for the definition of regular (non-cyclic, directed) object graphs. Both the Composite type and the isArray flag implicitly turn off searching on that field.

Multiple Search Providers

There could be multiple Search Providers for each entity type, e.g. Clients can be provided by both Salesforce or Microsoft Dynamics, and Outlook.

Providers are not required to offer the same set of entity fields. That is, the union of all fields from all providers defines the entity type. However, all Providers must use the same field descriptors for a given entity type they support. The only exception is the search type, where providers can specify different search type for any of the fields.

Full-Text Search

A Provider can add a special field, called ANY, to an entity type definition, which allows clients to execute a partial search against all indexed/searchable fields for that entity, or a full text search, if the provider supports that.

Search Mechanics

A Client issues a Search Query for an entity type, specifying one or more search fields and their values (selection), and can optionally limit the maximum number of entries that each provider should return, and the set of entity fields (projection) that are required in the result.

The query filter is a set of key/value pairs, where the key is the name of one of the entity type's fields, and the value is the value to search for.

Depending on the search type, the relevant providers will perform exact or partial matching (it is an error to pass a field in the search filter, if field's search type is None). Clients applications can optionally pass both a field value, and the requested search type. In case the client wants the search to be performed on all searchable fields, or the client knows that providers offer full-text search, then the client can specify field "ANY" which will be dispatched to all providers.

Both the entity type and filter are required, and any filtering is done on the Provider's side.

Dedicated JavaScript API

Client applications don't talk to providers directly, similarly to GNS, clients send requests to the GSS Desktop Manager (GSS DM) application and receive results over AGM streams.

GSS DM is responsible for multiplexing request to Providers and demultiplexing results back to client applications. It can also optionally implement local caching with configurable expiration.

JavaScript client applications talk to the GSS DM using a JavaScript API which internally utilises AGM Streaming.

The JavaScript API exposes interfaces for both search (GlueSearchService) and for creating search providers (GlueSearchProvider).

Multiple Asynchronous Results from Multiple Providers

When a search is performed, the GSS DM looks at the set of fields and finds the appropriate Search Providers. It then runs parallel queries against these providers. As each provider returns or starts streaming data, the DM will start streaming the results back to the client application.

Results of a Search Query are returned asynchronously and it is expected that different Providers will return results at different latency. Once all results from all Providers are delivered to the client application, the search query also fires a completion callback to indicate that the application should not expect more results. A provider can send multiple results for a single query if itself is multiplexing the requests so the client should expect more than one result from a provider.

Since one query can return multiple results from multiple Providers, each result is tagged with the Provider's name and also carries a flag indicating whether this is the last result from this Provider (in case the Provider batches results or searches across various sources of data internally).

Search Cancellation and Time Out

At any time, a client application can cancel a search request or change the query filter (e.g. user typing a few more characters).

Also, a client can specify a search timeout both globally or per query. If a search request times out (regardless of whether data was received), the query's state will become "timed out" and the completion callback will fire.

Push-based (AGM) and Request-based (REST) Protocols

GSS Providers can use 2 protocols to provide search results - AGM (subscription/push-based) and REST (request-based).

The GSS DM configuration is typically held in the Config Manager and can be configured with user-specific overrides and modified during runtime. As with GNS, local file configuration can also be used for development/test/demo purposes.

Limitations

• No filtering is performed on the GSS DM's side.
• No attempt is made to provide joins across Entity types in GSS.
• Transformations and aliasing in a projection are not supported by GSS.
• No conflation or throttling is performed in the GSS DM or by the JavaScript API

Architecture Diagram

+----------------------------------------------------------------------+
|                                                                      |
| [Desktop]                  +------------+                            |
|                            |    GSS     |                            |
| +----------+               |  Desktop   |               +----------+ |
| |    GSS   |  (subscribe)  |  Manager   |  (subscribe)  | Outlook  | |
| |  Client  |-------------->|            |-------------->| GSS AGM  | |
| |    App   |<- - - - - - - |(opt. cache)|<- - - - - - - | Provider | |
| +----------+    (push)     +------------+    (push)     +----------+ |
|                                 /\  \ ^                              |
+--------------------------------/--\--\-\-----------------------------+
| [Backend/Data Centre]         /    \  \ \_ _ _ _ _ _ _ _             |
|                       (poll) /      \  +------------+   \ (push)     |
|                             /        \   (subscribe) \   \           |
|                            v          v               v   |          |
|                       +----------+ +----------+ +------------+       |
|                       | Mosaic   | | MyBook   | | PBDS       |       |
|                       | REST GSS | | REST GSS | | AGM GSS    |       |
|                       | Provider | | Provider | | Provider   |       |
|                       +----------+ +----------+ +------------+       |
|                                                                      |
+----------------------------------------------------------------------+