Inquery

A search engine library implemented with BM25

How it Works

Inquery is a search engine which quickly retrieves pre-indexed documents. It is utilized by taking a set of documents (typically in the form of an object) and indexing them with an id. Later, the search method of Inquery can be used to retrieve an array of relevant ids sorted in order of relevance. The search method will allow a user's search query to have typos, or incomplete search terms. This means that, paired with the speed of Inquery, it is achievable, and not highly taxing to search over the documents as a user types.

Getting Started

Start using inquery by installing it to your project.

Apples-MacBook-Air:~ timfarrow$ npm install --save @spacebartech/inquery

Then, in your project's file, include Inquery and initialize it to begin using it.

const Inquery = require( '@spacebartech/inquery' );

const index = new Inquery();

Now that your Inquery is initialized, you can index any array of documents. In this example, I will read a set of documents from Firebase and then use Inquery to pre-index them, thus preparing Inquery for a search.

const Inquery = require( '@spacebartech/inquery' );
const Admin   = require( 'firebase-admin' );

const index = new Inquery();

Admin.database().ref( '/' )
  .child( 'organizations' )
  .child( 'Madera Unified' )
  .child( 'storage' )
  .child( 'events' )
  .on( 'value', ( snapshot ) => {

    // this object of events contains the "documents"
    // which we will index with Inquery;
    const events   = snapshot.val();
    const eventIds = Object.keys( events );

    eventIds.forEach( ( id ) => {

      // add a document to the index
      index.addDocument( events[id], id );

    } );

  } );

By calling Inquery's addDocument method, we index each of the events we retrieved from Firebase. After each event has been indexed, we can now use Inquery's search method to retrieve an array of results. The following example will be of an Express route which accepts a query and returns an array of results

router.post( '/search', ( req, res, next ) => {

  const { query } = req;
  const results   = index.search( query ); // array of objects ordered by relevance containing a cache of the document indexed;

  res.send( {
    status : 200,
    body   : results,
  } );

} );

That's about all there is to it! Happy searching!

Reference

Complete reference of all methods

addDocument

@param document: String|Object

@param id: String

returns undefined;

This method will add a document to the index and then cache it. After adding a document to your index, it will now be considered for any call of Inquery's search method.

Search

@param query: String

returns Object[];

This method takes a search query and finds all relevant, previously added documents. It will return an array of objects which look like:

{
  relevance: Float,
  document : Object|String // depending on what you used in the addDocument method
}

getIndex

No params

returns Object;

This method will export the indexed terms list which will vaguely resemble this pattern

{
  'tokenizedTerm' : {
    foundIn : [
      {
        id    : String, // key of indexed document
        count : Number, // number of times the term is found in the document
        idf   : Float,  // the inverse document frequency of the term (e.g. the term occurs three times, of three hundred total terms, the idf is 3/300 or .1)
                        // this number is used to help determine relevance
      },
      ...
    ]
  },
  ...
}