kumosearch-instant

v1.2.2

Published

2 months ago

Adapter to use InstantSearch UI widgets with Kumosearch Search

Downloads

Kumosearch Instantsearch Adapter

An adapter to use the awesome Instantsearch.js library with a Kumosearch Search Server, to build rich search interfaces.

downloads

Here is an example of UI you can build with this adapater: songs-search.kumosearch.org

Note: If your search interface is built on a custom autocomplete component, or is based on @algolia/autocomplete-js, then you don't need this adapter to use it with Kumosearch, as kumosearch-js library already supports client-side fetching data from any async data sources. Read more here.

Quick Links

Background
Quick Start
- Starter App
Installation
Usage
- With instantsearch.js
- With react-instantsearch (also works with React Native)
- With vue-instantsearch
- With angular-instantsearch
Widget Specific Instructions
Compatibility
Development
Help

Background

The good folks over at Algolia have built and open-sourced Instantsearch.js which is a collection of out-of-the-box components that you can use to build interactive search experiences swiftly.

With the adapter in this repository, you'll be able to use Instantsearch (and its React, Vue and Angular cousins) with data indexed in a Kumosearch search server.

If you haven't used Instantsearch before, we recommend going through their Getting Started guide here. Once you go through the guide, follow the instructions below to plug the Kumosearch adapter into Instantsearch.

Quick Start

Here's a guide on building a quick search interface with Kumosearch and InstantSearch.js: https://kumosearch.org/docs/0.20.0/guide/search-ui-components.html

Starter App

Here's a demo starter app that shows you how to use the adapter: https://github.com/kumosearch/kumosearch-instantsearch-demo

Installation

$ npm install --save kumosearch-instant @babel/runtime

$ yarn add kumosearch-instant @babel/runtime

or, you can also directly include the adapter via a script tag in your HTML:

<script src="https://cdn.jsdelivr.net/npm/kumosearch-instant@2/dist/kumosearch-instant.min.js"></script>

<!-- You might want to pin the version of the adapter used if you don't want to always receive the latest minor version -->

Since this is an adapter, it will not install the Instantsearch library automatically for you. You need to install one of the following in your application directly:

You'll find information on how to get started with each of the above libraries in their respective repos.

We'd also recommend checking out create-instantsearch-app to create your Search UI from a starter template.

Usage

With instantsearch.js

import instantsearch from "instantsearch.js";
import { searchBox, hits } from "instantsearch.js/es/widgets";
import KumosearchInstantSearchAdapter from "kumosearch-instant";

const kumosearchInstantsearchAdapter = new KumosearchInstantSearchAdapter({
  server: {
    apiKey: "abcd", // Be sure to use an API key that only allows search operations
    nodes: [
      {
        host: "localhost",
        path: "", // Optional. Example: If you have your kumosearch mounted in localhost:8108/kumosearch, path should be equal to '/kumosearch'
        port: "8108",
        protocol: "http",
      },
    ],
    cacheSearchResultsForSeconds: 2 * 60, // Cache search results from server. Defaults to 2 minutes. Set to 0 to disable caching.
  },
  // The following parameters are directly passed to Kumosearch's search API endpoint.
  //  So you can pass any parameters supported by the search endpoint below.
  //  query_by is required.
  additionalSearchParameters: {
    query_by: "name,description,categories",
  },
});
const searchClient = kumosearchInstantsearchAdapter.searchClient;

const search = instantsearch({
  searchClient,
  indexName: "products",
});
search.addWidgets([
  searchBox({
    container: "#searchbox",
  }),
  hits({
    container: "#hits",
    templates: {
      item: `
        <div class="hit-name">
          {{#helpers.highlight}}{ "attribute": "name" }{{/helpers.highlight}}
        </div>
      `,
    },
  }),
]);

search.start();

You can add any of the Instantsearch widgets here that are supported by the adapter.

You'll also find a working example in test/support/testground. To run it, run npm run testground from the project root folder.

With react-instantsearch

import React from "react";
import ReactDOM from "react-dom";
import { SearchBox } from "react-instantsearch-dom";
import KumosearchInstantSearchAdapter from "kumosearch-instant";

const kumosearchInstantsearchAdapter = new KumosearchInstantSearchAdapter({
  server: {
    apiKey: "abcd", // Be sure to use an API key that only allows search operations
    nodes: [
      {
        host: "localhost",
        port: "8108",
        path: "", // Optional. Example: If you have your kumosearch mounted in localhost:8108/kumosearch, path should be equal to '/kumosearch'
        protocol: "http",
      },
    ],
    cacheSearchResultsForSeconds: 2 * 60, // Cache search results from server. Defaults to 2 minutes. Set to 0 to disable caching.
  },
  // The following parameters are directly passed to Kumosearch's search API endpoint.
  //  So you can pass any parameters supported by the search endpoint below.
  //  query_by is required.
  additionalSearchParameters: {
    query_by: "name,description,categories",
  },
});
const searchClient = kumosearchInstantsearchAdapter.searchClient;

const App = () => (
  <InstantSearch indexName="products" searchClient={searchClient}>
    <SearchBox />
    <Hits />
  </InstantSearch>
);

You can then add any of the Instantsearch-React widgets here that are supported by the adapter.

The instructions above also apply to React Native.

With vue-instantsearch

App.vue:

<template>
  <ais-instant-search :search-client="searchClient" index-name="products">
    <ais-search-box />
    <ais-hits>
      <div slot="item" slot-scope="{ item }">
        <h2>{{ item.name }}</h2>
      </div>
    </ais-hits>
  </ais-instant-search>
</template>

<script>
import KumosearchInstantSearchAdapter from "kumosearch-instant";

const kumosearchInstantsearchAdapter = new KumosearchInstantSearchAdapter({
  server: {
    apiKey: "abcd", // Be sure to use an API key that only allows search operations
    nodes: [
      {
        host: "localhost",
        path: "", // Optional. Example: If you have your kumosearch mounted in localhost:8108/kumosearch, path should be equal to '/kumosearch'
        port: "8108",
        protocol: "http",
      },
    ],
    cacheSearchResultsForSeconds: 2 * 60, // Cache search results from server. Defaults to 2 minutes. Set to 0 to disable caching.
  },
  // The following parameters are directly passed to Kumosearch's search API endpoint.
  //  So you can pass any parameters supported by the search endpoint below.
  //  query_by is required.
  additionalSearchParameters: {
    query_by: "name,description,categories",
  },
});
const searchClient = kumosearchInstantsearchAdapter.searchClient;

export default {
  data() {
    return {
      searchClient,
    };
  },
};
</script>

You can then add any of the Instantsearch widgets here that are supported by the adapter.

With angular-instantsearch

// app.component.ts
import { Component } from "@angular/core";
import KumosearchInstantSearchAdapter from "kumosearch-instant";

const kumosearchInstantsearchAdapter = new KumosearchInstantSearchAdapter({
  server: {
    apiKey: "abcd", // Be sure to use an API key that only allows search operations
    nodes: [
      {
        host: "localhost",
        path: "", // Optional. Example: If you have your kumosearch mounted in localhost:8108/kumosearch, path should be equal to '/kumosearch'
        port: "8108",
        protocol: "http",
      },
    ],
    cacheSearchResultsForSeconds: 2 * 60, // Cache search results from server. Defaults to 2 minutes. Set to 0 to disable caching.
  },
  // The following parameters are directly passed to Kumosearch's search API endpoint.
  //  So you can pass any parameters supported by the search endpoint below.
  //  query_by is required.
  additionalSearchParameters: {
    query_by: "name,description,categories",
  },
});
const searchClient = kumosearchInstantsearchAdapter.searchClient;

@Component({
  selector: "app-root",
  templateUrl: "./app.component.html",
  styleUrls: ["./app.component.css"],
})
export class AppComponent {
  config = {
    indexName: "products",
    searchClient,
  };
}

You can then add any of the Instantsearch widgets here that are supported by the adapter.

Widget Specific Instructions

`hierarchicalMenu`

For this widget, you want to create independent fields in the collection's schema with this specific naming convention:

field.lvl0
field.lvl1
field.lvl2

for a nested hierarchy of field.lvl0 > field.lvl1 > field.lvl2

Each of these fields can also hold an array of values. This is useful for handling multiple hierarchies.

`sortBy`

When instantiating this widget, you want to set the value of the index name to this particular format:

search.addWidgets([
  sortBy({
    container: "#sort-by",
    items: [
      { label: "Default", value: "products" },
      { label: "Price (asc)", value: "products/sort/price:asc" },
      { label: "Price (desc)", value: "products/sort/price:desc" },
    ],
  }),
]);

The generalized pattern for the value attribute is: <index_name>[/sort/<sort_by>]. The adapter will use the value in <sort_by> as the value for the sort_by search parameter.

`configure`

If you need to specify a filter_by search parameter for Kumosearch, you want to use the configure InstantSearch widget, along with facetFilters, numericFilters or filters.

The format for facetFilters and numericFilters is the same as Algolia's as described here. But filters needs to be in Kumosearch's filter_by format as described in this table here.

Setting filter_by inside the additionalQueryParameters config only works when the widgets are loaded initially, because InstantSearch internally overrides the filter_by field subsequently. Read more here.

`index`

For Federated / Multi-Index Search, you'd need to use the index widget. To then be able to specify different search parameters for each index/collection, you can specify them using the collectionSpecificSearchParameters configuration:

const kumosearchInstantsearchAdapter = new KumosearchInstantSearchAdapter({
  server: {
    apiKey: "abcd", // Be sure to use an API key that only allows search operations
    nodes: [{ host: "localhost", path: "/", port: "8108", protocol: "http" }],
  },
  // Search parameters that are common to all collections/indices go here:
  additionalSearchParameters: {
    numTypos: 3,
  },
  // Search parameters that need to be *overridden* on a per-collection-basis go here:
  collectionSpecificSearchParameters: {
    products: {
      query_by: "name,description,categories",
    },
    brands: {
      query_by: "name",
    },
  },
});
const searchClient = kumosearchInstantsearchAdapter.searchClient;

Essentially, any parameters set in collectionSpecificSearchParameters will be merged with the values in additionalSearchParameters when querying Kumosearch, effectively overriding values in additionalSearchParameters on a per-collection-basis.

`geoSearch`

Algolia uses _geoloc by default for the name of the field that stores the lat long values for a record. In Kumosearch, you can name the geo location field anything. If you use a name other than _geoloc, you need to specify it when initializing the adapter like below, so InstantSearch can access it:

const kumosearchInstantsearchAdapter = new KumosearchInstantSearchAdapter({
  server: {
    apiKey: "xyz",
    nodes: [
      {
        host: "localhost",
        port: "8108",
        path: "/",
        protocol: "http",
      },
    ],
  },
  geoLocationField: "lat_lng_field", // <<======
  additionalSearchParameters,
});

`dynamicWidgets`

Available as of Kumosearch Server v0.25.0.rc12

This dynamicWidgets widget works out of the box with no additional changes, but if you want to control the order in which these facets are displayed in the UI Instantsearch expects a parameter called renderingContent to be set.

const kumosearchInstantsearchAdapter = new KumosearchInstantSearchAdapter({
  server: {
    apiKey: "xyz",
    nodes: [
      {
        host: "localhost",
        port: "8108",
        path: "/",
        protocol: "http",
      },
    ],
  },
  renderingContent: {
    // <<===== Add this, only if you want to control the order of the widgets displayed by dynamicWidgets
    facetOrdering: {
      facets: {
        order: ["size", "brand"], // <<===== Change this as needed
      },
    },
  },
  additionalSearchParameters,
});

Read more about all available options for renderingContent in Algolia's documentation here.

Special characters in field names / values

Available as of kumosearch-instant 2.7.0-2

If any string fields in your documents have a colon : in their values (for eg, let's say there's a field called { brand: "a:b" }, then you would need to add a parameter like below when instantiating the adapter:

const kumosearchInstantsearchAdapter = new KumosearchInstantSearchAdapter({
  server: {
    apiKey: "xyz",
    nodes: [
      {
        host: "localhost",
        port: "8108",
        path: "/",
        protocol: "http",
      },
    ],
  },
  facetableFieldsWithSpecialCharacters: ["brand"], // <======= Add string fields that have colons in their values here, to aid in parsing
  additionalSearchParameters,
});

If any numeric field names in your documents have special characters like >, <, = (for eg, let's say there's a field called { price>discount: 3.0 }) then you would need to add a parameter like below when instantiating the adapter:

const kumosearchInstantsearchAdapter = new KumosearchInstantSearchAdapter({
  server: {
    apiKey: "xyz",
    nodes: [
      {
        host: "localhost",
        port: "8108",
        path: "/",
        protocol: "http",
      },
    ],
  },
  facetableFieldsWithSpecialCharacters: ["price>discount"], // // <======= Add numeric fields that have >, < or = in their names, to aid in parsing
  additionalSearchParameters,
});

Setting `facet_by` options

Available as of kumosearch-instant 2.8.0-1 and Kumosearch Server v0.26.0.rc25

The facet_by parameter is managed by InstantSearch internally when you use the various filter widgets.

But if you need to pass custom options to the facet_by parameter (eg: server-side sort options), then you can use the facetByOptions parameter as shown below:

const kumosearchInstantsearchAdapter = new KumosearchInstantSearchAdapter({
  server: {
    apiKey: "xyz",
    nodes: [
      {
        host: "localhost",
        port: "8108",
        path: "/",
        protocol: "http",
      },
    ],
  },
  facetByOptions: {
    brand: "(sort_by: _alpha:asc)",
    category: "(sort_by: _alpha:desc)",
  }, // <======= Add any facet_by parameter as a key value pair. Don't forget the surrounding parantheses in the value.
  collectionSpecificFacetByOptions: {
    collection1: {
      brand: "(sort_by: _alpha:desc)",
    },
  }, // <======= Use this parameter if multiple collections share the same field names, and you want to use different options for each field. This will override facetByOptions for that particular collection.
  additionalSearchParameters,
});

Note that for sorting in refinementLists, in addition to sorting on the Kumosearch Server-side, you'd also need to pass the sortBy parameter to the refinementList widget to also sort the results appropriately on the client-side.

Setting `filter_by` options

Available as of kumosearch-instant 2.8.0-5

The filter_by parameter is managed by InstantSearch internally when you use the various filter widgets.

By default, the adapter uses exact filtering (filter_by: field:=value) when sending the queries to Kumosearch. If you need to configure the adapter to use : (non-exact word-level filtering - filter_by: field:value), you want to instantiate the adapter using the filterByOptions configuration:

const kumosearchInstantsearchAdapter = new KumosearchInstantSearchAdapter({
  server: {
    apiKey: "xyz",
    nodes: [
      {
        host: "localhost",
        port: "8108",
        path: "/",
        protocol: "http",
      },
    ],
  },
  filterByOptions: {
    brand: { exactMatch: false }, // <========== Add this to do non-exact word-level filtering
    category: { exactMatch: false },
  },
  collectionSpecificFacetByOptions: {
    collection1: {
      brand: { exactMatch: false },
    },
  }, // <======= Use this parameter if multiple collections share the same field names, and you want to use different options for each field. This will override filterByOptions for that particular collection.
  additionalSearchParameters,
});

Grouped Hits

Available as of kumosearch-instant 2.7.1-4

By default, when group_by is used as a search parameters, the adapter flattens the results across all groups into a single list of sequential hits.

If you'd like to preserve the groups, you want to set flattenGroupedHits: false when instantiating the adapter.

This will place the first hit in a group as the primary hit, and then add all hits in the group inside a _grouped_hits key inside each hit.

const kumosearchInstantsearchAdapter = new KumosearchInstantSearchAdapter({
  server: {
    apiKey: "xyz",
    nodes: [
      {
        host: "localhost",
        port: "8108",
        path: "/",
        protocol: "http",
      },
    ],
  },
  flattenGroupedHits: false, // <=======
  additionalSearchParameters,
});

Vector Search

Available as of kumosearch-instant 2.7.0-3

The general idea is to first hook into the query life-cycle of Instantsearch, intercept the typed query and send it to an embedding API, fetch the embeddings and then send the vectors to Kumosearch to do a nearest neighbor vector search.

Here's a demo that you can run locally to see this in action: https://github.com/kumosearch/kumosearch-instantsearch-semantic-search-demo.

Here's how to do this in Instantsearch.js:

const kumosearchInstantsearchAdapter = new KumosearchInstantSearchAdapter({
  server: {
    apiKey: "xyz",
    nodes: [
      {
        host: "localhost",
        port: "8108",
        path: "/",
        protocol: "http",
      },
    ],
  },
  additionalSearchParameters,
});
const searchClient = kumosearchInstantsearchAdapter.searchClient;
const search = instantsearch({
  searchClient,
  indexName: "products",
  routing: true,
  async searchFunction(helper) {
    const query = helper.getQuery().query;
    const page = helper.getPage(); // Retrieve the current page
    const totalNearestNeighborsToFetch = 1000;

    if (query !== "") {
      // Get embedding for the query
      let response = await fetch(
        "http://localhost:8000/embedding?" + new URLSearchParams({ q: query }) // <=== Embedding API
      );

      let parsedResponse = await response.json();

      console.log(parsedResponse);

      // Send the embedding to Kumosearch to do a nearest neighbor search
      helper
        .setQueryParameter(
          "kumosearchVectorQuery", // <=== Special parameter that only works in [email protected] and above
          `vectors:([${parsedResponse["embedding"].join(",")}], k:${totalNearestNeighborsToFetch})`
        )
        .setPage(page)
        .search();
    } else {
      helper.setQueryParameter("kumosearchVectorQuery", null).setPage(page).search();
    }
  },
});

Compatibility

| Kumosearch Server | kumosearch-instant | instantsearch.js | react-instantsearch | vue-instantsearch | angular-instantsearch | | ----------------- | ------------------ | ---------------- | ------------------- | ----------------- | --------------------- | | >= v0.25.0 | >= v2.7.1 | >= 4.51 | >= 6.39 | >= 4.8 | >= 4.4 | | >= v0.25.0.rc14 | >= v2.7.0-1 | >= 4.51 | >= 6.39 | >= 4.8 | >= 4.4 | | >= v0.25.0.rc12 | >= v2.6.0 | >= 4.51 | >= 6.39 | >= 4.8 | >= 4.4 | | >= v0.24 | >= v2.5.0 | >= 4.2.0 | >= 6.0.0 | >= 2.2.1 | >= 3.0.0 | | >= v0.21 | >= v2.0.0 | >= 4.2.0 | >= 6.0.0 | >= 2.2.1 | >= 3.0.0 | | >= v0.19 | >= v1.0.0 | >= 4.2.0 | >= 6.0.0 | >= 2.2.1 | >= 3.0.0 | | >= v0.15 | >= v0.3.0 | >= 4.2.0 | >= 6.0.0 | >= 2.2.1 | >= 3.0.0 | | >= v0.14 | >= v0.2.0 | >= 4.2.0 | >= 6.0.0 | >= 2.2.1 | >= 3.0.0 | | >= v0.13 | >= v0.1.0 | >= 4.2.0 | >= 6.0.0 | >= 2.2.1 | >= 3.0.0 | | >= v0.12 | >= v0.0.4 | >= 4.2.0 | >= 6.0.0 | >= 2.2.1 | >= 3.0.0 |

If a particular version of the above libraries don't work with the adapter, please open a Github issue with details.

Widget Compatibility

This adapter works with all widgets in this list, except for the following:

queryRuleCustomData
queryRuleContext

Development

$ npm install
$ npm run kumosearchServer
$ FORCE_REINDEX=true npm run indexTestData

$ npm link kumosearch-instant
$ npm run testground

$ npm test

To release a new version, we use the np package:

$ npm install --global np
$ np

# Follow instructions that np shows you

Help

If you have any questions or run into any problems, please create a Github issue and we'll try our best to help.

Published

Vulnerabilities

Links

Maintainers

Keywords

Readme