bulksearch
v0.1.3
Published
Superfast, lightweight and read-write optimized full text search library.
Downloads
72
Readme
BulkSearch
Superfast, lightweight and read-write optimized full text search library.
When it comes to the overall speed, BulkSearch outperforms every searching library out there and also provides flexible search capabilities like multi-word matching, phonetic transformations or partial matching. It is basically based on how a hdd manage files on the filesystem. Adding, updating or removing items are also fast as searching for them, but also requires some additional amount of memory. When your index don't needs to be updated continuously then FlexSearch may be a better choice. BulkSearch also provides you a asynchronous processing model to perform queries in background.
Benchmark:
- Comparison: https://jsperf.com/compare-search-libraries
- Detailed: https://jsperf.com/bulksearch
Supported Platforms:
- Browser
- Node.js
Supported Module Definitions:
- AMD (RequireJS)
- CommonJS (Node.js)
- Closure (Xone)
- Global (Browser)
All Features:
Installation
HTML / Javascript
<html>
<head>
<script src="js/bulksearch.min.js"></script>
</head>
...Note: Use bulksearch.min.js for production and bulksearch.js for development.
Use latest from CDN:
<script src="https://cdn.rawgit.com/nextapps-de/bulksearch/master/bulksearch.min.js"></script>Node.js
npm install bulksearchIn your code include as follows:
var BulkSearch = require("bulksearch");Or pass in options when requiring:
var index = require("bulksearch").create({/* options */});AMD
var BulkSearch = require("./bulksearch.js");Compare BulkSearch vs. FlexSearch
API Overview
Global methods:
- BulkSearch.create(<options>)
- BulkSearch.addMatcher({KEY: VALUE})
- BulkSearch.register(name, encoder)
- BulkSearch.encode(name, string)
Index methods:
- Index.add(id, string)
- Index.search(string, <limit>, <callback>)
- Index.search(string, <page>, <callback>)
- Index.search(options, <callback>)
- Index.update(id, string)
- Index.remove(id)
- Index.reset()
- Index.destroy()
- Index.init(<options>)
- Index.optimize()
- Index.info()
- Index.addMatcher({KEY: VALUE})
- Index.encode(string)
Usage
Create Index
BulkSearch.create(<options>)
var index = new BulkSearch();alternatively you can also use:
var index = BulkSearch.create();Create index with custom options
var index = new BulkSearch({
// default values:
type: "integer",
encode: "icase",
boolean: "and",
size: 4000,
multi: false,
strict: false,
ordered: false,
paging: false,
async: false,
cache: false
});Read more: Phonetic Search, Phonetic Comparison, Improve Memory Usage
Add items to an index
Index.add(id, string)
index.add(10025, "John Doe");Search items
Index.search(string|options, <limit|page>, <callback>)
index.search("John");Limit the result:
index.search("John", 10);Perform queries asynchronously:
index.search("John", function(result){
// array of results
});Pass parameter as an object:
index.search({
query: "John",
page: '1:1234',
limit: 10,
callback: function(result){
// async
}
});Update item from an index
Index.update(id, string)
index.update(10025, "Road Runner");Remove item from an index
Index.remove(id)
index.remove(10025);Reset index
index.reset();Destroy index
index.destroy();Re-Initialize index
Index.init(<options>)
Note: Re-initialization will also destroy the old index!
Initialize (with same options):
index.init();Initialize with new options:
index.init({
/* options */
});Add custom matcher
BulkSearch.addMatcher({REGEX: REPLACE})
Add global matchers for all instances:
BulkSearch.addMatcher({
'ä': 'a', // replaces all 'ä' to 'a'
'ó': 'o',
'[ûúù]': 'u' // replaces multiple
});Add private matchers for a specific instance:
index.addMatcher({
'ä': 'a', // replaces all 'ä' to 'a'
'ó': 'o',
'[ûúù]': 'u' // replaces multiple
});Add custom encoder
Define a private custom encoder during creation/initialization:
var index = new BulkSearch({
encode: function(str){
// do something with str ...
return str;
}
});Register a global encoder to be used by all instances
BulkSearch.register(name, encoder)
BulkSearch.register('whitespace', function(str){
return str.replace(/ /g, '');
});Use global encoders:
var index = new BulkSearch({ encode: 'whitespace' });Call encoders directly
Private encoder:
var encoded = index.encode("sample text");Global encoder:
var encoded = BulkSearch.encode("whitespace", "sample text");Mixup/Extend multiple encoders
BulkSearch.register('mixed', function(str){
str = this.encode("icase", str); // built-in
str = this.encode("whitespace", str); // custom
return str;
});BulkSearch.register('extended', function(str){
str = this.encode("custom", str);
// do something additional with str ...
return str;
});Get info
index.info();Returns information about the index, e.g.:
{
"bytes": 103600,
"chunks": 9,
"fragmentation": 0,
"fragments": 0,
"id": 0,
"length": 7798,
"matchers": 0,
"size": 10000,
"status": false
}Note: When the fragmentation value is about 50% or higher, your should consider using cleanup().
Optimize / Cleanup index
Optimize an index will free all fragmented memory and also rebuilds the index by scoring.
index.optimize();Pagination
Note: Pagination can simply reduce query time by a factor of 100.
Enable pagination on initialization:
var index = BulkSearch.create({ paging: true });Perform query and pass a limit (items per page):
index.search("John", 10);The response will include a pagination object like this:
{
"current": "0:0",
"prev": null,
"next": "1:16322",
"results": []
}Explanation:
Perform query and pass a pointer to a specific page:
index.search("John", {
page: "1:16322", // pointer
limit: 10
});Options
Phonetic Encoding
Compare Phonetic Search
Reference String: "Björn-Phillipp Mayer"
Memory Usage
Note: The data type of passed IDs has to be specified on creation. It is recommended to uses the most lowest possible data range here, e.g. use "short" when IDs are not higher than 65,535.
Author BulkSearch: Thomas Wilkerling License: Apache 2.0 License