search-text-tokenizer
v2.1.1
Published
A tokenizer for Google-like search queries
Downloads
55
Maintainers
Readme
search-text-tokenizer
A search query tokeniser inspired by Google.
- Split a space-delimitered query string into an array of terms
- Treat quoted terms as phrases
- Support tagged terms (tag:term)
- Detect excluded terms (-term)
Examples
var tokenizer = require( 'search-text-tokenizer' );
console.log( tokenizer( 'red bull' ) );
// [ { term: 'red' }, { term: 'bull' } ]
var result = console.log( tokenizer( '"red bull" "gives you wings"' ) );
// [ { term: 'red bull', phrase: true }, { term: 'gives you wings', phrase: true } ]
result = console.log( tokenizer( 'author:tolkien' ) );
// [ { term: 'tolkien', tag: 'author' } ]
result = console.log( tokenizer( '-car' ) );
// [ { term: 'car', exclude: true } ]
Installation
$ npm install search-text-tokenizer
Running test
To run the test suite enter these commands in the project directory.
$ npm install
$ npm test
License
MIT © Tatsuya Oiwa, Dannii Willis, James Anthony Bruno