npm package discovery and stats viewer.

Discover Tips

  • General search

    [free text search, go nuts!]

  • Package details

    pkg:[package-name]

  • User packages

    @[username]

Sponsor

Optimize Toolset

I’ve always been into building performant and accessible sites, but lately I’ve been taking it extremely seriously. So much so that I’ve been building a tool to help me optimize and monitor the sites that I build to make sure that I’m making an attempt to offer the best experience to those who visit them. If you’re into performant, accessible and SEO friendly sites, you might like it too! You can check it out at Optimize Toolset.

About

Hi, 👋, I’m Ryan Hefner  and I built this site for me, and you! The goal of this site was to provide an easy way for me to check the stats on my npm packages, both for prioritizing issues and updates, and to give me a little kick in the pants to keep up on stuff.

As I was building it, I realized that I was actually using the tool to build the tool, and figured I might as well put this out there and hopefully others will find it to be a fast and useful way to search and browse npm packages as I have.

If you’re interested in other things I’m working on, follow me on Twitter or check out the open source projects I’ve been publishing on GitHub.

I am also working on a Twitter bot for this site to tweet the most popular, newest, random packages from npm. Please follow that account now and it will start sending out packages soon–ish.

Open Software & Tools

This site wouldn’t be possible without the immense generosity and tireless efforts from the people who make contributions to the world and share their work via open source initiatives. Thank you 🙏

© 2024 – Pkg Stats / Ryan Hefner

wade

v0.3.3

Published

Blazing fast, 1kb search library

Downloads

2,679

Readme

Wade

Blazing fast, 1kb search

Build Status

Installation

NPM

npm install wade

CDN

<script src="https://unpkg.com/wade"></script>

Usage

Initialize with strings in the form of an array

const search = Wade(["Apple", "Orange", "Lemon", "Tomato"]);

Now you can search for a substring within the array, and Wade will return the index of it.

search("App");
/*
[{
  index: 0,
  score: 1
}]
*/

Combined with libraries like Moon, you can create a simple real-time search.

Loading/Saving Data

To save data as an object, use Wade.save on your search function, and then use these later when initializing Wade.

For example:

// Create the initial search function
const search = Wade(["Apple", "Orange", "Lemon", "Tomato"]);
const instance = Wade.save(search);

// Save `instance` somewhere...

Later, you can get the same search function without having Wade recreate an index every time by doing:

// Retrieve `instance`, then
const search = Wade(instance);

Pipeline

Wade uses a pipeline to preprocess data and search queries. By default, this pipeline will:

  • Make everything lowercase
  • Remove punctuation
  • Remove stop words

A pipeline consists of different functions that process a string and modify it in some way, and return the string.

You can easily modify the pipeline as it is available in Wade.pipeline, for example:

// Don't preprocess at all
Wade.pipeline = [];

// Add custom processor to remove periods
Wade.pipeline.push(function(str) {
  return str.replace(/\./g, "");
});

All functions will be executed in the order of the pipeline (0-n) and they will be used on each document in the data.

The stop words can be configured to include any words you like, and you can access the array of stop words by using:

Wade.config.stopWords = [/* array of stop words */];

The punctuation regular expression used to remove punctuation can be configured with:

Wade.config.punctuationRE = /[.!]/g; // should contain punctuation to remove

Algorithm

The algorithm behind the search is fairly simple. First, a trie data structure is generated off of the data. When performing a search, the following happens:

  • The search query is processed through the pipeline
  • The search query is then tokenized into keywords
  • Each keyword except the last is searched for and scores for each item in the data are updated according to the amount of keywords that appear in the document.
  • The last keyword is treated as a prefix, and Wade performs a depth-first search and updates the score for all data prefixed with this keyword. The score is added depending on how much of the word was included in the prefix. This allows for searching as a user types.

License

Licensed under the MIT License by Kabir Shah