is-web-crawler
v1.1.0
Published
A light weight JS library to check if a user agent is a web crawler.
Downloads
200
Readme
is-web-crawler
A javascript based library to check if the current user agent is a web crawler. It uses the list of known web crawlers using the package crawler-user-agents.
Get started
npm install is-web-crawler
Usage
Quick Usage
If you want to know if the current user agent is a web crawler.
import { isCrawlerUserAgent } from 'is-web-crawler';
const isCurrentUserAgentABot = isCrawlerUserAgent();
NOTE: The regex pattern matching under the hood is cached, so calling this method multiple times will not affect performance.
Only use the top X known crawlers
If you only want to tackle the top X web crawlers (from the list provided by crawler-user-agents), then simply pass in the number as an argument:
const topCrawlers = 10;
const isCurrentUserAgentABot = isCrawlerUserAgent(topCrawlers);
Checking a custom user agent string
There might a scenario where you want to check a custom user agent string, instead of the current user agent. In this case, you can use the isCrawler
method instead.
import { isCrawlerUserAgent } from 'is-web-crawler';
const myUserAgent = ... get user agent ...;
const topCrawlers = 10;
// the second argument is optional
const isCurrentUserAgentABot = isCrawler(myUserAgent, topCrawlers);
NOTE: This method by passes the caching, so it if you need to cache the value, you need to implement this in your application.