raspar

A simple to use Promise-based web scraper with local caching.

Usage

import fetch from 'raspar';

const contents = await fetch('http://www.google.com/humans.txt');

console.log(contents);

Options

import fetch from 'raspar';

const options = {
  cacheDirectory: 'temp/cache/',
  requestOptions: {
    headers: {
      'User-Agent': 'request'
    },
    method: 'POST'
  },
  ttl: 1800
};

const contents = await fetch('http://www.google.com/humans.txt', options);

console.log(contents);

| Name | Description | Default Value | | -------------- | ------------------------------------------------------------------------------------------------------------------------------- | ------------- | | cacheDirectory | Directory to store cache. | temp/cache/ | | requestOptions | Request options object. Read more github.com/node-fetch/node-fetch | {} | | ttl | TTL (Time to live) in seconds. | 1800 |