@chcaa/spa-scraper
v1.0.0
Published
CLI to scrape SPAs
Downloads
2
Readme
SPA Scraper
A tool for scraping single page applications
Installation
- Install node.js version 16.x or higher
Usage
Run one of the following:
$ npx @chcaa/spa-scraper@latest ...
Local Usage
- Clone this repository
- Navigate to the root of the repository and run
$ npm install
$ node src/index.js ...
CLI options
-I, --include-directories <list>
[optional] - Specify a comma-separated list of directories you wish to follow when downloading.-q, --query-parameters
[optional] - If you want to handle pages with different query parameters differently.-s, --selector <selector>
[optional] - Specify a CSS selector that defines when the page is loaded in. By default it waits until the network is idle, but that might not work if an unreachable resource has an indefinite timeout.
Example
$ npx @chcaa/spa-scraper@latest https://example.dk/ -I /dir1/,/dir2/
node src/index.js https://example.dk/ -I /dir1/,/dir2/