xw-spikingss
v2.1.5
Published
A lightweight web crawler.
Downloads
12
Maintainers
Readme
Spiking
Chinese | English
A common web crawler tool developed using Node.js.
Advantage
- Graphical Interface (Electron)
- Custom initial link
- Headless browser support (puppeteer)
- Concurrent / multithreaded
- Efficient crawling mechanism
- Automatic similarity
- Progress visualization
- Using a proxy
Getting started
git clone https://github.com/Xciyang/spiking.git
cd spiking
npm install
npm start
Usage
Download the latest [Release] (https://github.com/Xciyang/spiking/releases) version.
Please read usage: English(en_US)
To-do list
- Localization
- Tasks Saving
- Custom filtering method