finscrapers

Finscrapers is a Node-based CLI that helps investors to scrape data from websites with Puppeteer. That is how private investors can take sophisticated decisions based on facts, instead of solely gut instincts. Without paying thousands of dollars for a Bloomberg Terminal. Instead Finscrapers will collect data from a financial website and save them to Elasticsearch to display them in any report, or dashboard. The best is, it's free and open source!

Currently supported financial sources:

• Yahoo Finance

Installation

Add Finscrapers as global package.

npm install -g @finscrapers/finscrapers

Usage

1. See --help

Run the script with Node and you will see flags listed below.

finscrapers --help`

| Flag | Required | Description | Example | | ------------------- | ------------------------------- | ----------------------------------------------------------------------- | --------------------------------- | | --elasticsearch, -e | yes | Path to Elasticsearch host, default is http://localhost:9200 | https://myelasticsearchserver.com | | --symbols, -s | no (defaults to ./symbols.json) | Relative path of the files with the symbols to scrape | ~/symbols.json | | --headless, -h | no | Runs Puppeteer in headless mode (you will not see the browser) | | | --slowMo, -m | no (defaults to 300) | Slows down Puppeteer operations by the specified amount of milliseconds | 1000 |

2. Start Elasticsearch server

Take the manual installation route with Elasticsearch Reference, easier, but more costly, subscribe Elasticsearch Service or AWS Elasticsearch Service.

3. Configure symbols.json

A file containing the symbols needs to be created before executing the script. Let's create a file called symbols.json:

{
  "symbols": ["IBM", "MSFT", "FB", "AMZN"]
}

4. Run Finscrapers:

finscrapers -e https://myelasticsearchserver.com -s symbols.json

5. Run Kibana:

You need to install Kibana, as mentioned in the section Start Elasticsearch server, investigate the data, and finally invest like Warren Buffet. If you get rich, don't forget to buy us a coffee.

Legal disclaimer

We are not affiliated with Oath (EMEA) Limited. Please make yourself familiar with Yahoo Terms before using finscrapers-cli.

Contributing

Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.

Please make sure to update tests as appropriate.

License

MIT