zappos-product-crawler
v2.0.0
Published
Crawls Zappos.com to get product info with breadcrumb attributes
Downloads
6
Readme
zappos-product-crawler
Gets Zappos product list from a search link with breadcrumb attributes included.
First, do:
npm install
then
for Zappos:
node zappos.js <zapposSearchLink> <searchPageLimit>
If you do not put search page limit, it defaults to 5 pages.
To run:
- If you want the result pages to be returned faster, go to setTimeout( next, 500 ); line and change 500 to some other number in milliseconds
- If you want to change how many product pages are simultaneously crawled at any given time, in the line with async.mapLimit( merged, 10, function ( item, callback ) change 10 to some other integer.