grabby
v0.2.2
Published
Enhanced request library with some specific cases like win1251 encoding and encoding support.
Downloads
27
Maintainers
Readme
Grabby
Features
- request compatibility
- promise interface
- encoding detection(utf8, win1251)
- compressing detection(gzip, deflate)
- debug support
Install
npm install grabby --save
Nice and fancy
var grabby = require('grabby');
// same as original node request
var request = {url: 'http://yandex.ru'};
// returns Promise
grabby.requestHtml(request).then(function (html) {
// for example parse http with cheerio
var $ = cheerio.load(html);
});
Advanced usage
var grabby = require('grabby');
// set fail reason and grabby will continue to try
// useful if you meet nginx with request per minute limit
grabby.requestHtml({
url: '',
attempt: {
reason: [409], // status code
reason: function (error, response) { // or your own reason
/* check response */
},
delay: 100, // constant in ms
delay: function (n) { // or even your custom value
return 100 * n;
},
limit: 10 // reject promise after 10 tries
}
});
Debug support
Debug mode is provided through the use of the debug module. To enable:
DEBUG=grabby node your_program.js
or even in your script
// set debug environment
process.env['debug'] = 'grabby';
Read the debug module documentation for more details.
Tests
npm test