scraperkit
v0.0.3-alpha.3
Published
Common utility functions that regularly need in web scrapper apps.
Downloads
3
Readme
Scraperkit
Common utility functions that regularly need in web scrapper apps.
getHTML
const { getHTML } = require('scraperkit');
const res = await getHTML('https://example.com', {
prerender: true,
puppeteerLaunchConfig: {
executablePath: `/Applications/Google Chrome.app/Contents/MacOS/Google Chrome`
}
});
console.log(res);
getSelectors
const htmlContent = `<html>...</html>`;
const selectors = getSelectors(htmlContent, ['meta', 'h1']);
TODO
- [ ] Add proxy support for getHTML
- [ ] Make it easy to debug issues
Resources
- https://github.com/axios/axios/issues/925