url-scraper
v1.0.2
Published
Url scraper which takes the text input and finds the links/urls, scraps them using cheerio and will returns an object with original text, parsed text (using npm-text-parser) and array of objects where each object contains scraped webpage's information.
Downloads
16
Maintainers
Readme
url-scraper
Url scraper which takes the text input and finds the links/urls, scraps them using cheerio and will returns an object with original text, parsed text (using npm-text-parser) and array of objects where each object contains scraped webpage's information.
Installation
npm i url-scraper
Usage
var urlScraper = require('url-scraper');
scrap(text)
Receives the input text and finds the links/url, scraps them and returns an object with original, parsed and array of scrapped websites info.
var inputString = "This is awesome it parses the url's dude and http://krishcdbry.com done !"
urlScraper
.scrap(inputString)
.then(function(response) {
console.log(response); // It returns the response object when promise gets resolved satisfies.
});
//{
// original_text: 'This is awesome it scraps the sites dude and http://heartynote.com done !',
// parsed_text: 'This is awesome it scraps the sites dude and <a href="http://heartynote.com" target="_blank">http://heartynote.com</a> done !',
// scraped_data:
// [
// { domain: 'heartynote.com',
// title: 'heartynote welcomes u !!',
// description: 'Bring your life',
// thumb: 'http://heartynote.com/pngs/thums/hearty.png',
// canonical: 'http://krishcdbry.com',
// isValid: true,
// _links: {
// self : http://krishcdbry.com
// }
// }
// ]
// }
Demo
Demo @url-scraper | https://tonicdev.com/npm/url-scraper
Author
Krishcdbry [[email protected]]
Licence
MIT @krishcdbry