patofinder
v1.0.0
Published
An elegant way to search for urls with web scraping and javascript
Downloads
616
Maintainers
Readme
Patofinder
Patofinder is a open-source node library (with CLI included) that resolve urls of a page from a get request. It doesnt just get all paths, but also filter and look for specific types or extensions.
Installation
Use the node package manager NPM to install it. You can install it locally to import the into your javascript/typescript code or install it globally and use it directly as a command:
npm i patofinder -g
npm i patofinder
Usage
Import the find function into your js/ts code and pass the url target and the selected search type:
import * as patofinder from 'patofinder';
async function foo() {
/*
* Get all image paths from foobar.com
*/
const images = await patofinder.find('http://www.foobar.com/', 'img');
console.log(images); // ['http://www.foobar.com/logo.jpg', 'http://www.foobar.com/sample.png']
}
foo();
Or run directly from your prompt:
patofinder --path http://www.foobar.com/ --type audio
Errors
The library just throw a error. If a request fails or the provided path is invalid, it will just return a empty array, but if a invalid type was provided, a error will be throw:
import * as patofinder from 'patofinder';
async function foo() {
const valid = await patofinder.find('http://www.foobar.com/', 'css');
const invalid = await patofinder.find('http://www.invalidpath.org/', 'php');
console.log(valid); // ['http://www.foobar.com/styles/main.css']
console.log(invalid); // []
await patofinder.find('http://www.foobar.com/', 'foo'); // error
}
foo();
Examples
The library has few functions and commands, but you can see some use cases in the docs folder:
- Available command line calls: see cli examples
- Available types and extensions: see type examples
- Available find examples: see find examples
- Available regexp examples: see regexp examples
- Available interfaces examples: see interfaces examples
Live
On the release of version 1.0.0 a test sample api was created on the heroku servers. See here for more.
Contributing
Pull requests are welcome. For major changes or TypeSearches creation please open an issue first to discuss what you would like to change. You can also contribute by reporting errors or suggesting new features here.