url-examiner

A small Node.js tool & library that checks URL from a given CSV input file and report the analysis results in a new CSV output file.

Usage

Prerequisites

You need a working version of:

• Node v14+ (recommended v14.18.0)
• npm 6+ (recommended v6.14.15)

As a tool

# Basic
npx url-examiner -i input_file.csv -o output_file.csv

# Advanced (fullname)
npx url-examiner \ 
  --input input_file.csv \ 
  --output output_file.csv \
  --timeout 5000 \
  --headers "Authorization: Bearer xxx.yyy.zzz"
  --headers "User-Agent: Mozilla/5.0 (Windows NT 6.3; Win64; x64; rv:64.0) Gecko/20100101 Firefox/80.0"
  --from 10
  --to 200
  --bulk 50

# Advanced (shortcut)
npx url-examiner \ 
  -i input_file.csv \ 
  -o output_file.csv \
  -m 5000 \
  -H "Authorization: Bearer xxx.yyy.zzz"
  -H "User-Agent: Mozilla/5.0 (Windows NT 6.3; Win64; x64; rv:64.0) Gecko/20100101 Firefox/80.0"
  -f 10
  -t 200
  -b 50

Options:

| option | description | default value | | ------ | ----------- | ------------- | | -d --delay number | delay (in ms) between two URL calls | None | | -b --bulk number | number of concurrent HTTP calls during analysis phase | 10 | | -f --from number | line "from" | None | | -H --headers headers... | HTTP request headers | None | | (required) -i --input input_file | input file path | None | | -m --timeout max-time | max time allowed to succeed an HTTP check request | 1000 | | -o --ouput filepath | ouput file path | None | | -s --separator character | column separator | ; | | -t --to number | line "to" | None |

Input file:

Supported file extension: .csv

Supported data format (without head line):

<reference>;<url>

Output file:

As a library

npm install url-examiner

import Program from './Program';

const program = new Program();

program.run('my-app -i input_file.csv -o output_file.csv').then(() => process.exit(0));