web-get-js

An adaptation of wget in Node.js module to download files from the web, with support for processing data before saving.

Installation

To install the module, simply type:

npm install web-get-js

Usage

Basic Usage

You can use the wget function to download files from a URL and save them to the specified location.

Examples

Example 1: Simple URL download with callback

Downloads the file from the provided URL and saves it to the current directory.

const { wget } = require('web-get-js');

const url = 'https://example.com/file.zip';

wget(url, (err) => {
    if (err) {
        console.error('Download failed:', err);
    } else {
        console.log('Download complete');
    }
});

Example 2: URL download with callback to process data

Downloads the file from the provided URL, and you can process the data in the callback.

const { wget } = require('web-get-js');

const url = 'https://example.com/file.zip';

wget(url, (err, data) => {
    if (err) {
        console.error('Download failed:', err);
    } else {
        console.log('Download complete with data:', data);
    }
});

Example 3: Download with options specifying destination

Downloads the file from the provided URL and saves it to the specified destination.

const { wget } = require('web-get-js');

const url = 'https://example.com/file.zip';
const dest = './downloads/file.zip';

wget({ url: url, dest: dest }, (err) => {
if (err) {
console.error('Download failed:', err);
} else {
console.log('Download complete');
}
});

Example 4: Dry run download with callback to process data

Does not download the file but retrieves the data from the provided URL for processing in the callback.

const { wget } = require('web-get-js');

const url = 'https://example.com/file.zip';

wget({ url: url, dry: true }, (err, data) => {
if (err) {
console.error('Dry run failed:', err);
} else {
console.log('Dry run data:', data);
}
});

Example 5: Download with processing data before saving

Downloads the file from the provided URL, processes the data in the callback, and saves the processed data to the specified destination.

const { wget } = require('web-get-js');

const url = 'https://example.com/file.zip';
const dest = './downloads/file.zip';

wget({ url: url, dest: dest, process: true }, (err, data) => {
if (err) {
console.error('Download failed:', err);
} else {
console.log('Data to process:', data);
const dataToSave = data.toString() + " end of data.";
return dataToSave; // Processed data will be saved instead of the original data
}
});

Function Descriptions

wget

Main wget function that supports multiple signatures for downloading files.

Parameters:

options - URL as a string or an object containing options.
    {
        url: "https://example.com/file.zip" || new URL("/file.zip", "https://example.com/"); ,
        dest: "./path-to-your-destination/file.zip",
        dry: true || false,
        process: true || false,
    }
callback - The callback function to be called once the download is complete or in dry-run mode.

License

This project is licensed under the MIT License.