Fork Notes

• Updated all dependencies
• Stripped it down to just the essential features (removed toFile, transform, etc. since you can do those yourself)
• Rewrote all of the streaming behavior
• Fixed issues with handling errors
• Promises to be responsive to PRs

All thanks and credit to the original author.

got-resume.js

Fetch via HTTP/HTTPS using got with automatic resume after network failures

Current status

Usage

Use got to make an HTTP request with automatic retries for network errors.

Designed for downloading large files. If transfer fails part-way through, will retry, resuming from point where previous attempt finished, using HTTP range headers.

gotResume( [url], [options] ) -> Stream

const stream = gotResume('http://google.com/')
stream.pipe(fs.createWriteStream('foo.html'))

stream.on('error', err => console.log('Failed!'))
stream.on('end', () => console.log('Finished!'))

Options

url

Alternative way to provide URL.

const stream = gotResume({ url: 'http://google.com/' })

attempts

Max number of attempts in a row yielding no data (i.e. failed connection, empty response) before aborting.

Set to 0 for no limit. Default 10.

const stream = gotResume('http://google.com/', { attempts: 0 })

attemptsTotal

Max number of total attempts before aborting.

Set to 0 for no limit. Default 0.

backoff

Function to determine wait in milliseconds before retry. Called with arguments (attempt, transfer).

attempt is what attempt number for current chunk (reset to zero when a new chunk is successfully received).

transfer is the internal Transfer object (see below).

If function returns false, the transfer is cancelled. If using this mechanism, options.attempts should be set to 0 so it does not interfere.

If not provided, default backoff function starts with 1000ms and doubles each time:

function backoff(attempt) {
  return Math.pow(2, attempt - 1) * 1000
}

length

Length of response expected in bytes. If undefined, length will be determined from HTTP content-length header.

If server does not provide content-length header, and options.length is not set, transfer will be considered complete when first successful request completes.

If options.length is set, only that number of bytes will be fetched (i.e. file will be truncated).

needLength

Set to true if you require the length of the transfer to be retrieved at start of the transfer. Default: false

Explanation: By default got will use transfer encoding (e.g. gzip). This makes the content-length HTTP header unreliable. Setting options.needLength disables encoding so length should be retrieved accurately (if server provides it).

pre

An async function that is run before each chunk request. Must return a Promise. Request will commence once promise resolves.

Useful where some authentication requires being set up before the transfer HTTP request, or where resource has a different URL each time (e.g. some file transfer services).

pre function is called with Transfer object (see below). To set URL for next chunk, pre should set transfer.url. To alter got options, should set transfer.gotOptions.

function pre(transfer) {
  transfer.gotOptions.headers['user-agent'] = 'Stealth 2.0'
  return Promise.resolve()
}

log

Function to receive logging information e.g. HTTP responses.

const stream = gotResume('http://google.com/', { log: console.log })

got

Options to pass to got. See got documentation for details.

const stream = gotResume('http://google.com/', { got: { method: 'POST' } })

Events

error

Emitted with a gotResume.TransferError on stream when transfer fails and has exhausted retries.

end

Emitted when transfer completes.

NB Is also emitted after error event if transfer fails.

progress

Emitted when data received. Emitted with progress object of form {transferred: 100, total: 3000}.

request

Emitted with HTTP request object when first HTTP request made to server.

NB Not emitted again for each retry HTTP request. You cannot abort the transfer with request.abort() as the request may be finished if a retry has happened.

response

Emitted when first successful HTTP response is received. NB Not emitted again for each retry HTTP request.

Useful for e.g. determining length of transfer:

const stream = gotResume('http://google.com/')
stream.on('response', res => console.log('Length: ', stream.transfer.length))

Cancellation

The stream returned by gotStream() has an additional method .cancel(). Calling .cancel() will abort the transfer and cause the stream to emit an error event with a gotResume.CancelError.

If the transfer is complete before .cancel() is called, no error event will be emitted.

If options.pre function is supplied and .cancel() is called while options.pre is running, .cancel() method on the promise returned by options.pre will be called if it exists. Otherwise the transfer will abort once the promise resolves.

Transfer object

A transfer in progress is represented internally as an instance of gotResume.Transfer class.

Transfer object is stored as stream.transfer and also passed to options.pre function.

Tests

Use npm test to run the tests. Use npm run cover to check coverage.

Few tests so far but seems to work fine!