Americas Test Kitchen Downloader

Command-line utility for downloading an offline copy of Americas Test Kitchen HLS video streams.

Features:

• accepts URLs that identify:
  • a single episode
• accepts CLI options to identify:
  • a single episode
  • a single season
  • all episodes in every season
• downloads:
  • the highest available quality for each video stream
  • vtt subtitles for all available languages
  • will continue upon restart after an abrupt interruption
• resulting file structure:

    |- {title_series}/
    |  |- {title_episode}/
    |  |  |- hls/
    |  |  |  |- video/
    |  |  |  |  |- *.ts
    |  |  |  |- audio/
    |  |  |  |  |- {language}/
    |  |  |  |  |  |- *.ts
    |  |  |  |  |- {language}.m3u8
    |  |  |  |- video.m3u8
    |  |  |  |- master.m3u8
    |  |  |- mp4/
    |  |  |  |- video.mp4
    |  |  |  |- video.{language}.vtt
    |  |  |  |- video.{language}.srt

Installation:

npm install --global @warren-bank/node-hls-downloader-americastestkitchen

Usage:

atkdl <options>

options:
========
"-h"
"--help"
    Print a help message describing all command-line options.

"-v"
"--version"
    Display the version.

"-q"
"--quiet"
    Do not print a verbose log of operations.

"-ll" <integer>
"--log-level" <integer>
    Specify the log verbosity level.
      0 = no output (same as --quiet)
      1 = include only episode URLs
      2 = include only episode ffmpeg commands
      3 = include all operational metadata (default)

"-dr"
"--dry-run"
    Do not write to the file system.

"-nm"
"--no-mp4"
    Do not use "ffmpeg" to bundle the downloaded video stream into an .mp4 file container.

"-mc" <integer>
"--max-concurrency" <integer>
"--threads" <integer>
    Specify the maximum number of URLs to download in parallel.
    The default is 1, which processes the download queue sequentially.

"-P" <dirpath>
"--directory-prefix" <dirpath>
    Specifies the directory where the resulting file structure will be saved to.
    The default is "." (the current directory).

"-u" <URL>
"--url" <URL>
    Specify an episode URL.

"-i <filepath>"
"--input-file <filepath>"
    Read episode URLs from a local text file. Format is one URL per line.

"-a"
"--all"
    Process all episodes in all seasons.

"-s" <integer>
"--season" <integer>
    Process all episodes in the specified season.

"-e" <integer>
"--episode" <integer>
    Process one specific episode in the specified season.
    Requires: '--season'

Example:

• download an episode (by URL):

    # Season 20, Ep 7 - The Very Best Paris-Brest
    atkdl -q -mc 5 -u 'https://www.americastestkitchen.com/episode/653-the-very-best-paris-brest'

• download an episode (by CLI options):

    # Season 20, Ep 7 - The Very Best Paris-Brest
    atkdl -q -mc 5 -s 20 -e 7

• download all episodes in one season:

    # Season 20
    atkdl -q -mc 5 -s 20

• download all episodes in all seasons:

    atkdl -q -mc 5 -a

• print a trace of the operations that would occur IF all episodes in one season were to be downloaded:

    # Season 20
    atkdl -dr -ll 1 -s 20
    atkdl -dr -ll 2 -s 20
    atkdl -dr -ll 3 -s 20

• download all episodes in one season (advanced):

    # Season 20
    atkdl -dr -ll 1 -s 20 >'episode_urls.txt'
    atkdl -dr -ll 2 -s 20 >'convert_mp4s.sh'
  
    atkdl -nm -mc 5 -i 'episode_urls.txt' >'log.txt' 2>&1
  
    ./convert_mp4s.sh

Suggestions:

1. download with options: --no-mp4 --log-level 3
   • redirect stdout to a log file
   • when download completes, check the log file for any error messages
   • if any .ts chunks encountered a download problem
     • identify the url of the Americas Test Kitchen page that was being processed when this error occurred
     • redownload that page (using the same --directory-prefix)
       • all previously downloaded data not be modified or deleted
       • only missing data will be retrieved
2. repeat the above process until the log file shows no download errors
3. finally, convert the HLS stream to mp4
   • the ffmpeg command to perform this conversion is included in the log file
   • when converting the episodes in a series, a list of all ffmpeg commands can be generated with the options: --dry-run --log-level 2

Requirements:

• Node.js version: v8.6.0 (and higher)
  • ES6 support
    • v0.12.18+: Promise
    • v4.08.03+: Object shorthand methods
    • v5.12.00+: spread syntax for iterable objects
    • v6.04.00+: rest parameters
    • v6.04.00+: destructuring, declarations
    • v6.04.00+: Proxy constructor
    • v6.04.00+: Proxy 'apply' handler
    • v6.04.00+: Reflect.apply
    • v7.10.01+: async functions
    • v8.06.00+: object rest/spread properties
  • libraries and dependencies
    • v6.13.00+: Browser-compatible URL class
• FFmpeg
  • not required in PATH when using the --no-mp4 CLI option
    • successfully tested with version: 4.1.3

Credits:

• youtube-dl 'Americas Test Kitchen' extractor
  • provides a methodology to obtain video stream URLs

Legal:

• copyright: Warren Bank
• license: GPL-2.0