listdrive

A google drive listing tool

Features:

• Authentication to google api using Oauth2 or service account
• List shared drive
• List file in a google drive folder
• List recursively a google drive folder
• Use advanced filter and query to list google drive folder
• Download files from a google drive folder and export native google drive files to pdf (default)
• Download tool can upload directly the files on a S3 bucket.

Installation

# Install globaly
npm i -g @novadiscovery/listdrive

# Or use npx
npx @novadiscovery/listdrive --help

The download to s3 feature use classic environnement permission for aws (instance profile, local credential, environnement variable, etc.)

The read and download feature on google drive require a configuration file at a standard location (eg. .listdriverc or ~/.config/listdrive/config). See rc file documentation for more details.

Here a sample file (json with comment):

{
  // This file contain required configuration for listdrive
  //
  // See https://www.npmjs.com/package/rc about the possible location for this
  // file and how value will be merged if you have more than one configuration
  // file on your filesystem.


  // Authentication with google drive api can use two different strategy.
  // For both both them, you'll need first to create first a dedicated project
  // on google developer console in order to use google api.
  //
  // For now, only one scope is required:
  // - https://www.googleapis.com/auth/drive.readonly
  //
  // Notice that this project probably already exists: reach your system
  // administrator first before creating another project.
  //
  // The two strategy are:
  //
  // - Oauth: authentication using a user token, in which case, listdrive
  //   will use a user account to read google drive. For this strategy,
  //   you need to fill the `clientId` and `clientSecret` below (or use a
  //   configuration file provided by your system administrator), then
  //   run `listdrive auth` to create a new token
  //
  // - Service Account: a service account is created directly on the developer
  //   console. It will create a virtual user with an email, and you can
  //   share specific ressource with this virtual user. The develope console
  //   allows you to download what you need to put in the service_account field.
  //
  google: {

    // Strategy 1: Oauth
    "clientId": "xxx",
    "clientSecret": "xxx",
    "tokens": {
      // execute `listdrive auth` to create a new token
    },

    // Strategy 2: Service account
    "service_account": {
      // xxx
    }
  }
}

Usage

listdrive [command]

Commands:
  listdrive auth                 Run authentication process
  listdrive list <folderId>      List files in a folder
  listdrive shared-drives        List shared drives
  listdrive download <folderId>  Download has PDF files in a folder

Options:
  --help           Show help                                           [boolean]
  --version        Show version number                                 [boolean]
  --dry                                                                [boolean]
  --recursive, -R                                     [boolean] [default: false]
  --match          Filter with minimatch pattern                        [string]
  --query          Append to file list query (see google drive api
                   https://developers.google.com/drive/api/v3/search-files)
                                                                        [string]
  --force-query    Force the search query (you will have to ensure folder
                   matching and filtering)                              [string]
~/Git/nopnop/listdrive(branch:master*) »                                         

listdrive auth

Run authentication process (to use the Oauth strategy):

listdrive auth

Please, visite this url and update config: https://accounts.google.com/o/oauth2/v2/auth?access_type=offline&scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fdrive.readonly&response_type=code&client_id=xxx
✔ Provide code: … xxxxxxxxxxx
{
  access_token: 'xxxxx',
  refresh_token: 'xxxxx',
  scope: 'https://www.googleapis.com/auth/drive.readonly',
  token_type: 'Bearer',
  expiry_date: 0
}

listdrive list

# List file in a google drive folder identified by folder_id
# (see google drive url for folder_id)
listdrive list <folder_id>

# If the folder is in a shared drive, you need to provide
# the shared drive id too (see the url of the root
# shared drive folder)
# Notice that to list the root folder of a shared drive
# the folder_id is equal to the shared_drive_id
listdrive list <folder_id> --drive <shared_drive_id>

# List recursively (be carefull, can be slow)
listdrive list <folder_id> --recursive

# You can use --indent option to indent each file name
# according to the depth with two space (exemple with jq)
# to produce a natural file tree (you
# may not have access_type to the folder below)
listdrive list <folder_id> --drive <shared_drive_id> --recursive  --indent | jq -r '.name'

# This can be usefull with spreadstream to create
# an indexed spreadsheet..
listdrive list <folder_id> --drive <shared_drive_id>  | spreadstream --json --id <spreadsheet_id> --sheet index --replace

# Filter result with a minimatch wildcard
# Eg. List only file of folder with a name starting with `Welco`
listdrive list <folder_id> --drive <folder_id>  --match 'Welco*'

# Append a subquery to the google api query:
# Exemple: Search for file in the folder containing hello
# see https://developers.google.com/drive/api/v3/search-files
listdrive list <folder_id> --query "fullText contains 'hello'"

# Low level query: Force all the query string
# (bypass  recursive, folder id and drive id selection)
listdrive list <folder_id_but_not_used> --force-query "trashed = true and folder in xxxx and ..."

listdrive download

Download and export files

# Download file in folder id to the data folder
listdrive download <folder_id> --output data

# You can filder / append some query
listdrive download <folder_id> --output data --match "JOB*" --query "mimeType = application/pdf"

# The output target can be a s3 bucket
listdrive download <folder_id> --output s3://nova-sandbox-bucket/test/foo/bar

# By default, if the target is not older than the source,
# the file is ignored unless you force the download:
listdrive download <folder_id> --output data --force

# If you need to controle the target file names,
# you can provide a function exported by a
# node module or a js file (will use `require()`
# internally)
# This function received a google api drive file and must
# return a string.
# This project provide a slugifier renaming with mtime has
# an exemple:
listdrive download <folder_id> --output data --name-it '@novadiscovery/listdrive/src/name-it-mtime-slugify'

listdrive shared-drives

List shared drive:

listdrive shared-drives

{"kind":"drive#drive","id":"xxxx","name":"Open Share"}
{"kind":"drive#drive","id":"yyyy","name":"Another shared drive"}