pull-request-data

A handy dandy little CLI tool for fetching a bunch of JSON files from the requests-production bucket.

This is useful for testing the Snapchat API client with actual real world data.

Installation

npm install --global @mishguru/pull-request-data

Usage

This will scan the requests-production S3 bucket for any Snapchat users.

It will download the first file in each folder for the following types:

• All Updates: all_updates-*
• Conversations: conversations_stories-*
• Stories: get_stories-*

JSON files are saved in a directory called ./request-data by default.

mish run --env internal -- pull-request-data

Options

There are few options you can set.

--bucket, -b

Set which bucket to fetch files from.

Default: requests-production

This tool has only been tested with two buckets:

• requests-production
• requests-internal

pull-request-data --bucket requests-internal

--output, -o

Set which directory to save files to.

If the directory does not exist, it will be created.

Default value: ./request-data

pull-request-data --output ./payloads

--usernames

Select which usernames to fetch files for.

By default, a list of usernames will be assembled by scanning the S3 bucket all folders in the eroot starting with a letter.

pull-request-data --usernames stayradiated todaysmish

--take

Select how many files of each category to download for each user.

Default value: 1

pull-request-data --take 5

--skip

Used in conjunction wth --take to get a sample of the available files.

Default value: 1000

pull-request-data --take 2 --skip 4

The name skip is a bit of a misnomer - it actually works as a modulus of the index, so it skips n-1 files.

- 0.json [pulled, 0 % 4 = 0]
- 1.json
- 2.json
- 3.json
- 4.json [pulled, 4 % 4 = 0, reached take limit]
- 5.json
- 6.json
- 7.json
- 8.json