filepunk

v0.3.0

Published

2 years ago

[![Build Status](https://travis-ci.org/blockai/filepunk.svg?branch=master)](https://travis-ci.org/blockai/filepunk)

Downloads

0High
0Medium
0Low

olalonde

filepunk

File upload and transformation microservice.

Related projects:

filepunk-client Low level JavaScript client
filepunk-widget JavaScript file upload UI widget (built with React + Redux)

Table of Contents

filepunk

Install

npm install -g filepunk

Requires Node v6+

Usage

See ./test directory for usage examples.

filepunk
open http://localhost:3000/v1/upload

API

Authentication

TODO: verify signature and expiry on policies...

For now, setting SECURE_MODE=true disables DELETE and disallows setting arbitrary keys, effectively making making filepunk immutable (write only).

Files

POST /v1/upload Upload a file

Upload a file (multipart/form-data):

fileUpload: mixed file to upload

Upload a URL (application/json or application/x-www-form-urlencoded):

url: mixed URL of file to upload

Response will contain an additional source key with the URL.

Query parameters:

key: string (optional) Path where to store file. Defaults to a unique ID at the root followed by an underscore, followed by the filename (e.g. 3AB239102DB_myphoto.png).
filename: string (optional) Name of the file. If not set, we do our best to come up with one.
contentType: string (optional) Mime type of the file. If not set, we do our best to guess it, defaulting to text/plain.

curl -X POST \
  -F fileUpload=@test/files/lasvegas.gif \
  http://localhost:3000/v1/upload?key=olalonde%2Flasvegas.gif&contentType=image%2Fgif

curl -X POST \
  -d url=https://api.blockai.com/v1/registrations/e9zj8a/preview \
  http://localhost:3000/v1/upload?filename=presidio.jpg

{
  "url": "http://localhost/v1/files/lxHieAHdas",
  "id": "lxHieAHdas",
  "key": "avatars/username.png",
  "filename": "usename.png",
  "type": "image/png",
  "size": 1337
}

POST /v1/tus Implements the TUS protocol for resumable file upload.

Same API as POST /v1/store except the query parameters must be specified as Tus-Metadata headers and the JSON response can be fetched by issuing a GET request on the upload URL once the upload is fully completed.

GET /v1/files/{fileId} Get the content of a file

GET /v1/files/{fileId}?dl=true Download a file

GET /v1/files/{fileId}?inline=true Forces browser to display file in browser

GET /v1/files/{fileId}?cache=false Tells browser to not cache file

GET /v1/files/{fileId}+name.jpg Can be used with reverse proxies that cache based on file extension

GET /v1/files/{fileId}/metadata Retrieve file metadata (size, type, etc.)

DELETE /v1/files/{fileId} Permanently delete and remove all traces of a file

curl -X DELETE http://localhost/v1/files/lxHieAHdas

Transformations

General URL structure: /v1/transform/{conversion tasks}/{fileId or external URL}

Chaining tasks:

/v1/transform/[task]=[options]/[task]=[options]/fileId_OR_url

Transforms are cached in the store.

Examples:

# resize then blur
/v1/transform/resize=width:200,height:300,options:!/blur=radius:20/Q7pJwejb1B85

# thumbnail
/v1/transform/thumb/zndL1PQNx07E
/v1/transform/thumb=height:200/zndL1PQNx07E

# screenshot url on mobile, thumb and sepia
/v1/transform/urlscreenshot=agent:mobile/thumb/sepia/https://en.wikipedia.org/wiki/Main_Page

# convert gif/psd/etc. to png
/v1/transform/format=format:png/0VpW1J03R6nB

# convert gif/psd/png/jpg/etc. to pdf
/v1/transform/format=format:pdf/0VpW1J03R6nB

# save web page to pdf
/v1/transform/urlscreenshot/format=format:pdf/http://wikipedia.org

# make logo gray and 50px height for press page
/v1/transform/thumb=height:50/gray/https://tctechcrunch2011.files.wordpress.com/2011/11/techcrunch_transparent.png

# get EXIF data from image
/v1/transform/identify/http://c1.staticflickr.com/3/2815/12382975864_2cd7755b03_n.jpg

# make photo montage
/v1/transform/collage=files:["https://www.filestackapi.com/api/file/6a9QVg1LS4uoPN7B4HYA"]/https://www.filestackapi.com/api/file/6a9QVg1LS4uoPN7B4HYA

# screenshot of wikipedia with height exactly 500
/v1/transform/urlscreenshot/resize=height:500,options:!/https://en.wikipedia.org/wiki/Main_Page/

# photo montage
/v1/transform/collage=files:[%22https://www.filestackapi.com/api/file/6a9QVg1LS4uoPN7B4HYA%22,%22https://www.filestackapi.com/api/file/WmOJgubRRiRtLR9UGGJc%22]/https://www.filestackapi.com/api/file/0ZgN5BtJTfmI1O3Rxhce

# photo montage
/v1/transform/collage=files:["https://github.com/recurser/exif-orientation-examples/blob/master/Landscape_5.jpg?raw=true","https://raw.githubusercontent.com/recurser/exif-orientation-examples/master/Landscape_6.jpg?raw=true"],width:1200,height:400,autorotate:false/https://github.com/recurser/exif-orientation-examples/blob/master/Landscape_4.jpg?raw=true

List & upload files from external stores (e.g. Instagram)

General URL structure:

GET /v1/external/{provider}/{token}/{path}?format={format}

Where:

provider: name of external provider (e.g. instagram)
token: url safe base64 encoded json representing some auth token or a username / user id. e.g.: token = urlSafeBase64.encode(JSON.stringify({ accessToken: 'deadbeef' })) An alternative syntax for the token is {key}={value}, e.g. username=olalonde which internally is converted to { username: 'olalonde' }
path: path inside external store
format: one of info, store
- info returns misc info about external file
- store uploads the file to filepunk and returns the file info. (in addition query, can contain any params used for /v1/upload

The responses of format=info include some of the following fields:

// /v1/external/flickr/23ab22cc/?format=info
{
  "next": null,
  "provider": "flickr",
  "view": "list",
  "contents": [
    {
      "mimetype": "image/jpeg",
      "metadata": {
        "width": "2139",
        "height": "1604"
      },
      "link_path": "/flickr/23ab22cc/29001856331",
      "display_name": "",
      "modified": "1 month ago",
      "is_dir": false,
      "thumb_exists": true,
      "thumbnail": "https://farm9.staticflickr.com/8026/29001856331_3770866bc4_t.jpg",
      "filename": "29001856331.jpg"
    }
  ],
  "filename": "Flickr"
}

// /v1/external/flickr/23ab22cc/29001856331?format=info
{
  "mimetype": "image/jpeg",
  "metadata": {
    "width": "2139",
    "height": "1604"
  },
  "link_path": "/flickr/23ab22cc/29001856331",
  "display_name": "",
  "modified": "1 month ago",
  "is_dir": false,
  "thumb_exists": true,
  "thumbnail": "https://farm9.staticflickr.com/8026/29001856331_3770866bc4_t.jpg",
  "filename": "29001856331.jpg"
}

Examples:

# store instagram post kIRWzIBC7C by blockai
/v1/external/instagram/eyJ1c2VybmFtZSI6ImJsb2NrYWkifQ/kIRWzIBC7C?format=store

# store instagram post BKyanP7gPrY by blockai (with tags), using
# alternative token syntax
/v1/external/instagram/username=blockai/BKyanP7gPrY?format=store

# instagram post BDin77DxtAH by oreo (video)
/v1/external/instagram/eyJ1c2VybmFtZSI6Im9yZW8ifQ/BDin77DxtAH

Transformation tasks

Misc tasks

urlscreenshot

Takes screenshot of URL

store

Stores the result of the transform to filepunk

Image tasks

thumb {}: Creates thumbnail from image
sepia: applies sepia image filter
resize { width, height, options, filter }
autoorient: auto orient
blur { radius = 20, sigma }: applies blur filter
border { width = 10, height = 10, color = '#000000' }: adds a border
charcoal { factor = 1 }: applies charcoal filter
chop { width, height, x, y }: chops a piece of the image
colors { num }: color intensity
contrast { multiplier }: change contrast
crop { width, height, x, y }: crop the image
cycle { amount }: cycle the image
collage { files, width, height, margin }: create collage from multiple files
despeckle: despeckle the image
edge { radius }: applies edge filter
emboss { radius }: emboss filter
enhance: enhance the image
flatten: flatten the image
flip: creates a mirror image (vertical direction)
flop: creates a mirror image (horizontal direction)
frame { width, height, outerwidth, innerwidth }: adds a frame to image
fuzz { distance, percent }: fuzz filter
gamma { r, g, b }: change gamma
gray: desaturate an image
gaussian { radius, sigma }: gaussian filter
implode { factor }: implode filter
magnify { factor }: magnify
matte: matte filter
median { radius }
minify { factor }
monochrome: black and white filter
motionblur { radius, sigma, angle }: motion blur filter
negative: negative filter
noise { radius, type = 'uniform' }: adds noise
normalize
opaque { color }
noprofile: remove exif data, etc.
paint { radius }
quality { val }
raise { width, height }
resample { horizontal, vertical }
roll { horizontal, vertical }
rotate { color = '#000000', degrees }
sepia
format { format }
shade { azimuth, elevation }
shadow { radius, sigma }
shave { width, height, percent }
sharpen { radius, sigma }
shear { xdegrees, ydegrees }
solarize { threshold }
spread { amount }
stegano { offset }
strip
swirl { degrees }
transparent { color }
trim
unsharp { radius, sigma, amount, threshold }
wave { amplitude, wavelength }

Published

Vulnerabilities

Links

Maintainers

Keywords

Readme

filepunk

Install

Usage

API

Authentication

Files

Transformations

List & upload files from external stores (e.g. Instagram)

Transformation tasks

Misc tasks

urlscreenshot

store

Image tasks