Streamer2

Table of Contents

Project Descriptions

Public API

• Put a video
• Get a video
• Get a master playlist
• Get a playlist
• Get health status

Heartbeat Mechanism

• Register
• Report

Environment Variables

Project Descriptions

Streamer2 is a Node.js(v4.4.7) app, developed on OS X El Capitan 10.11.5, that serves video streams to webpage. It recieves variant streams of different resolution types from Recorder and serves adaptive streaming to web app with HLS technique.

The goal of Streamer2 is to replace Streamer1 to enhance some aspects:

• Shorten boot time to 5 seconds or less.
• Lower request-response cycle to 100 milliseconds or less.
• Use JavaScript objects instead of in-memory database to store data.
• Add adaptive streaming control.

Public API

Put a video

PUT /cameras/:cameraId/streams/:streamIdx/resolutions/:resolutionType

{
  "resolution": {
    "width": [integer],
    "height": [integer]
  },
  "rateControl": {
    "bitrate": [integer],
    "frameRateLimit": [float]
  },
  "duration": [float],
  "data": [array]
}

URL parameters:

• cameraId: ID of the camera, should consist of 24 alphanumeric characters
• streamIdx: index of the stream, should be an integer
• resolutionType: 'native' or 'vga' (if width*height is greater than 307200(640*480), please send videos of both resolution types; otherwise, send only 'native' resolution type)

JSON content:

• width: width of the video resolution
• height: height of the video resolution
• bitrate: video bitrate in Kbps that is used to determine bandwidth value in HLS playlists which controls adaptive streaming
• frameRateLimit: video frame rate in fps that is used to determine stream info in HLS master playlist which describes maximum frame rate of the stream, specify 0 if there is no frame rate limit
• duration: video length in seconds that is used to determine video duration in HLS playlists which also determines target duration.(MUST NOT be less than 1/30; also, rounded to the nearest integer, MUST be less than or equal to the target duration)
• data: binary data of a .ts file in byte array of decimals or octets

If there is no corresponding master playlist, it will create one master playlist and one alternate playlist. Otherwise, it will update the corresponding alternate playlist.

Response if the playlist created:

HTTP/1.1 201 Created

or if the playlist updated:

HTTP/1.1 200 OK

or if the request URL contains invalid parameters:

HTTP/1.1 400 Bad Request

Example request:

PUT /cameras/567da100f2f90000011fa0c5/streams/0/resolutions/native

{
  "resolution": {
    "width": 1280,
    "height": 720
  },
  "rateControl": {
    "bitrate": 192,
    "frameRateLimit": 30.000
  },
  "duration": 10.0,
  "data": [116, 102, 115, 116,...]
}

Get a video

GET /cameras/:cameraId/streams/:streamIdx/resolutions/:resolutionType/:fileSequence.ts

URL parameters:

• cameraId: ID of the camera, should consist of 24 alphanumeric characters
• streamIdx: index of the stream, should be an integer
• resolutionType: 'native' or 'vga'
• fileSequence: index of the file, should be an integer

Response if the viedo exists:

HTTP/1.1 200 OK

with a .ts file.

or if the request URL contains invalid parameters:

HTTP/1.1 400 Bad Request

or if the video does not exist:

HTTP/1.1 404 Not Found

Example request:

GET /cameras/567da100f2f90000011fa0c5/streams/0/resolutions/native/0.ts

Get a master playlist

GET /cameras/:cameraId/streams/:streamIdx/all.m3u8

URL parameters:

• cameraId: ID of the camera, should consist of 24 alphanumeric characters
• streamIdx: index of the stream, should be an integer

Response if the playlist exists:

HTTP/1.1 200 OK

with all.m3u8 file, like:

#EXTM3U
#EXT-X-STREAM-INF:BANDWIDTH=974098,CODECS="avc1.4d601f",RESOLUTION=1280x720,FRAME-RATE=15.000
/cameras/567da100f2f90000011fa0c5/streams/0/native.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=160176,CODECS="avc1.4d601f",RESOLUTION=640x480,FRAME-RATE=15.000
/cameras/567da100f2f90000011fa0c5/streams/0/vga.m3u8

(no FRAME-RATE tag if frame rate limit is 0)

or if the request URL contains invalid parameters:

HTTP/1.1 400 Bad Request

or if the video does not exist:

HTTP/1.1 404 Not Found

Example request:

GET /cameras/567da100f2f90000011fa0c5/streams/0/all.m3u8

Get a playlist

GET /cameras/:cameraId/streams/:streamIdx/:resolutionType.m3u8

URL parameters:

• cameraId: ID of the camera, should consist of 24 alphanumeric characters
• streamIdx: index of the stream, should be an integer
• resolutionType: 'native' or 'vga'

Response if the playlist exists:

HTTP/1.1 200 OK

with prog_index.m3u8 file, like:

#EXTM3U
#EXT-X-TARGETDURATION:10
#EXT-X-VERSION:3
#EXT-X-MEDIA-SEQUENCE:0
#EXTINF:10.0,
/cameras/567da100f2f90000011fa0c5/streams/0/resolutions/native/0.ts
#EXTINF:10.0,
/cameras/567da100f2f90000011fa0c5/streams/0/resolutions/native/1.ts
#EXTINF:10.0,
/cameras/567da100f2f90000011fa0c5/streams/0/resolutions/native/2.ts
#EXTINF:10.0,
/cameras/567da100f2f90000011fa0c5/streams/0/resolutions/native/3.ts
#EXTINF:10.0,
/cameras/567da100f2f90000011fa0c5/streams/0/resolutions/native/4.ts

or if the request URL contains invalid parameters:

HTTP/1.1 400 Bad Request

or if the video does not exist:

HTTP/1.1 404 Not Found

Example request:

GET /cameras/567da100f2f90000011fa0c5/streams/0/native.m3u8

Get health status

GET /health

Response:

HTTP/1.1 200 OK

with:

{
  "streamIds": [array],
  "cpuLoad": [float],
  "mem": {
    "load": [float],
    "current": [integer],
    "total": [integer]
  }
}

JSON content:

• streamIds: an array of strings which consist of cameraId and streamIdx
• cpuLoad: CPU usage in percentage
• load: memory usage in percentage
• current: current memory usage amount in MB
• total: total memory size in MB

Heartbeat Mechanism

Register

POST /streamers

{
  "port": [integer],
  "publicIp": [string]
}

JSON content:

• port: port number of the service of Streamer2
• publicIp: public IP address of Streamer2

Streamer2 registers to data API when it starts to run and will receive a streamer ID like 567da0e045800b0001142588 after registration.

Example request:

POST /streamers

{
  "port": 15925,
  "publicIp": "123.123.123.123"
}

Report

PUT /streamers/:streamerId

{
  "streamIds": [array],
  "cpuLoad": [float],
  "mem": {
    "load": [float],
    "current": [integer],
    "total": [integer]
  }
}

URL parameters:

• streamerId: ID of the streamer recieved from Recorder after registration, should consist of 24 alphanumeric characters

JSON content:

• streamIds: an array of strings which consist of cameraId and streamIdx
• cpuLoad: CPU usage in percentage
• load: memory usage in percentage
• current: current memory usage amount in MB
• total: total memory size in MB

Streamer2 reports to data API at intervals determined by CLIO_HEARTBEAT_INTERVAL, or whenever health information changes.

Example request:

PUT /streamers/567da0e045800b0001142588

{
  "streamIds": [
    "567da100f2f90000011fa0c5/0",
    "567da11313dc8d0001a53afc/0",
    "567da12ef0be2400011a2843/1"
  ],
  "cpuLoad": 5.5,
  "mem": {
    "load": 42.7,
    "current": 922,
    "total": 4096
  }
}

Environment Variables

Required:

• CLIO_DATA_API_URL: the URL of data API (for example, 'http://data-api'), MUST be provided for Streamer2 to register to data API so that it can start running.

Optional:

• CLIO_STREAMER_PLAYLIST_LENGTH: number of video segments kept on one playlist
• CLIO_STREAMER_RETAINED_LENGTH: number of video segments not kept on playlist but retained in memory per playlist
• CLIO_HEARTBEAT_INTERVAL: intervals at which Streamer2 reports to data API
• CLIO_STREAMER_PORT: port number of the service of Streamer2
• CLIO_STREAMER_LOG_TRANSPORT : the storage device for logs, can be 'console', 'file', or 'all'
• CLIO_STREAMER_LOG_PATH : the filename of the logfile to write output to
• CLIO_STREAMER_LOG_LEVEL: the maximum level of messages that should be logged, can be 'debug', 'info', or 'error'

Default values as follows:

export CLIO_STREAMER_PLAYLIST_LENGTH=3
export CLIO_STREAMER_RETAINED_LENGTH=6
export CLIO_HEARTBEAT_INTERVAL=10
export CLIO_STREAMER_PORT=15925
export CLIO_STREAMER_LOG_TRANSPORT=all
export CLIO_STREAMER_LOG_PATH=$PWD
export CLIO_STREAMER_LOG_LEVEL=info

Compilation Note:

During tests, load test script attempts to fire up instance on Google Compute Engine according to you local gcloud installation and settings. Default project set by gcloud will be used and load test instance will be launched in the zone according to your environment variable $ZONE value.

Load test relies on ab from apache2-utils package to function.