ray-streaming

RayStreaming SDK for Web

Guides

Quick start

see live-cat or live-dragon or live-qjc

Example

File transfer

// download
// reply download apply
connection.event.fileTransferApply.on((request) => {
  connection.replyFileTransferApply(request.fileUnique, allowed) // allowed: boolean
})
// receive file, file.data is Blob
connection.event.receivedFile.on((file) => console.log(file.data, file.name))

// upload
connection.sendFile(file, (e) => {
  if (e.state === 'queue') {
    // wait for remote reply
    console.log('queue')
  } else if (e.state === 'progress') {
    // sending
    console.log(e.progress)
  } else if (e.state === 'error') {
    // remote reject or error
    console.error(e.reason)
  } else if (e.state === 'done') {
    // finish
    console.log('done')
  }
})

API reference

Connection

Manage the Signaling and PeerConnection connection with node machine

new Connection(url, iceServers[, iceTransportPolicy][, minBitrate][, maxBitrate][, startBitrate][, enableAFK]))

• url<string> signaling endpoint
• iceServers <RTCIceServer[]> RTCIceServer
• iceTransportPolicy <RTCIceTransportPolicy> iceTransportPolicy Default: 'all'
• minBitrate <number> append x-google-min-bitrate in create offer Default: 2000
• maxBitrate <number> x-google-max-bitrate Default: 5000
• startBitrate <number> x-google-start-bitrate Default: 4000
• enableAFK <boolean> disconnect when detecting user away from keyboard if true Default: true

connection.emitUIInteraction(msg)

• msg <string>

Send UI interaction message to remote application

connection.emitUIInteraction(JSON.stringify({ eventName: 'A' })).then((result) => {
  if (result) {
    console.log('execute success') // NOTE: only promise send to application
  }
})

connection.screenshot([params])

• params <Object>
  • index <number> screen index Default: 0
  • left <number> x coordinate origin Default: 0
  • top <number> y coordinate origin Default: 0
  • width <number> width of screenshot Default: 1920
  • height <number> height of screenshot Default: 1080

screenshot origin stream

connection.clipboard(text)

• text <string> string from clipboard

sync clipboard text to node

connection.changeBandwidth(bandwidth)

• bandwidth <number> kbps

change bandwidth with fixed bitrate

connection.changeBandwidth(startBitrate, maxBitrate, minBitrate)

• startBitrate <number> kbps
• maxBitrate <number> kbps
• minBitrate <number> kbps

change bandwidth with bitrate range

connection.destroy()

Close all connect, off event listener

connection.sendFile(file, [onStateChange])

• file <File>
• onStateChange <StateChangeHandler>

upload file to remote node

Player

Provide the container to play the remote stream,

new Player(hostElement[, onPlay][, voicedAtPlay][, orientationLock])

• hostElement <HTMLElement> any block element with fixed size
• onPlay <() => void> invoke when video play
• voicedAtPlay <boolean> remove muted attribute when play if set true Default: true
• orientationLock <boolean> video will rotate to screen orientation with the same aspect ratio if set false Default: true
• onRotate <(r: boolean) => void> call when hostElement resize or video track resolution change

player.resizePlayer()

Resize player style to fill host element if host element size is smaller than player or resize player style to actual size

player.showTextOverlay(text)

• text <string>

Show prompt with specify text and mask video stream

player.destroy()

Remove created elements

RunningState

State of the remote node or front-end widget, manager by launcher or execution context

// NOTE: Usually do not need to be modified
const runningInfo = new RunningState()

Helper

Helper.createDefaultEventStream(target, runningInfo)

• target <HTMLVideoElement> the video element
• runningInfo <RunningState> RunningState instance

Create default stream for basic interaction, transform to ArrayBuffer from the event-adapter definition

Helper.createGamepadStream()

Create stream for gamepad, transform to ArrayBuffer from the event-adapter definition

Helper.createKeyboardStream(target[, filterKeys])

• target <HTMLVideoElement> the video element
• filterKeys <string[]> block specify key Default: []

Create stream for keyboard, transform to ArrayBuffer from the event-adapter definition

Helper.createClipboardStream(target)

• target <HTMLVideoElement> the video element

listen target's focus event and transform to clipboard text

Helper.attachListener(target, runningInfo, connection[, options])

• target <HTMLVideoElement>
• runningInfo <RunningState>
• connection <Connection>
• options <Object>
  • enableGamepad <boolean> create gamepad event by default Default: false
  • enableClipboard <boolean> create clipboard event by default Default: false
  • filterKeys <string[]> block specify key Default: []

the default behavior for consuming the event stream

Rivatuner

use getStats to gather statistics from provided Connection

new Rivatuner()

rivatuner.offer(connection)

• connection <Connection> Connection instance

Provide Connection

rivatuner.scan()

Start timer

rivatuner.stop()

Stop timer

rivatuner.toggle()

Toggle scan or stop

rivatuner.report()

• Returns: <Object>
  • fps: <number>
  • latency: <number> ms
  • bitrate: <number> kbs
  • packetLossRate: <number>
  • averageJitterBufferDelay: <number> ms
  • framesReceived <number>
  • framesDecoded <number>
  • keyFramesDecoded <number>
  • framesDropped <number>
  • framesPerSecond <number>

Report current statistics

rivatuner.destroy()

Release listener

Dashboard

use Dashboard to count network-related information and custom information which has to use Rivatuner if using default dashboard.

new Dashboard(hostElement, fieldsGroups, dashboardState)

• hostElement <HTMLElement>
• fieldsGroups Array of objects that each object is a graph
• dashboardState offer a default config to Dashboard

Dashboard.createDefaultDashboard(hostElement)

new DashboardConfig(options?: Partial<Options>)

• options Setting default configs to Dashboard.

interface Options {
  xCount: number
  yCount: number
  dataCount: number
  paddingLeft: number
  paddingRight: number
  paddingTop: number
  paddingBottom: number
  xAxisGap: number
  legendRadius: number
  themeColor: string
}

import { Dashboard, DashboardConfig, Rivatuner } from 'ray-streaming'

const rivatuner = new Rivatuner()

// remember to offer connection to rivatuner using rivatuner.offer()

// create dashboard with default fields
const dashboard = Dashboard.createDefaultDashboard(hostElement)
setInterval(() => {
  // 'fps' is named fps in rivatuner report while is named framesPerSecond in dashboard.
  const { fps, ...rest } = rivatuner.report()
  dashboard.insert({
    ...rest,
    framesPerSecond: fps,
  })
}, 500)

let p = window.performance
// create dashboard using default fields and custom fields
const dashboard = Dashboard.createDefaultDashboard(hostElement, [
  {
    totalJSHeapSize: '',
    usedJSHeapSize: '',
    jsHeapSizeLimit: '',
  },
  {
    delay: 'ms',
  },
])
setInterval(() => {
  const { fps, ...rest } = rivatuner.report()
  dashboard.insert({
    ...rest,
    framesPerSecond: fps,
    totalJSHeapSize: p.memory?.totalJSHeapSize ?? 0,
    usedJSHeapSize: p.memory?.usedJSHeapSize ?? 0,
    jsHeapSizeLimit: p.memory?.jsHeapSizeLimit ?? 0,
    delay: calculatedDelay ?? 0,
  })
}, 500)

// create dashboard with custom fields
const dashboard = new Dashboard(
  hostElement,
  [
    {
      totalJSHeapSize: '',
      usedJSHeapSize: '',
      jsHeapSizeLimit: '',
      delay: 'ms',
    },
  ],
  new DashboardConfig(),
)
setInterval(() => {
  dashboard.insert({
    totalJSHeapSize: p.memory?.totalJSHeapSize ?? 0,
    usedJSHeapSize: p.memory?.usedJSHeapSize ?? 0,
    jsHeapSizeLimit: p.memory?.jsHeapSizeLimit ?? 0,
    delay: calculatedDelay ?? 0,
  })
}, 500)

Caveat

• Need to initialize Player after WeixinJSBridgeReady event fired if use in WeChat web-view

if (navigator.userAgent.includes('miniProgram') || navigator.userAgent.includes('MicroMessenger')) {
  document.addEventListener('WeixinJSBridgeReady', () => {
    // initialize here
  })
}