video-quality-tools
v3.0.3
Published
Set of tools to evaluate video stream quality.
Downloads
31
Readme
Video Quality Tools module - helps to measure live stream characteristics by RTMP/HLS/DASH streams
Features:
- fetching live video stream on demand, displaying info on video/audio characteristics;
- monitoring live video streams in real time, running quality checks: fps and bitrate drops, GOP structure changes and more.
video-quality-tools
requires ffmpeg
and ffprobe
.
Installation
Using npm:
$ npm install --save video-quality-tools
Using yarn:
$ yarn add video-quality-tools
Basic Concepts
Error Handling
There are a lot of methods in module that may throw an error. All errors are subclasses of a basic javascript Error
object and may be distinguished by prototype. To do this, just import the Error
object from the module.
const {Errors} = require('video-quality-tools');
Errors
object contains a set of error classes that may be thrown.
For example: StreamInfo
constructor throws Errors.ConfigError
for incorrect options.
const {StreamsInfo, Errors} = require('video-quality-tools');
try {
const streamsInfo = new StreamsInfo(null, 'rtmp://host:port/appInstance/name');
} catch (err) {
if (err instanceof Errors.ConfigError) {
console.error('Invalid options:', err);
}
}
Exit Reasons
StreamsInfo
and FramesMonitor
use ffprobe
to fetch the data. The underlying ffprobe process may be killed
by someone, it may fail due to spawn
issues, it may exit normally or with error code or it may be killed by the module
itself if frames have invalid format.
To distinguish exit reasons you may import ExitReasons
object.
const {ExitReasons} = require('video-quality-tools');
There are such available classes:
ExitReasons.StartError
ExitReasons.ExternalSignal
ExitReasons.NormalExit
ExitReasons.AbnormalExit
ExitReasons.ProcessingError
Description of a specific reason class may be found in further chapters.
One-time Live Stream Info
To fetch one-time info you need to create StreamsInfo
class instance
const {StreamsInfo} = require('video-quality-tools');
const streamsInfoOptions = {
ffprobePath: '/usr/local/bin/ffprobe',
timeoutInMs: 2000
};
const streamsInfo = new StreamsInfo(streamsInfoOptions, 'rtmp://host:port/appInstance/name');
Constructor throws:
Errors.ConfigError
if options have invalid type or value;Errors.ExecutablePathError
ifoptions.ffprobePath
is not found or it's not an executable.
After that you may run fetch
method to retrieve video and audio info. Method can be called as many times as you want.
// async-await style
const streamInfo = await streamsInfo.fetch();
// or using old-school promise style
streamsInfo.fetch()
.then(info => {
console.log('Video info:');
console.log(info.videos);
console.log('Audio info:');
console.log(info.audios);
})
.catch(err => console.error(err));
Method may throw the Errors.StreamsInfoError
if it can't receive stream, stream is invalid, ffprobe
exits
with error or returns unexpected output.
The videos
and audios
fields of the returned info
object are arrays. Usually there is only one
video or audio stream for RTMP streams. Each element of the videos
or audios
array has almost the same
structure as the ffprobe -show_streams
output has. You may find a typical output of fetch
command below.
videos
and audios
may be an empty array if there are no appropriate streams in the live stream.
{ videos:
[ { index: 1,
codec_name: 'h264',
codec_long_name: 'H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10',
profile: 'Main',
codec_type: 'video',
codec_time_base: '33/2000',
codec_tag_string: '[0][0][0][0]',
codec_tag: '0x0000',
width: 854,
height: 480,
coded_width: 854,
coded_height: 480,
has_b_frames: 0,
sample_aspect_ratio: '1280:1281',
display_aspect_ratio: '16:9',
pix_fmt: 'yuv420p',
level: 31,
chroma_location: 'left',
field_order: 'progressive',
refs: 1,
is_avc: 'true',
nal_length_size: '4',
r_frame_rate: '30/1',
avg_frame_rate: '1000/33',
time_base: '1/1000',
start_pts: 2062046,
start_time: '2062.046000',
bits_per_raw_sample: '8',
disposition: [Object] } ],
audios:
[ { index: 0,
codec_name: 'aac',
codec_long_name: 'AAC (Advanced Audio Coding)',
profile: 'LC',
codec_type: 'audio',
codec_time_base: '1/44100',
codec_tag_string: '[0][0][0][0]',
codec_tag: '0x0000',
sample_fmt: 'fltp',
sample_rate: '44100',
channels: 2,
channel_layout: 'stereo',
bits_per_sample: 0,
r_frame_rate: '0/0',
avg_frame_rate: '0/0',
time_base: '1/1000',
start_pts: 2061964,
start_time: '2061.964000',
disposition: [Object] } ] }
Live Frames Monitor
To measure live stream info you need to create instance of FramesMonitor
class
const {FramesMonitor} = require('video-quality-tools');
const framesMonitorOptions = {
ffprobePath: '/usr/local/bin/ffprobe',
timeoutInMs: 2000,
bufferMaxLengthInBytes: 100000,
errorLevel: 'error',
exitProcessGuardTimeoutInMs: 1000,
analyzeDurationInMs: 9000
};
const framesMonitor = new FramesMonitor(framesMonitorOptions, 'rtmp://host:port/appInstance/name');
Constructor throws:
Errors.ConfigError
orTypeError
if options have invalid type or value;Errors.ExecutablePathError
ifoptions.ffprobePath
is not found or it's not an executable.
Frames Monitor Config
The first argument of FramesMonitor
must be an options
object. All options
object's fields are mandatory:
ffprobePath
- string, path to ffprobe executable;timeoutInMs
- integer, greater than 0, specifies the maximum time to wait for (network) read/write operations to complete;bufferMaxLengthInBytes
- integer, greater than 0, specifies the buffer length for ffprobe frames. This setting prevents from hanging and receiving incorrect data from the stream, usually 1-2 KB is enough;errorLevel
- specifies log level for debugging purposes, must be equal to ffprobe's-loglevel
option. May be one of the following values:trace
,debug
,verbose
,info
,warning
,error
,fatal
,panic
,quiet
. For most caseserror
level is enough;exitProcessGuardTimeoutInMs
- integer, greater than 0, specifies the amount of time after which the monitoring process will be hard killed if the attempt of soft stop fails. When you try to stop a monitor withstopListen()
method theFramesMonitor
sendsSIGTERM
signal to ffprobe process. ffprobe may ignore this signal (some versions do it pretty often). If ffprobe doesn't exit afterexitProcessGuardTimeoutInMs
milliseconds,FramesMonitor
sendsSIGKILL
signal and forces underlying ffprobe process to exit.- analyzeDurationInMs - integer, greater than 0, specifies the maximum analyzing time of the input.
Listening of Frames
After creation of the FramesMonitor
instance, you may start listening live stream data. To do so, just
run framesMonitor.listen()
method. After that framesMonitor
starts emitting frame
event as soon as ffprobe
decodes frame from the stream. It emits video and audio frames.
const {FramesMonitor, processFrames, ExitReasons} = require('video-quality-tools');
const framesMonitor = new FramesMonitor(options, 'rtmp://host:port/appInstance/name');
framesMonitor.on('frame', frameInfo => {
console.log(frameInfo);
});
framesMonitor.listen();
listen()
method doesn't return anything but may throw Errors.AlreadyListeningError
.
Stop Listening of Frames
To stop listening, call framesMonitor.stopListen()
method. It returns promise. Rejection of that promise means
that the underlying ffprobe process can't be killed with a signal. Method tries to send SIGTERM
signal first
and wait for exitProcessGuardTimeoutInMs
milliseconds (ref. Frames Monitor Config section). If the
process doesn't exit after that timeout, method sends SIGKILL
and forces it to exit. Resolved promise means that
the process was successfully killed.
try {
const {code, signal} = await framesMonitor.stopListen();
console.log(`Monitor was stopped successfully, code=${code}, signal=${signal}`);
} catch (err) {
// instance of Errors.ProcessExitError
console.log(`Error listening url "${err.payload.url}": ${err.payload.error.message}`);
}
frame
event
This event is generated on each video and audio frame decoded by ffprobe. The structure of the frame object is the following:
{ media_type: 'video',
key_frame: 0,
pkt_pts_time: 3530.279,
pkt_size: 3332,
width: 640,
height: 480,
pict_type: 'P' }
or
{ media_type: 'audio',
key_frame: 1,
pkt_pts_time: 'N/A',
pkt_size: 20 }
exit
event
Underlying process may not start at all, it may fail after some time or it may be killed with signal. In such situations
FramesMonitor
class instance emits exit
event and passes one of the ExitReasons
instances. Each instance has its
own reason-specific payload
field. There is a list of reasons:
ExitReasons.StartError
- ffprobe can't be spawned, the error object is stored inpayload.error
field;ExitReasons.NormalExit
- ffprobe has exited with code = 0,payload.code
is provided;ExitReasons.AbnormalExit
- ffprobe has exited with non-zero exit code,payload.code
contains the exit code of the ffprobe process andpayload.stderrOutput
contains the last 5 lines from ffprobe's stderr output;ExitReasons.ProcessingError
- monitor has detected a logical issue and forces ffprobe to exit, this exit reason contains error object inpayload.error
field that may be eitherErrors.ProcessStreamError
orErrors.InvalidFrameError
;ExitReasons.ExternalSignal
- ffprobe process was killed by someone or by another process with the signal, the signal name can be found inpayload.signal
field;
framesMonitor.on('exit', reason => {
switch(reason.constructor) {
case ExitReasons.AbnormalExit:
assert(reason.payload.code);
assert(reason.payload.stderrOutput); // stderrOutput may be empty
break;
case ExitReasons.NormalExit:
assert(reason.payload.code);
break;
case ExitReasons.ExternalSignal:
assert(reason.payload.signal);
break;
case ExitReasons.StartError:
assert.instanceOf(reason.payload.error, Error);
break;
case ExitReasons.ProcessingError:
assert.instanceOf(reason.payload.error, Error);
break;
}
});
error
event
May be emitted only once and only in case the framesMonitor.stopListen()
method receives error
event on
killing of an underlying ffprobe process.
framesMonitor.on('error', err => {
// indicates error during the kill process
// when ProcessingError occurs we may encounter that can not kill process
// in this case this error event would be emitted
assert.instanceOf(err, Error);
});
Video Quality Info
video-quality-tools
ships with functions that help determining live stream info based on the set of frames
collected from FramesMonitor
:
processFrames.networkStats
processFrames.encoderStats
processFrames.networkStats(frames, durationInMsec)
Receives an array of frames
collected for a given time interval durationInMsec
.
This method doesn't analyze GOP structure and isn't dependant on fullness of GOP between runs. Method shows only
frame rate of audio and video streams received, bitrate of audio and video. Instead of processFrames.networkStats
this method allows to control quality of network link between sender and receiver (like RTMP server).
Remember that this module must be located not far away from receiver server (that is under analysis). If link between receiver and module affects delivery of RTMP packages this module indicates incorrect values. It's better to run this module near the receiver.
const {processFrames} = require('video-quality-tools');
const INTERVAL_TO_ANALYZE_FRAMES = 5000; // in milliseconds
let frames = [];
framesMonitor.on('frame', frame => {
frames.push(frame);
});
setInterval(() => {
try {
const info = processFrames.networkStats(frames, INTERVAL_TO_ANALYZE_FRAMES);
console.log(info);
frames = [];
} catch(err) {
// only if arguments are invalid
console.log(err);
process.exit(1);
}
}, INTERVAL_TO_ANALYZE_FRAMES);
There is an output for the example above:
{
videoFrameRate: 29,
audioFrameRate: 50,
videoBitrate: 1403.5421875,
audioBitrate: 39.846875
}
Check examples/networkStats.js to see an example code.
processFrames.encoderStats(frames)
It relies on GOP structure of the stream.
The following example shows how to gather frames and pass them to the function that analyzes encoder statistic.
const {processFrames} = require('video-quality-tools');
const AMOUNT_OF_FRAMES_TO_GATHER = 300;
let frames = [];
framesMonitor.on('frame', frame => {
frames.push(frame);
if (AMOUNT_OF_FRAMES_TO_GATHER > frames.length) {
return;
}
try {
const info = processFrames.encoderStats(frames);
frames = info.remainedFrames;
console.log(info.payload);
} catch(err) {
// processing error
console.log(err);
process.exit(1);
}
});
There is an output for the example above:
{ areAllGopsIdentical: true,
bitrate:
{ mean: 1494.9075520833333,
min: 1440.27734375,
max: 1525.95703125 },
fps: {
mean: 30,
min: 30,
max: 30 },
gopDuration: {
mean: 2,
min: 1.9,
max: 2.1 },
displayAspectRatio: '16:9',
width: 1280,
height: 720,
hasAudioStream: true
}
In given example the frames are collected in frames
array and than use processFrames.encoderStats
function for
sets of 300 frames (AMOUNT_OF_FRAMES_TO_GATHER
). The function searches the
key frames
and measures the distance between them.
It's impossible to detect GOP structure for a set of frames with only one key frame, so processFrames.encoderStats
returns back all passed frames as an array in remainedFrames
field.
If there are more than 2 key frames, processFrames.encoderStats
uses full GOPs to track fps and bitrate and returns
all frames back in the last GOP that was not finished. It's important to remember the remainedFrames
output
and push a new frame to the remainedFrames
array when it arrives.
For the full GOPs processFrames.encoderStats
calculates min/max/mean values of bitrates (in kbit/s), framerates
and GOP duration (in seconds) and returns them in payload
field. The result of the check for the similarity
of GOP structures for the collected GOPs is returned in areAllGopsIdentical
field. Fields width
, height
and displayAspectRatio
are taken from data from first frame of the first collected GOP. Value of hasAudioStream
reflects presence of audio frames.
To calculate display aspect ratio method processFrames::calculateDisplayAspectRatio
uses list of
video aspect ratio standards
with approximation of frames width and height ratio. If ratio can't be found in list of known standards, even in delta
neighbourhood, then
GCD algorithm is used to simplify returned value.
processFrames.encoderStats
may throw Errors.GopNotFoundError
.
Also, you may extend the metrics. Check src/processFrames.js
to find common functions.