Metadata Synchronizer - Megh Computing

Megh VAS products output video and metadata streams separately. This package can be used to correlate the out-of-band metadata with the MP4 video using timestamps present in both data streams:

• For the MP4 video, the media presentation timestamp (PTS) of each frame should be used.
• For the metadata, the timestamp is included in the ZeroMQ multi-part message.

Features

• Hybrid package: CommonJS and ESM supported
• Cross-platform: Node.js (16+) and modern browsers (ES2020+)
• First-class TypeScript support

Installation

Install the package as a dependency using npm:

$ npm install @megh-computing/metadata-synchronizer

or, install using yarn:

$ yarn add @megh-computing/metadata-synchronizer

Example Usage

import type {ImageFrame} from "@megh-computing/image-frame-pool";
import {MetadataSynchronizer} from "@megh-computing/metadata-synchronizer";

...

// Handle up to 10 seconds of de-sync between video and metadata (at 30fps)
// const synchronizer = new MetadataSynchronizer(30 /*fps*/ * 10 /*seconds*/);
const synchronizer = new MetadataSynchronizer(30 /*fps*/ * 10 /*seconds*/, {
    // Optional frame memory management context for pooling and re-using frames
    clone(frame: ImageFrame): ImageFrame {
        return frame.clone();
    },
    release(frame: ImageFrame) {
        frame.release();
    },
});
// Configures the tolerance between timestamps before they are considered equal
synchronizer.setEqualityTolerance(0.001); // 1 millisecond
synchronizer.on("match", (frame: ImageFrame, metadata) => {
    // Render metadata over frame and display
});
synchronizer.on("frameDropped", (frame: ImageFrame) => {
    // Display frame with previous frame's metadata or no metadata at all
});
synchronizer.on("metadataDropped", (metadata) => {
    // Do something with unused metadata?
});
synchronizer.on("warning", (type: string, message: string) => {
    // By default, warnings will be printed to the console unless a handler is added
    if (type === "trim") {
        // Trim warnings indicate that capacity of queues has been reached,
        // and the matching performance may degrade due to missing frames or metadata
        console.warn("trim error:", message);
    } else {
        // Ignore other warnings
    }
});

...

// Simply append the frames and metadata to the synchronizer,
// and the synchronizer will emit the events configured above
synchronizer.appendFrame(frame, frameTimestamp);
synchronizer.appendMetadata(metadata, metadataTimestamp);

// The synchronizer requires that the timestamps are in ascending order,
// so if/when either stream restarts, then the synchronizer must be reset
synchronizer.resetFrame();
synchronizer.resetMetadata();

// The oldest metadata timestamp can be queried to help determine if the metadata stream has restarted,
// and therefore the metadata side of the synchronizer must be reset
const oldestMetadataTimestamp = synchronizer.getOldestMetadataTimestamp();
if (oldestMetadataTimestamp != null) {
    // Check if timestamp is less than oldest timestamp
    if (metadataTimestamp < oldestMetadataTimestamp) {
        // If so, then assume metadata has reset
        synchronizer.resetMetadata();
    }
}
synchronizer.appendMetadata(metadata, metadataTimestamp);

License

Copyright (c) 2021 Megh Computing, Inc.

All rights reserved. No warranty, explicit or implied, provided. Unauthorized use, modification, or distribution is strictly prohibited. Homepage: https://megh.com/

Please contact us if you use Megh VAS and would like to be issued a license to use this package.