npm package discovery and stats viewer.

Discover Tips

  • General search

    [free text search, go nuts!]

  • Package details

    pkg:[package-name]

  • User packages

    @[username]

Sponsor

Optimize Toolset

I’ve always been into building performant and accessible sites, but lately I’ve been taking it extremely seriously. So much so that I’ve been building a tool to help me optimize and monitor the sites that I build to make sure that I’m making an attempt to offer the best experience to those who visit them. If you’re into performant, accessible and SEO friendly sites, you might like it too! You can check it out at Optimize Toolset.

About

Hi, 👋, I’m Ryan Hefner  and I built this site for me, and you! The goal of this site was to provide an easy way for me to check the stats on my npm packages, both for prioritizing issues and updates, and to give me a little kick in the pants to keep up on stuff.

As I was building it, I realized that I was actually using the tool to build the tool, and figured I might as well put this out there and hopefully others will find it to be a fast and useful way to search and browse npm packages as I have.

If you’re interested in other things I’m working on, follow me on Twitter or check out the open source projects I’ve been publishing on GitHub.

I am also working on a Twitter bot for this site to tweet the most popular, newest, random packages from npm. Please follow that account now and it will start sending out packages soon–ish.

Open Software & Tools

This site wouldn’t be possible without the immense generosity and tireless efforts from the people who make contributions to the world and share their work via open source initiatives. Thank you 🙏

© 2024 – Pkg Stats / Ryan Hefner

synaudio

v0.3.5

Published

Library that finds the synchronization point between two or more audio clips.

Downloads

1,491

Readme

synaudio

synaudio is a JavaScript library that finds the synchronization points between two or more similar audio clips.

  • Synchronize two or more audio clips by finding the sample offsets with the Pearson correlation coefficient.
  • Correlation algorithm implemented as WebAssembly SIMD.
  • Built in Web Worker implementations for concurrency.

View the live Demo!


Installing

Install via NPM

  • npm i synaudio

Usage

  1. Create a new instance of SynAudio.

    import SynAudio from 'synaudio';
    
    const synAudio = new SynAudio({
      correlationSampleSize: 5000,
      initialGranularity: 16,
    });

Sync Two Clips

SynAudio can synchronize two clips: one base and one comparison. The comparison clip must be a subset of the base clip in order for there to be a valid match. If you don't know the ordering of the clips, see Sync Multiple Clips

  • Call the sync, syncWorker, or syncWorkerConcurrent method on the instance to find the synchronization point in samples between two audio clips.

    • See the API section below for details on these methods.
    // example "base" object
    const base = {
      channelData: [leftChannelFloat32Array, rightChannelFloat32Array]
      samplesDecoded: 5678
    };
    
    // example "comparison" object
    const comparison = {
      channelData: [leftChannelFloat32Array, rightChannelFloat32Array]
      samplesDecoded: 1234
    }
    
    const {
      sampleOffset, // position relative to `base` where `comparison` matches best
      correlation, //  covariance coefficient of the match [ ranging -1 (worst) to 1 (best) ]
    } = await synAudio.syncWorkerConcurrent(
      base, //         audio data to use a base for the comparison
      comparison, //   audio data to compare against the base
      4 //             number of threads to spawn
    );

Sync Multiple Clips

syncMultiple will find the best linear match(es) between a set of two or more clips. Internally, SynAudio will determine the correlation of every order combination of each clip and then will find the path(s) in this graph where the correlation is the highest.

  • Call the syncMultiple method on the instance to find the best synchronization path between two or more audio clips.

    // example "clips" array
    const clips = [
      {
        name: "clip1",
        data: {
          channelData: [leftChannelFloat32Array, rightChannelFloat32Array]
          samplesDecoded: 64445
        }
      },
      {
        name: "clip2",
        data: {
          channelData: [leftChannelFloat32Array, rightChannelFloat32Array]
          samplesDecoded: 24323
        }
      },
      {
        name: "clip3",
        data: {
          channelData: [leftChannelFloat32Array, rightChannelFloat32Array]
          samplesDecoded: 45675
        }
      }
    ];
    
    const results = await synAudio.syncMultiple(
      clips, // array of clips to compare
      8 //      number of threads to spawn
    );
  • The results object will contain a two dimensional array of of match groups containing matching clips. Each match group represents an ordered list of matching audio clips where each clip relates to the previous. The sample offset within each match group is relative to the first clip in the series.

    • In the below example, there are two match groups with the first group containing three clips, and the second containing two clips. There was no significant correlation (no correlation >= options.correlationThreshold) found between the clips in the two match groups. If a clip were to exist that relates the two groups together, then the result would contain only one match group, and relate all other clips to the first one in sequential order.
    // results example
    [
      // first match group (no relation to second group)
      [ 
        {
          name: "cut_1601425_Mpeg", // first clip in match
          sampleOffset: 0,
        },
        {
          name: "cut_2450800_Mpeg",
          correlation: 0.9846370220184326,
          sampleOffset: 849375, // position where this clip starts relative to the first clip
        },
        {
          name: "cut_2577070_Mpeg",
          correlation: 0.9878544973345423,
          sampleOffset: 975645, // position where this clip starts relative to the first clip
        },
      ],
      // second match group (no relation to first group)
      [
        {
          name: "cut_194648_Mpeg",
          sampleOffset: 0,
        },
        {
          name: "cut_287549_Mpeg",
          correlation: 0.9885798096656799,
          sampleOffset: 92901, // position where this clip starts relative to the first clip
        },
      ]
    ]

API

SynAudio

Class that that finds the synchronization point between two or more similar audio clips.

new SynAudio({
  correlationSampleSize: 1234,
  initialGranularity: 1
});

Options

declare interface SynAudioOptions {
  correlationSampleSize?: number; // default 11025
  initialGranularity?: number; // default 16
  correlationThreshold?: number; // default 0.5
}
  • correlationSampleSize optional, defaults to 11025
    • Number of samples to compare while finding the best offset
    • Higher numbers will increase accuracy at the cost of slower computation
  • initialGranularity optional, defaults to 16
    • Number of samples to jump while performing the first pass search
    • Higher numbers will decrease accuracy at the benefit of much faster computation
  • correlationThreshold optional, defaults to 0.5
    • Threshold that will filter out any low correlation matches
    • Only applicable to syncMultiple

Methods

declare class SynAudio {
  constructor(options?: SynAudioOptions);

  /* 
   * Two Clips
  */
  public async sync(
    base: PCMAudio,
    comparison: PCMAudio
  ): Promise<TwoClipMatch>;

  public syncWorker(
    base: PCMAudio,
    comparison: PCMAudio
  ): Promise<TwoClipMatch>;

  public syncWorkerConcurrent(
    base: PCMAudio,
    comparison: PCMAudio,
    threads?: number // default 1
  ): Promise<TwoClipMatch>;

  /* 
   * Multiple Clips
  */
  public syncMultiple(
    clips: AudioClip[],
    threads?: number // default 8
  ): Promise<MultipleClipMatchList[]>;
}

Two Clips

  • sync(base: PCMAudio, comparison: PCMAudio): Promise<TwoClipMatch>
    • Executes on the main thread.
    • Parameters
      • base Audio data to compare against
      • comparison Audio data to use as a comparison
    • Returns
      • Promise resolving to TwoClipMatch that contains the correlation and sampleOffset
  • syncWorker(base: PCMAudio, comparison: PCMAudio): Promise<TwoClipMatch>
    • Execute in a separate thread as a web worker.
    • Parameters
      • base Audio data to compare against
      • comparison Audio data to use as a comparison
    • Returns
      • Promise resolving to TwoClipMatch that contains the correlation and sampleOffset
  • syncWorkerConcurrent(base: PCMAudio, comparison: PCMAudio, threads: number): Promise<TwoClipMatch>
    • Splits the incoming data into chunks and spawns multiple workers that execute concurrently.
    • Parameters
      • base Audio data to compare against
      • comparison Audio data to use as a comparison
      • threads Number of threads to spawn optional, defaults to 1
    • Returns
      • Promise resolving to TwoClipMatch that contains the correlation and sampleOffset

Multiple Clips

  • syncMultiple(clips: AudioClip[], threads?: number): Promise<MultipleClipMatch[][]>
    • Executes on the main thread.
    • Parameters
      • clips Array of AudioClip(s) to compare
      • threads Maximum number of threads to spawn optional, defaults to 8
    • Returns
      • Promise resolving to MultipleClipMatchList[] Two dimensional array containing a list of each matching audio clip groups

Types

Input Types

interface PCMAudio {
  channelData: Float32Array[];
  samplesDecoded: number;
}
  • channelData
    • Array of Float32Array of audio data
    • Each Float32Array represents a single channel
    • Each channel should be exactly the same length
  • samplesDecoded
    • Total number of samples in a single audio channel
interface AudioClip {
  name: string;
  data: PCMAudio;
}
  • name
    • Name of the audio clip
  • data
    • Audio data for the clip

Return Types

interface TwoClipMatch {
  correlation: number;
  sampleOffset: number; 
}
  • correlation
    • Correlation coefficient of the base and comparison audio at the sampleOffset
    • Ranging from -1 (worst) to 1 (best)
  • sampleOffset
    • Number of samples relative to base where comparison has the highest correlation
declare type MultipleClipMatchList =
  | []
  | [MultipleClipMatchFirst, ...MultipleClipMatch];

declare interface MultipleClipMatchFirst {
  name: string;
  sampleOffset: 0;
}

declare interface MultipleClipMatch {
  name: string;
  correlation: number;
  sampleOffset: number;
}
  • name
    • Name of the matching clip
  • correlation
    • Correlation coefficient between the previous clip and this cli
    • Ranging from -1 (worst) to 1 (best)
  • sampleOffset
    • Number of samples relative to the root clip (first clip in the match)

Developing

Prerequisites

  1. Install Emscripten by following these instructions.
    • This repository has been tested with Emscripten 3.1.46.

Building

  1. Make sure to source the Emscripten path in the terminal you want build in.
    • i.e. $ source path/to/emsdk/emsdk_env.sh
  2. Run npm i to install the dependencies.
  3. Run make clean and make to build the libraries.
    • You can run make -j8 where 8 is the number of CPU cores on your system to speed up the build.
  4. The builds will be located in the dist folder.

Testing

  1. Run npm run test to run the test suite.