Flocking - Creative audio synthesis for the Web!

What is Flocking?

Flocking is a JavaScript audio synthesis framework designed for artists and musicians who are building creative and experimental Web-based sound projects. It supports Firefox, Chrome, Safari on Mac OS X, Windows, Linux, iOS, and Android.

Unlike comparable tools, Flocking is declarative. Its goal is to promote a uniquely community-minded approach to instrument design and composition. In Flocking, unit generators and synths are specified as JSON, making it easy to save, share, and manipulate your synthesis algorithms. Send your synths via Ajax, save them for later using HTML5 local data storage, or algorithmically produce new instruments on the fly.

Because it's just JSON, every instrument you build using Flocking can be easily modified and extended by others without forcing them to fork or cut and paste your code. This declarative approach will also help make it easier to create new authoring, performance, metaprogramming, and social tools on top of Flocking.

Flocking was inspired by the SuperCollider desktop synthesis environment. If you're familiar with SuperCollider, you'll feel at home with Flocking.

To learn more about Flocking's architecture and approach, please see:

Clark, C. and Tindale, Adam. "Flocking: A Framework for Declarative Music-Making on the Web" in Georgaki, A. and Kouroupetroglou (eds.). The Joint Proceedings of the ICMC and SMC, (2014). slides

Flocking does not currently support the Web Audio API's new AudioWorklet specification, which allows for JavaScript-based signal processing code to run in a separate, real-time thread. This means that Flocking is not currently well-suited to applications that involve a lot of graphics rendering or user interaction. Work is underway on a new core for Flocking, called Signaletic, which will provide full support for all Web Audio native nodes as well as the use of AudioWorklets.

Community

Flocking has an inclusive and supportive community with several forums where you can ask for help and share the projects you're working on.

Mailing List

The flocking mailing list is the place to ask questions, share code, and request new features.

Chat

Flocking has an IRC channel. Join #flocking on irc.freenode.net.

Status and Roadmap

Flocking is in active development. It has bugs and it is growing quickly.

The project's development roadmap is documented in the wiki. Plans include:

• Better support for interleaving Flocking unit generators with Web Audio API nodes
• A new format for specifying connections between unit generators
• A "live data merging" environment for live coding
• Graphical editing of Flocking synth defs

Unplanned features, bug fixes, and contributions are welcome and appreciated, of course. The Flocking project adheres to the Contributor Covenant guidelines, and is an open and welcoming community.

Documentation and Demos

• Flocking's documentation is in the repository
• Demos can be found in the Flocking Playground
• Other examples are located in the examples repository

Getting Started

The latest stable release of Flocking can be downloaded from the Flocking releases page.

Concatenated and minified Flocking builds, suitable for development and production respectively, are included in the dist directory. Flocking can also be built manually using Grunt.

Here's how to include Flocking's development file in your HTML page:

<!-- This includes Flocking and all its dependencies, including jQuery and Infusion -->
<script src="flocking/dist/flocking-all.js"></script>

For more information on using Flocking in a browser, read the Getting Started tutorial. If module loaders are your thing, Flocking also supports the CommonJS and AMD styles.

Using Flocking

Flocking consists of a handful of primary components that are configured using JSON specifications. These include: Unit Generators (ugens), Synths, Schedulers, and the Environment.

Unit Generators

Unit generators, or ugens for short, are the basic building blocks of synthesis; they do the work of generating or processing audio signals in Flocking. UGens have multiple inputs and a single output. Some unit generators support multiple input or output multiple channels.

A unit generator can be wired to other unit generators, supporting sophisticated signal processing graphs. Unit generators implement one primary method, gen(numSamps), which is responsible for processing a block of audio samples.

Typically, however, you never need to interact with unit generator instances directly. Instead, you create declarative "unit generator definitions" (ugenDefs) objects, letting Flocking take care of creating the actual unit generator instances. UGenDefs are composed into trees called synthDefs. Here's an example of a ugenDef:

{
    id: "carrier",             // A unique ID used when updating this ugen's input values.

    ugen: "flock.ugen.sinOsc", // The fully-qualified name of the desired unit generator,
                               // specified as a "key path" (a dot-separated string).

    rate: "audio",             // The rate at which the unit generator should run.

    inputs: {                  // The input values for this unit generator. Each UGen has different inputs.
        freq: 440              // For convenience, these inputs don't need to be nested inside the "inputs"
    },                         // container, but you might want to for readability.

    options: {
        interpolation: "linear" // Other non-signal options such as interpolation rates, etc.
                                // Options are also specific to the type of unit generator.
    }
}

Synths

A Synth is a self-contained collection of unit generators that represents a synthesizer, instrument, or signal processor of some kind. Multiple synths can run at the same time, and they can be connected together using shared interconnect buses. For example, a mixing board Synth could be created to mix and process signals from several tone-generating Synths and effect Synths.

To create a synth, you specify a synthDef option, which is a declarative tree of unit generator definitions. Here's a simple example of a sine oscillator (named carrier) whose amplitude is modulate by another sine oscillator, mod:

{
    id: "carrier",                  // Name this unit generator "carrier," exposing it as an input to the synth.
    ugen: "flock.ugen.sinOsc",      // Sine oscillator ugen.
    freq: 440,                      // Give it a frequency of 440 Hz, or the A above middle C.
    mul: {                          // Modulate the amplitude of this ugen with another ugen.
        id: "mod",                      // Name this one "mod"
        ugen: "flock.ugen.sinOsc",      // Also of type Sine Oscillator
        freq: 1.0                       // Give it a frequency of 1 Hz, or one wobble per second.
    }
}

Synths can be updated in realtime by using the get() and set() methods. Any unit generator with an id property in its ugenDef will automatically be exposed as a named input to the synth. To update a unit generator, a key path is used to specify the desired input. Key paths are dot-delimited, path-like strings that allow you to address any part of the unit generator tree. Here's an example of a key path:

"carrier.freq.mul" // Refers to the amplitude (mul) of the carrier's frequency.

Getting Synth Inputs

An input can be retrieved from a synth by invoking the get() method with a key path. If the target of the path is a value unit generator, its value will be returned directly. If it is any other kind of input, its ugen instance will be returned instead.

var freq = synth.get("carrier.freq");

Setting Synth Inputs

Synth inputs can be set by calling the aptly-named set() method. Flocking's signal processing pipeline is dynamic, so unit generators can be added or replaced at any time. Behind the scenes, everything is a unit generator, even static values.

Updating a value:

synth.set("carrier.freq", 440);

Replacing the target unit generator with a new one:

synth.set("carrier.freq", {
    ugen: "flock.ugen.sinOsc",
    freq: 2
});

Updating multiple inputs at once:

synth.set({
    "carrier.freq": 440,
    "carrier.mul": 0.5,
    "modulator.freq": 123
});

Rates

To ensure reasonable performance on resource-constrained devices such as phones and low-power computers (e.g. Chromebooks, Raspberry Pi), Flocking uses a block-based architecture. By default, ugens and synths will produce 64 samples per block. This value is configurable by specifying the blockSize option when initializing Flocking.

There are three primary types of signal rates in Flocking: audio, control, and constant. Audio rate produces a full block of samples, control rate produces one sample per block, and constant rate will alway produce the same value. Synths also support two other rates, demand and scheduled. A demand rate synth will only produce a value when its gen() method is invoked. Scheduled synths are under the control of a scheduler instead of the sample output clock.

The Environment

An Environment represents a whole "audio system" or "context" in Flocking. It is responsible for evaluating all Synths instances and outputting their samples to the current audio output strategy. An environment manages a list of Synth instances and evaluates them in order. You should instantiate only one flock.enviro for your entire application.

The Flocking shared environment is created by calling flock.init():

var environment = flock.init();

Before you'll hear any sound, the environment needs to be started. You only need to start the environment once. This is done using the start() method:

environment.start();

To stop the environment from generating samples, use the stop() method:

environment.stop();

Synths and the Environment

By default, synths are automatically added to the end (or tail) of the environment's list of synths. This means that synths will start playing immediately when you create them.

If you want to defer the playing of a Synth to a later time, you can override the addToEnvironment option when you instantiate it:

var mySynth = flock.synth({
    synthDef: {
        ugen: "flock.ugen.sin",
        freq: 440
    },
    addToEnvironment: false
});

If you need to manage the Environment's list of synths manually, you can use the methods provided by the flock.nodeListComponent grade.

To add a synth to the head of the graph so that it will be evaluated first:

enviro.head(mySynth);

To add a synth to the tail of the graph so that it will be evaluated after all other synths):

enviro.tail(mySynth);

Schedulers

A scheduler allows you to schedule changes to Synths at a particular time. Currently, there is one type of Scheduler, the asynchronous scheduler. Unfortunately, it is driven by the browser's notoriously inaccurate setTimeout() and setInterval() clocks, which means that it will drift by up to 75 ms depending on the browser's load. In practice, however, this drift is sufficient for scheduling many kinds of changes, and if sample-level accuracy is needed, unit generators such as flock.ugen.sequence, flock.ugen.change and flock.ugen.random can be used.

A block-accurate scheduler is planned for an upcoming release of Flocking. In the meantime the asynchronous scheduler does a decent job of keeping "pleasantly inaccurate" time.

If you need to, you can always schedule arbitrary events using plain old functions:

// Fade out after 8 seconds.
band.scheduler.once(8, function () {
    band.sinSynth.set({
        "carrier.mul.start": 0.25,
        "carrier.mul.end": 0.0,
        "carrier.mul.duration": 1.0
    });
});

The Flocking scheduler, in future releases, will be removed and replaced with Bergson, a highly accurate scheduler written specifically for audio and video applications.

Testing

Flocking uses Testem and Vagrant to run its test suite across many of its supported environments, both on the host and in virtual machines running Windows and Linux.

Prerequisites

1. Vagrant
2. VirtualBox
3. The GPII CI Plugin: 'vagrant plugin install vagrant-gpii-ci'

Running Flocking's Test Suite

Run all tests on your host:

• npm test

Run only browser tests:

• npm run browser-test

Run all VM-based tests:

• npm run all-vm-test

Run all tests on a Vagrant-based Windows VM:

• npm run windows-vm-test

Run all tests on a Linux VM:

• npm run linux-vm-test

Stop all Vagrant VMs:

• vagrant halt

Destroy all Vagrant boxes, freeing up hard disk space:

• npm run destroy-vms

Compatibility

Flocking is currently tested on the latest versions of Firefox, Chrome, Safari, and Microsoft Edge on Mac, Windows, Linux, iOS, and Android.

License

Flocking is distributed under the terms of both the MIT or GPL 2 Licenses. As a result, you can choose the license that best suits your project. The full text of Flocking's MIT and GPL licenses are at the root of the repository.

Credits

Flocking is developed by Colin Clark and the community. It was named after a composition by James Tenney, a composer, thinker, and early pioneer of computer music who was my composition teacher and a huge influence on my work. I hope you find this library useful enough to create projects as beautiful and inspiring as Jim's Flocking.

Thanks to:

• Adam Tindale for the tanh distortion unit generator, documentation improvements, and several of the Playground demos
• Alfredo Matas for test automation on Windows and Linux
• Johnny Taylor for styling improvements to the Playground
• Dan Stowell for the Freeverb and Delay1 unit generators
• Mayank Sanganeria for the flock.ugen.granulator unit generator
• Vitus for his contributions to the original version of the interactive Playground
• Myles Borins for pushing the limits of Flocking early in its development
• Antranig Basman for code review, architectural advice, and help with maths
• Alex Geddie for teaching me a ton about synthesis and computer music