NgxAudiostream

This Angular 9 library provide a tool to record audio from microphone. Audio is formatted so it can be then sent to Google Speech-to-text.

A web worker is used to process the audio, so the web page won't lag.

Examples/demo

An example that implements this library is available in this repository (it's the npm-audio-test project). You can run it by:

1. Cloning this repository;
2. Installing the dependencies running npm install;
3. Running npm run build
4. Running npm run start on Angular 9;
5. Navigating to localhost:4200.

You can run it using the Docker image:

1. Clone this repository;
2. Build the Docker image running docker build --tag npm-audio:1.0 .;
3. Run the Docker image using docker run --publish 4200:80 --name nac npm-audio:1.0;
4. Navigate to localhost:4200.

You can remove this container by running docker rm --force nac.

This app will record your environment and send you back the raw audio in the console when you press the stop button. To listen to this audio, you must turn it into a compatible format. To do so, you can copy the JS object in the console and run the projects/npm-audio-test/src/app/python-test/python_read_audio.py python function on it. You will then be able to listen to the registered audio (it mays have a terrible sound, because of the Google encoding format).

Usage

This library can be used thanks to a component (a button that shows when audio is recording) or a service.

Using the library component

You must first import the npm-audio module in yours.

import { NgxAudiostreamModule } from 'npm-audio';

@NgModule({
  ...,
  imports: [
    NgxAudiostreamModule,
    ...]
})

Then, you can set the npm-audio component wherever you want in your page. You can see how work all the parameters below.

<npm-audio
  [userCanToggle]="userCanToggle"
  [directlyStartRecording]="directlyStartRecording"
  (audioEmitter)="processMsg($event)"
></npm-audio>

You can import the AudioMessage interface, that match the logs created by the audioEmitter output.

import { AudioMessage } from 'npm-audio';

You can then send theses messages to a node server to process the audio (using socket.io for example).

Using the service

You must first import the service in your component.

import { NgxAudiostreamService } from 'npm-audio';

@Component({
  selector: 'your-component',
  templateUrl: './your.component.html',
  styleUrls: ['./your.component.css'],
})
export class YourComponent implements OnInit {

  constructor(private ngxAudiostreamService: NgxAudiostreamService) {}

Then you must init the service.

this.ngxAudiostreamService.setUpService();

You can now use all its methods to control the library (see more information below).

this.ngxAudiostreamService.startRecording();
this.ngxAudiostreamService.stopRecording();

You can also retrieve messages from the service. You can use the AudioMessage interface for them.

import { AudioMessage } from 'npm-audio';
...
this.recordingStateSubscription = this.ngxAudiostreamService.audioSubject.subscribe(
  (msg: AudioMessage) => {
    ...
  }
);

You can then send theses messages to a node server to process the audio (using socket.io for example).

Collect the audio

There are 3 states of messages sent by the library, as explained in the AudioMessage interface.

interface StartStreamMessage {
  state: 'startStream';
  data: null;
}

interface EndStreamMessage {
  state: 'endStream';
  data: string | Error;
}

interface BinaryDataMessage {
  state: 'binaryData';
  data: Buffer;
}

type AudioMessage = StartStreamMessage | EndStreamMessage | BinaryDataMessage;

• The startStream message tells the app the stream begins. There is nothing in data.
• The endStream message tells the app the stream ends. In data, you can find why it was ended (an error, stopStream was called...).
• The binaryData message provides buffers containing a part of the audio signal. data contains this buffer, that can be adjusted by the optional parameters (see below).

API

This part describes all the input options.

• For the component:

  <npm-audio
    [userCanToggle]="userCanToggle"
    [directlyStartRecording]="directlyStartRecording"
    [options]="options"
    (audioEmitter)="processMessages($event)"
  ></npm-audio>

  Input:

  | Key | Type | Required | Description | | ---------------------- | --------------------- | -------- | ------------------------------------------------------------------------------------ | | userCanToggle | boolean | Yes | Enable users to click the button to start/stop the recorder | | directlyStartRecording | boolean | Yes | Start the recording when the component is initalised. May not work on some browsers. | | options | NgxAudiostreamOptions | Optional | Some optional parameters (see below). |

  Output:

  | Key | Message type | Description | | ------------ | ------------ | ---------------------------------------------------------- | | audioEmitter | AudioMessage | Send information about the recording and the audio signal. |

• For the service:

  Methods:

  this.ngxAudiostreamService.setUpService(options): null;

  This method is used to set up all parameters needed by the library to record audio and send it to the backend. You must call it before any other method !

  | Input parameter | Type | Required | Description | | --------------- | --------------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------ | | serverUrl | string (URL) | Yes | The node server URL, used by the web worker to import socket.io (the socket.io module is available on serverUrl/socket.io/socket.io.js). | | options | NgxAudiostreamOptions | Optional | Some optional parameters (see below). |

  this.ngxAudiostreamService.startRecording(): null;

  This method tells the library to start the recording, with all the parameters given when calling the setUpService method.

  this.ngxAudiostreamService.stopRecording(): null;

  This method tells the library to stop the recording.

  Subscriptions:

  this.audioSubscription = this.ngxAudiostreamService.audioSubject.subscribe(
    (message: AudioMessage) => {
      ...
    }
  );

  This subject sends information about the recording and the audio signal.

• About NgxAudiostreamOptions

  interface NgxAudiostreamOptions {
    latencyHint?: number | 'interactive' | 'balanced' | 'playback';
    bufferSize?: number;
    outSampleRate?: number;
  }

  | Key | Type | Required | Description | | ------------- | --------------------------------------------------- | --------------------------------- | -------------------------------------------------------------------------------------------------------------------------- | | latencyHint | number or 'interactive' or 'balanced' or 'playback' | Optional (default: 'interactive') | Used when creating the AudioContext (go see https://developer.mozilla.org/en-US/docs/Web/API/AudioContextLatencyCategory). | | bufferSize | Positive integer | Optional (default: 2048) | Size of the buffers containing the audio and sent to the server. | | outSampleRate | Positive integer (< 44100) | Optional (default: 16000) | Sample rate of the audio signal sent to the back. |

• About AudioMessage

interface StartStreamMessage {
  state: 'startStream';
  data: null;
}
interface EndStreamMessage {
  state: 'endStream';
  data: string | Error;
}

interface BinaryDataMessage {
  state: 'binaryData';
  data: Buffer;
}

type AudioMessage = StartStreamMessage | EndStreamMessage | BinaryDataMessage;

| Key | Type | Description | | ----- | -------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------- | | state | 'startStream' or 'endStream' or 'binaryData' | Give information about the message state (if the recording begins, if it ends, or if the message provides a part of the audio signal). | | data | string or Error or Buffer | Any information given to understant why the recording ended, or a buffer that contains a part of the audio signal. |

Additional information

• This library was created on the work of vin-ni, particulary on this repository. You can use it to implement a Node server streaming to the Google Speech-to-text API.

• You can check additional information about Google Speech-to-text methods by checking on the official documentation.

Copyrights

Integrate or build upon it for free in your personal or commercial projects. Don't republish, redistribute or sell "as-is". You can contact me at [email protected].