STP Plugin: Microsoft Cognitive Services Speech

The Sketch-thru-Plan (STP) recognizer can employ transcribed speech generated by potentially different recognizers. To promote code reuse and make it possible to more easily swap recognizers, the functionality should be packaged as a plugin that conforms to a well-known interface.

This plugin is implemented based on Microsoft's Cognitive Services Speech to Text. It implements two different strategies:

1. The simpler strategy performs recognition in one single shot. The expectation is that the recognizeOnce method is invoked by the client app whenever a pen/mouse down is detected. The results are returned via a Promise (see quicktstart for an examples)
2. The second strategy performs recognition over a period of time controlled by the client app via startRecognizing() and stopRecognizing(). The expectation is that startRecognizing() is invoked by the client app whenever a pen/mouse down is detected, and that stopRecognizing() is called when pen/mouse up is detected. Results are returned via event handlers the client app needs to subscribe to (see samples for examples)

Accessing the plugin functionality

You can get the plugin from npm:

npm install --save @hyssostech/azurespeech-plugin

Or you can embed directly as a script using jsdelivr. As always, it is recommended that a specific version be used rather than @latest to prevent breaking changes from affecting existing code

<!-- Include _after_ the external services such as the Microsoft Cognitive Services Speech -->
<script src="https://cdn.jsdelivr.net/npm/@hyssostech/azurespeech-plugin@latest/dist/stpazurespeech-bundle-min.js"></script>

Referencing the plugin

The plugin is built as a UMD library, and is therefore compatible with plain vanilla (IIFE), AMD and CommonJS. Also included is an ESM bundle (stpazurespeech-bundle.esm.js).

When used in vanilla javascript, an StpAS exported global can be used to access the SDK types:

const stpConn = new StpAS.AzureSpeechRecognizer(azureSubscriptionKey, azureServiceRegion);

In typescript, import @hyssostech/azurespeech-plugin after installing via npm:

import * as StpAS from "@hyssostech/azurespeech-plugin";
const stpConn = new StpAS.AzureSpeechRecognizer(azureSubscriptionKey, azureServiceRegion);

Or import individual types:

import { StpazurespeechConnector } from "@hyssostech/azurespeech-plugin";
const stpConn = new AzureSpeechRecognizer(azureSubscriptionKey, azureServiceRegion);

Examples

• This single-shot recognition approach is used in the quicktstarts
• The recognition over the duration of sketching is used in the samples

Building the project

The repository includes a pre-built dist folder that can be used directly for testing. If changes are made to the sample and there is a need to rebuild, run:

npm install
npm run build

Documentation

Additional documentation can be found in the docs folder.