Augnito Plate Button

Overview

Augnito Plate Button enables Augnito SDK support into within Plate enabling speech to text on it. It requires a React app with Plate and it's peer dependencies installed.

Some commands make use of specific plugins such as bold, start lists, etc.

Required peer dependencies

Before installing the Augnito Plate Button a React app with a functional Plate editor bust be already set up. The instructions on how to bootstrap a React app are out of the scope of this library.

Plate

npm install @udecode/plate

Plate's peers dependencies are also required

npm install slate slate-react slate-history slate-hyperscript react react-dom styled-components

Up to this point this is a basic Plate installation. Please check the Version Support section in order to avoid peer dependencies errors.

Plugins

In order to use some commands such as marks (bold, italics, etc), lists and others the related plugin must be installed and added to the plate instance. These must be installed and enabled as described on Plate's documentation.

Installation

Once all the dependencies are installed the Augnito Plate Button can be installed via npm using:

npm i augnito-plate-button

Basic Usage

1- Import the library and styles into the component where the editor will be hosted

import { ToolbarAugnito, AugnitoAPIServer } from 'augnito-plate-button';
import 'augnito-plate-button/dist/style.css';

2- Prepare your SDK provided values

const server = '<your server>';
const accountCode = '<your accountcode>';
const accessKey = '<your accesskey>';
const userTag = '<your usertag>';
const deviceId = '<your deviceid>';
const sourceApp = '<your sourceapp>';
const lmId = '<your lmid>';

3- Use the component

<Plate>
  <HeadingToolbar>
    <ToolbarAugnito
      server={server}
      accountCode={accountCode}
      accessKey={accessKey}
      userTag={userTag}
      deviceId={deviceId}
      sourceApp={sourceApp}
      lmId={lmId}
    ></ToolbarAugnito>
  </HeadingToolbar>
</Plate>

Full example component

Assuming a working React App with Plate and it's dependencies set up:

import './App.css';
import { Plate, HeadingToolbar } from '@udecode/plate';
import { ToolbarAugnito, AugnitoAPIServer } from 'augnito-plate-button';
import 'augnito-plate-button/dist/style.css';

const server = '<your server>';
const accountCode = '<your accountcode>';
const accessKey = '<your accesskey>';
const userTag = '<your usertag>';
const deviceId = '<your deviceid>';
const sourceApp = '<your sourceapp>';
const lmId = '<your lmid>';

function App() {
  return (
    <div className="App">
      <div className="editor">
        <Plate>
          <HeadingToolbar>
            <ToolbarAugnito
              server={server}
              accountCode={accountCode}
              accessKey={accessKey}
              userTag={userTag}
              deviceId={deviceId}
              sourceApp={sourceApp}
              lmId={lmId}
            ></ToolbarAugnito>
          </HeadingToolbar>
        </Plate>
      </div>
    </div>
  );
}

export default App;

ToolbarAugnito Props

Required

| prop | type | | ----------- | ------ | | server | AugnitoAPIServer (int where India = 0, UK = 1, US = 2)| | accountCode | string | | accessKey | string | | userTag | string | | sourceApp | string | | lmId | string |

Optional

| prop | type | notes | | ------------------------ | --------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------- | | deviceId | string | | handlePartialText | (partial:string) => void | callback fired when partial text is received . Check the Advance Example. | | handleCommand | (command: Partial<ActionRecipe>) => boolean | handles a command. Return true to interrupt normal flow. Check the Advance Example. | | handleFinalText | (final: Partial<ActionRecipe>) => boolean | handles the resulting final text. Return true to interrup normal flow. Check the Advance Example. | | handleIdleMic | () => void | callback fired when there has not been activity but the microphone is open. Check the Advance Example. | | augnitoStore | AugnitoStore | store to work with features such as select commands. | | includeActivityIndicator | boolean | toggle the button activity indicator. The indicator shows the mic status and received partial text. |

Commands

Besides regular speech to text the button can process the following commands: |command|notes| |------|------| |stop mic|stops dictation. Sames as clicking the dictation button once started.| |new line|same as hitting carriage return. If used within a list it creates a new list entry (see lists commands)| |go to document start|| |go to document end|| |go to next sentence|| |go to previous sentence|| |go to sentence start | goto beginning of sentence|| |go to sentence end | goto end of sentence|| |undo it|| |redo it|| |select next word|| |select next N words|| |deselect| removes the just the selection (contents on the selection remain unchanged).| |select last word|| |select last N words|| |select next sentence|if cursor not at the end of a sentence selects the current sentence.| |select last sentence|same principe with select next sentence.| |delete that|deletes the word under the cursor or the contents of a selection.| |delete it|same as delete that command.| |bold it|bolds the word under the cursor or the selected text. Requires createBoldPlugin().| |bold that| same as bold it command.| |italicize it| makes the current word or the selected text italic Requires createItalicPlugin().| |italicize that| same as italicize it command.| |underline it| underlines the current word or the selected text Requires createUnderlinePlugin().| |underline that| same as underline it command.| |capitalize it| capitalizes the current word or the selected text.| |capitalize that| same as capitalize it command.| |next field|moves to and selects the next dynamic field (next [] pair or next [content] occurrence).| |previous field|moves to and selects the previous dynamic field (previous [] pair or previous [content] occurrence).| |start number list|starts a numbered (ordered) list. Requires createListPlugin().| |start bullet list|starts a bullet list. Requires createListPlugin().| |stop bullet list | stop number list|ends the current list context. Requires createListPlugin().|

Dynamic selection commands

Support for selection command is included with the library. It requires the AugnitoSelectPlugin in order to work.

Augnito Select Plugin setup

1 - Import required parts

import {
  ToolbarAugnito,
  buildAugnitoStore,
  createAugnitoSelectPlugin,
  AugnitoAPIServer
} from 'augnito-plate-button';

import 'augnito-plate-button/dist/style.css';

2 - Setup plate to react to dynamic commands

Just like Plate's highlight the plugins now must be wrapped within a useMemo hook

const augnitoStore = useMemo(() => buildAugnitoStore(), []);
const searchTerm = augnitoStore.use.searchTerm();

const plugins = useMemo(
  () =>
    createPlugins(
      [
        //... list of desired plugins here
        createAugnitoSelectPlugin({
          options: {
            augnitoStore
          }
        })
      ],
      {
        components: createPlateUI()
      }
    ),
  [searchTerm]
);

3 - Include the AugnitoStore as a prop for ToolbarAugnito

<Plate plugins={plugins}>
  <HeadingToolbar>
    <ToolbarAugnito
      server={server}
      accountCode={accountCode}
      accessKey={accessKey}
      userTag={userTag}
      deviceId={deviceId}
      sourceApp={sourceApp}
      lmId={lmId}
      augnitoStore={augnitoStore}
    ></ToolbarAugnito>
  </HeadingToolbar>
</Plate>

Full example

import {
  Plate,
  HeadingToolbar,
  createBoldPlugin,
  createItalicPlugin,
  createUnderlinePlugin,
  createListPlugin,
  createPlateUI,
  createPlugins
} from '@udecode/plate';
import {
  ToolbarAugnito,
  buildAugnitoStore,
  createAugnitoSelectPlugin,
  AugnitoAPIServer
} from 'augnito-plate-button';

import './App.css';

import { useMemo } from 'react';
import 'augnito-plate-button/dist/style.css';

const server = '<your server>';
const accountCode = '<your accountcode>';
const accessKey = '<your accesskey>';
const userTag = '<your usertag>';
const deviceId = '<your deviceid>';
const sourceApp = '<your sourceapp>';
const lmId = '<your lmid>';

function App() {
  const augnitoStore = useMemo(() => buildAugnitoStore(), []);
  const searchTerm = augnitoStore.use.searchTerm();

  const plugins = useMemo(
    () =>
      createPlugins(
        [
          createListPlugin(), // list support
          createBoldPlugin(), // bold mark
          createItalicPlugin(), // italic mark
          createUnderlinePlugin(), // underline mark
          createAugnitoSelectPlugin({
            options: {
              augnitoStore
            }
          })
        ],
        {
          // Plate components
          components: createPlateUI()
        }
      ),
    [searchTerm]
  );

  return (
    <div className="App">
      <div className="editor">
        <Plate plugins={plugins}>
          <HeadingToolbar>
            <ToolbarAugnito
              server={server}
              accountCode={accountCode}
              accessKey={accessKey}
              userTag={userTag}
              deviceId={deviceId}
              sourceApp={sourceApp}
              lmId={lmId}
              augnitoStore={augnitoStore}
            ></ToolbarAugnito>
          </HeadingToolbar>
        </Plate>
      </div>
    </div>
  );
}

export default App;

List of Dynamic Commands

| command | notes | | --------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------- | | select <word | phrase> | searches and selects the word or phrase. If more than one appears a numbered indicator allows to specify which one to select. | | bold <word | phrase> | searches and bolds the word or phrase. If more than one appears a numbered indicator allows to specify which one to change. | | underline <word | phrase> | searches and underlines the word or phrase. If more than one appears a numbered indicator allows to specify which one to change. | | capitalize <word | phrase> | searches and capitalizes the word or phrase. If more than one appears a numbered indicator allows to specify which one to change. | | italicize <word | phrase> | searches and converts to cursive the word or phrase. If more than one appears a numbered indicator allows to specify which one to change. |

Advanced Example

The library can be further extended via ToolbarAugnito Props. The way to achieve this is to "short circuit" the handleCommand and handleFinalText methods.

Full example of a React (Typescript) app overriding these methods.

import './App.css';

import { useMemo, useCallback } from 'react';
import {
  Plate,
  HeadingToolbar,
  createBoldPlugin,
  createItalicPlugin,
  createUnderlinePlugin,
  createListPlugin,
  createPlateUI,
  createPlugins
} from '@udecode/plate';
import {
  ToolbarAugnito, // button (enabled TTS)
  buildAugnitoStore, // creates sync store
  createAugnitoSelectPlugin, // dynamic command support plugin
  ActionRecipe, // handler type for final text and command
  AugnitoAPIServer
} from 'augnito-plate-button';
import 'augnito-plate-button/dist/style.css'; // styles

// provided auth values
const server = '<your server>';
const accountCode = '<your accountcode>';
const accessKey = '<your accesskey>';
const userTag = '<your usertag>';
const deviceId = '<your deviceid>';
const sourceApp = '<your sourceapp>';
const lmId = '<your lmid>';

function App() {
  // support for dynamic commands
  const augnitoStore = useMemo(() => buildAugnitoStore(), []);
  const searchTerm = augnitoStore.use.searchTerm();

  // use memo required for dynamic commands
  const plugins = useMemo(
    () =>
      createPlugins(
        [
          createListPlugin(), // list support
          createBoldPlugin(), // bold mark
          createItalicPlugin(), // italic mark
          createUnderlinePlugin(), // underline mark
          createAugnitoSelectPlugin({
            options: {
              augnitoStore
            }
          })
        ],
        {
          // Plate components
          components: createPlateUI()
        }
      ),
    [searchTerm]
  );

  // #region custom handlers
  const handlePartialText = useCallback((partial: string) => {
    console.log(partial);
  }, []);

  const handleIdleMic = useCallback(() => {
    console.log('idle mic');
  }, []);

  const handleFinalText = useCallback((finalText: Partial<ActionRecipe>) => {
    const handled = true; // or false if the SDK should continue normal flow
    console.log({
      handler: 'final text',
      handled,
      text: finalText.receivedText
    });
    return handled;
  }, []);

  const handleCommand = useCallback((command: Partial<ActionRecipe>) => {
    const handled = true; // or false if the SDK should continue normal flow
    console.log({ handler: 'command', handled, action: command.action });
    switch (command.action) {
      case '':
        break;

      default:
        break;
    }
    return handled;
  }, []);
  // #endregion

  return (
    <div className="App">
      <div className="editor">
        <Plate plugins={plugins}>
          <HeadingToolbar>
            <ToolbarAugnito
              server={server}
              accountCode={accountCode}
              accessKey={accessKey}
              userTag={userTag}
              deviceId={deviceId}
              sourceApp={sourceApp}
              lmId={lmId}
              augnitoStore={augnitoStore}
              handleCommand={handleCommand}
              handleFinalText={handleFinalText}
              handleIdleMic={handleIdleMic}
              handlePartialText={handlePartialText}
            ></ToolbarAugnito>
          </HeadingToolbar>
        </Plate>
      </div>
    </div>
  );
}

export default App;

Version support

The Augnito Plate Button and Augnito Select Plugin use many functions, API, structures and code from Plate, Slate, Rect and such. To ensure performance and bundle size those dependencies are expected to be installed on the consumer solution source (as stated on the required dependencies section) which means these are present only once on the project. The versions that should not break compatibility are: |library|semver| |------|------| |@udecode/plate|>=10.2.3| |react|^17| |react-dom|^17| |slate|>=0.75| |slate-history|>=0.66| |slate-hyperscript|>=0.67| |slate-react|>=0.74.2| |styled-components|^5|