Slang Web Form Filling Assistant Developer Guide

Slang’s Web form filling assistant (aka SlangFormFillingAssistant) is a fully trained voice assistant specialized for Web apps to help users fill out forms on their platform using voice.

The following guide describes the steps that apps need to take in order to integrate with the assistant.

Step 1: Add the assistant to your project

Just install @slanglabs/slang-form-filling-assistant package from npm

npm install @slanglabs/[email protected]

or if you're using yarn:

yarn add @slanglabs/[email protected]

Then to bring slang into scope:

import SlangFormFillingAssistant from "@slanglabs/slang-form-filling-assistant"

Step 2: Initialize slang assistant

First initialize the slang form filling assistant.

SlangFormFillingAssistant.init({
    buddyId: "<buddy_id>",  // you would have gotten it from the Slang team. Required.
    apiKey: "<api_key>",    // you would have gotten it from the Slang team. Required.
    environment: "prod",    // use "stage" when in development mode. Defaults to stage if omitted.
});

Step 3: Register your action listener

Create an object with an onResponse attribute, which is a function that will get called when slang receives a response for the current field.

Then call SlangFormFillingAssistant.setAction with that object.

SlangFormFillingAssistant.setAction(
    {
        onResponse: (fieldInfo, userJourney) => {
            // fieldInfo - { currentField, value }
            doSomethingWith(fieldInfo.value);
            userJourney.setSuccess();
            return userJourney.AppState.FORM_FILLING;
        },
        onReady: () => {
            // The onReady callback will be called right after initialization.
            console.log("Slang has been initialized")
            SlangFormFillingAssistant.ui.showTrigger();
        }
    });

The onResponse callback will be passed the fieldInfo and the userJourney.

FieldInfo

The fieldInfo will contain

• currentField - This is the currentField which got filled. For eg: PHONE or OTP
• value - This is the value that Slang nlu recognized. For eg: 9876543210

User Journey

The userJourney object will have the following methods for you to let Slang know about the current User Journey.

• setSuccess() - Indicates that the field was successfully filled. Slang will speak out the success prompt.
• setFailure() - Indicates that the field was NOT filled. Slang will speak out a generic failure prompt and then retry.
• setFailureBailout() - Same as setFailure() but without retrying.
• setPhoneNumberInvalid() - Indicates that the phone input was invalid. Slang will speak out the prompt for invalid phone input and retry.
• setPhoneNumberInvalidBailout() - Same as setPhoneNumberInvalid() but without retrying.
• setOTPInvalid() - Indicates that the OTP input was invalid. Slang will speak out the prompt for invalid OTP input and retry.
• setOTPInvalidBailout() - Same as setOTPInvalid() but without retrying.
• setAadharNumberInvalid() - Indicates that the Aadhar input was invalid. Slang will speak out the prompt for invalid Aadhar input and retry.
• setAadharNumberInvalidBailout() - Same as setAadharNumberInvalid() but without retrying.

APP State

From the onResponse method you have to return the current appState. In this case you have to return userJourney.AppState.FORM_FILLING.

Asynchronous tasks

onResponse can be async, in which case either return a promise or use async/await.

SlangFormFillingAssistant.setAction(
{
    onResponse: async (fieldInfo, userJourney) => {
            const { currentField, value } = fieldInfo;
            switch(currentField){
                case SlangFormFillingAssistant.FORM_FIELD.PHONE_MOBILE : {
                    if (value.length !== 10) {
                        userJourney.setPhoneNumberInvalid();
                        return userJourney.AppState.FORM_FILLING;
                    } else {
                        SlangFormFillingAssistant.readInput(SlangFormFillingAssistant.FORM_FIELD.OTP,{});
                        userJourney.setSuccess();
                        return userJourney.AppState.FORM_FILLING;
                    }
                }
                case SlangFormFillingAssistant.FORM_FIELD.OTP : {
                    const isOTPValid = await verifyOTP(value); // verify otp is an async network call
                    if (!isOTPValid) {
                        userJourney.setOTPInvalid();
                        return userJourney.AppState.FORM_FILLING;
                    } else {
                        userJourney.setSuccess();
                        return userJourney.AppState.FORM_FILLING;
                    }
                }
}}});

SlangFormFillingAssistant.readInput(SlangFormFillingAssistant.FORM_FIELD.PHONE_MOBILE,{});

Supported Fields

Currently the follwing fields are supported:

• PHONE
• OTP
• AADHAR

These enum values can be accessed via :

• SlangFormFillingAssistant.FORM_FIELD.PHONE_MOBILE
• SlangFormFillingAssistant.FORM_FIELD.OTP
• SlangFormFillingAssistant.FORM_FIELD.AADHAR

Let Slang know which field to prompt for next

SlangFormFillingAssistant.setInput(SlangFormFillingAssistant.FORM_FIELD.PHONE_MOBILE,{});

This lets Slang know which field to ask for, next time the user clicks on the mic button.

Automatically prompt the user for an input

SlangFormFillingAssistant.readInput(SlangFormFillingAssistant.FORM_FIELD.PHONE_MOBILE,{});

This will cause Slang to prompt the user for an input for the set field.

You can also call this method at the end of onResponse of one field, to create a continous flow. Like so:

SlangFormFillingAssistant.setAction(
    {
        onResponse: (fieldInfo, userJourney) => {
            SlangFormFillingAssistant.readInput(SlangFormFillingAssistant.FORM_FIELD.OTP,{})
            userJourney.setSuccess();
            return userJourney.AppState.FORM_FILLING;
        }
    });

Note: If you automatically prompt right after page load, and the user has not clicked on anything on the page yet, the Prompt might be muted on certain browsers (eg, chrome) due to browser policies.

Show or Hide the Trigger

To hide :

SlangFormFillingAssistant.ui.hideTrigger()

To show :

SlangFormFillingAssistant.ui.showTrigger()

Note: The assistant initializes in a hidden mode. To show the trigger for the first time you have to call SlangFormFillingAssistant.ui.showTrigger() on the onReady method in setAction. Like so:

SlangFormFillingAssistant.setAction(
    {
        onResponse: (fieldInfo, userJourney) => {
            SlangFormFillingAssistant.readInput(SlangFormFillingAssistant.FORM_FIELD.OTP,{})
            userJourney.setSuccess();
            return userJourney.AppState.FORM_FILLING;
        },
        onReady: () => {
            SlangFormFillingAssistant.ui.showTrigger();
        }
    });

Speech Recognition Hints

While you start filling any field by voice you can provide speech recognition hints like so :

SlangFormFillingAssistant.readInput(currentFormField,{
    speechRecognitionHints: {
      "en-IN": ["foo","bar"],
      "hi-IN": ["नमस्ते", "दुनिया"]
    }
});

This helps in improving speech recognition for that field.