cloud-sdk

Install

npm install @meetkaims/cloud-sdk

yarn add @meetkaims/cloud-sdk

Setup

For a frontend service, contact either Anthony or Vincent to have the domain be whitelisted. No additional configuration is needed.

To connect with an API key, create an account with MKMS and generate one. Then, initialize the client with mkcloud.setConfig:

import mkcloud from '@meetkaims/cloud-sdk'

mkcloud.setConfig({
  url: 'https://cloud-server.meetkaims.com'
  apiKey: '...'
});

You can also change the url to hit staging instead (NOTE: You will need to create a separate API key for staging):

import mkcloud from '@meetkaims/cloud-sdk'

mkcloud.setConfig({
  url: 'https://cloud-server.staging.meetkaims.com'
  apiKey: '...'
});

Usage

Generate a sentence embedding:

mkcloud.embed({
  queries: ['foo', 'bar'],
  language: 'en',
});

Orchestrate a process:

• Consult with Arnold to determine an available list of tools that can be customized for the task.

mkcloud.arnold({
  data: {
    query: 'give me an example',
    tool_configs: [
      {
        tool_id: -1,
        tool_name_id: 'example',
        tool_name: {
          'en': 'example tool',
        },
        description: {
          'en': 'example description',
        },
        return_direct: true,
      }
    ],
    language: 'en',
  },
  customerId: 'foo',
  endUserId: 'bar',
});

Setup a bot:

• Decorate functions that you wish to allow the bot to call.
• BotSense provide inputs to the bot.
• BotCommand are options the bot can take at each stage.
• Calling mkcloud.bot compiles senses and commands into a LLM request. The response is translated into function executions.

import { BotCommand, BotSense } from '@meetkaims/cloud-sdk'

@BotSense('vision')
def screenshot(self) => Image:
	return;

@BotCommand('tool', { description='...' })
def tool(self, params) => Any:
	return;

mkcloud.bot({
  query: 'use a tool',
  enabledCommands: ['tool'],
  enabledSenses: ['vision'],
});

Voice command:

• Given an audio (16000 hertz, 16 bit), returns the command that most closely matches the user request.
• If none are applicable, returns NONE.

mkcloud.voiceCommand({
  audio: new Blob(),
  commands: ['roll-down', 'roll-up'],
  audioChannelCount: 1,
});

Speech to Text:

• Expects audio in 16000 hertz, 16 bit. Returns text transcript of audio.

const wav = new WaveFile();
wav.fromScratch(1, 16000, '16', [0]);
mkcloud.speechToText({
  audio: new Blob([wav.toBuffer()], { type: 'audio/wav' });
});

Text to Speech:

• Returns audio as a readable stream in 16000 hertz, 16 bit.
• Check docstrings for additional configuration options.

mkcloud.textToSpeech({
  text: 'convert this to audio'
});

VIP:

• Returns a text response for the given VIP configuration.

mkcloud.vip({
  endUserId: randomUUID(),
  npcId: 493,
  text: 'hey'
});

VIP Speech:

• Returns audio as a readable stream in 16000 hertz, 16 bit.
• Check docstrings for additional configuration options.

mkcloud.vipSpeech({
  endUserId: randomUUID(),
  npcId: 493,
  text: 'hey'
});

Testing

• Use node >= 18 for testing or you will run into issues with FormData, Blob, ...
• The following environment variables are expected:
  • CLOUD_SDK_URL
  • CLOUD_SDK_API_KEY

yarn test