Gymie - Client

WebSocket client that consumes an API wrapping OpenAI Gym or Gym-like environments such as Gym Retro or Unity ML-Agents. Currently the best server is its counterpart Gymie-Server 😉

Content of this document

• Installation
• How to run the client
• API and how to use it
• Testing Gymie
• Licence

Installation

Gymie-Client is available as a NPM package, and can installed as a dependency as usual:

$ npm install gymie

You can also clone the repo and npm-link the library as follows, although there isn't really a good readon to do it this way, unless you wanna contribute to the library and test it locally.

$ git clone https://github.com/jscriptcoder/Gymie-Client
Cloning into 'Gymie-Client'...
...

$ cd Gymie-Client/
$ npm link
[email protected] preinstall /path/to/Gymie-Client
...

$ cd ~/path/to/project
$ npm link gymie
/path/to/project/node_modules/gymie -> /usr/local/lib/node_modules/gymie -> /path/to/Gymie-Client

During the installation Gymie-Server will also be installed. It's important to note that Gymie-Server requires Python>=3.6, so I suggest to conda-create an environment with such version if it's not already installed... or upgrade Python to at least this version.

How to run the client (and server)

Gymie-Client communicates with a server through WebSockets. This server will provide Gymie with an API to access the underlying Python library to create and interact with an environment. As mentioned before, this client comes with its counterpart server. You can start the server from the command line:

$ python -m gymie --host 0.0.0.0 --port 5000
(84581) wsgi starting up on http://0.0.0.0:5000

Once the server is running, Gymie-Client can start interacting with it as follows:

import Gymie from 'gymie'

const gymie = new Gymie()
await gymie.connect('http://0.0.0.0:5000') // connects to the server

const env = await gymie.make('LunarLander-v2') // instantiates an environment

// accesing the underlying Gym-like library.
const space = await env.actionSpace()
const initialState = await env.reset()
const randomAction = await env.actionSample()

API and how to use it

Complete API documentation can be found here: Gymie-Client API (generated by TypeDoc)

In the previous section we already saw how to import gymie, connect to the server, instantiate an environment and call a few API methods. Let's go a bit more in detail with a complete example of a random agent interacting with an environment:

import Gymie  from 'gymie'
import { Continuous, Discrete } from 'gymie/Env'
import { 
  ConnectFailed, 
  NoConnected, 
  ConnectionError, 
  ConnectionClosed 
} from 'gymie/errors'

const wsApi = 'http://0.0.0.0:5000/gym'
const envId = 'LunarLander-v2'
const { log } = console

;(async () => {
  const gymie = new Gymie()

  try {
    // Connects to the server
    await gymie.connect(wsApi)

    // Instantiates the environment, in this case it's got
    // a continuous state and discrete action space.
    const env = await gymie.make<Continuous, Discrete>(envId)
    const space = await env.actionSpace()
    log('Action Space:', space) // => Action Space: { name: 'Discrete', n: number }

    const initialState = await env.reset()
    log('Initial State:\n', initialState) // => Initial State: number[]

    let step = 0
    let totalReward = 0
    
    // Running loop: runs an episode until `done = true`
    log('---- START episode ----')
    while (true) {

      log(`\nStep: ${++step}`)

      // Samples a random action
      const action = await env.actionSample()
      log('Action:', action)  // => Action: number from [0..n)
      
      // Performs a step on the environment given the action
      const [nextState, reward, done, _] = await env.step(action)
  
      log('Next State:\n', nextState) // => Next State: number[]
      log(`Reward: ${reward}`) // => Next State: number
  
      totalReward += reward
  
      if (done) {
        log('---- END episode ----')
        break
      }
    }
  
    log(`\nEpisode Reward: ${totalReward}\n`)
  
    await env.close()

  } catch(err) {
    switch(true) {
      // This could happen while trying to connect to the server
      case err instanceof ConnectFailed: break

      // There is no connection and we try to instantiate an environment    
      case err instanceof NoConnected: break

      // There was a socket error
      case err instanceof ConnectionError: break

      // Server closed the connection. Code and reason comes in the message
      case err instanceof ConnectionClosed: break
    }
  } finally {
    gymie.close()
  }
})()

Testing Gymie

All unit-tests live next to the code they're testing, under the extension src/*.test.ts. You can run all the tests by executing:

$ npm test

License

MIT License - Copyright (c) 2020 Francisco Ramos