@idio/idio

v1.6.2

Published

3 years ago

2-Dependency Koa Web Server Compiled With Closure Compiler And Bundled With Essential Middleware: Static, Compress, Session, Cors, File Upload, Router And JSX Front End.

Downloads

180

0High
0Medium
0Low

zvr

idio web server middleware session static cors compress goa koa

@idio/idio

@idio/idio contains Koa's fork called Goa — web server compiled with Closure Compiler so that its source code is optimised and contains only 1 external dependency (mime-db). Idio adds essential middleware to Goa for session, static files, CORS and compression and includes the router. As the project grows, more middleware will be added and optimised.

This is a production-ready server that puts all components together for the ease of use, while providing great developer experience using JSDoc annotations for auto-completions. Idio is not a framework, but a library that enables idiomatic usage and compilation of the server and middleware.

idio~:$ \
yarn add @idio/idio
npm install @idio/idio

Example Apps

There are some example apps that you can look at.

File Upload: a front-end + back-end application for uploading photos. Demo requires GitHub authorisation without any scope permissions to enable session middleware showcase.
Akashic.Page: a service for managing email and web-push subscriptions, with JS widgets and Mongo database connection.

API

The package is available by importing its default function and named components:

import idio, { Keygrip, Router } from '@idio/idio'

async idio( `middlewareConfig=: !MiddlewareConfig,` `config=: !Config,`): !Idio

Start the server. Sets the proxy property to true when the NODE_ENV is equal to production.

middlewareConfig !MiddlewareConfig (optional): The middleware configuration for the idio server.
config !Config (optional): The server configuration object.

The app can be stopped with an async .destroy method implemented on it that closes all connections.

There are multiple items for middleware configuration:

MiddlewareConfig extends FnMiddlewareConfig: Middleware configuration for the idio server.

| Name | Type | Description | | ---------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | static | (!StaticOptions | !Array<!StaticOptions>) | Static middleware options. | | compress | (boolean | !CompressOptions) | Compression middleware options. | | session | !SessionOptions | Session middleware options. | | cors | !CorsOptions | CORS middleware options. | | form | !FormDataOptions | Form Data middleware options for receiving file uploads and form submissions. | | frontend | !FrontEndOptions | Front End middleware allows to serve source code from node_modules and transpile JSX. | | neoluddite | !NeoLudditeOptions | Records the usage of middleware to compensate their developers' intellectual work. | | csrfCheck | !CsrfCheckOptions | Enables the check for the presence of session with csrf property, and whether it matches the token from either ctx.request.body or ctx.query. | | github | (!GitHubOptions | !Array<!GitHubOptions>) | Sets up a route for GitHub OAuth authentication. The returned middleware will be installed on the app automatically so it doesn't need to be passed to the router. | | jsonErrors | (boolean | !JSONErrorsOptions | !Array<!JSONErrorsOptions>) | Tries all downstream middleware, and if an error was caught, serves a JSON response with error and stack properties (only if exposeStack is set to true). Client errors with status code 4xx (or that start with !) will have full message, but server errors with status code 5xx will only be served as { error: 'internal server error '} and the app will emit an error via app.emit('error') so that it's logged. | | jsonBody | (boolean | !JSONBodyOptions) | Allows to parse incoming JSON request and store the result in ctx.request.body. Throws 400 when the request cannot be parsed. | | logarithm | !LogarithmOptions | Options to record hits in ElasticSearch. |

The types for starting the server include the address, port and router configuration.

Config: Server configuration object.

| Name | Type | Description | Default | | ------ | ----------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------- | --------- | | port | number | The port on which to start the server. | 5000 | | host | string | The host on which to listen. | 0.0.0.0 | | router | !_goa.RouterConfig | The configuration for the router. | - |

After the app is started, it can be accessed from the return type.

Idio: The return type of the idio.

| Name | Type | Description | | --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | url* | string | The URL on which the server was started, such as http://localhost:5000. | | server* | !http.Server | The server instance. | | app* | !Application | The Goa application instance (with additional .destroy method). | | middleware* | !ConfiguredMiddleware | An object with configured middleware functions, which can be installed manually using app.use, or router.use. The context will be a standard Goa context with certain properties set by bundled middleware such as .session. | | router* | !Router | The router instance. |

All middleware can be accessed from the middleware property, so that it can be installed on individual basis on specific routes, if it's not used app-wise.

The example below starts a simple server with session and custom middleware, which is installed (used) automatically because it's defined as a function.

const { url, app,
  middleware: { session, form },
  router,
} = await idio({
  // Developers' payment scheme neoluddite.dev
  neoluddite: {
    env: process.env.NODE_ENV,
    key: '0799b7f0-d2c7-4903-a531-00c8092c2911',
    app: 'idio.example',
  },
  // Idio's bundled middleware.
  session: {
    algorithm: 'sha512',
    keys: ['hello', 'world'],
    prefix: 'example-',
  },
  static: {
    use: true,
    root: 'upload',
  },
  form: {
    config: {
      dest: 'upload',
    },
  },
  // Any middleware function to be use app-wise.
  async middleware(ctx, next) {
    console.log('//', ctx.method, ctx.path)
    await next()
  },
})
app.use(router.routes())
router.get('/', session, (ctx) => {
  ctx.body = 'hello world'
})
router.post('/upload', session, async (ctx, next) => {
  if (!ctx.session.user) {
    ctx.status = 403
    ctx.body = 'you must sign in to upload'
    return
  }
  await next()
}, form.single('/upload'), (ctx) => {
  // db.create({
  //  user: ctx.session.id,
  //  file: ctx.req.file.path,
  // })
  ctx.body = 'Thanks for the upload. Link: ' +
    `${url}/${ctx.file.filename}`
})

http://localhost:5000
// GET /
hello world

Middleware

Idio's advantage is that is has the essential middleware, that was compiled together with the server, so that the packages are reused and memory footprint is low.

Static

🗂 Explore Static Middleware Configuration

Used to serve static files, such as stylesheets, images, videos, html and everything else. Will perform mime-type lookup to serve the correct content-type in the returned header.

const { url, app } = await idio({
  static: {
    root: 'example', use: true,
  },

// or multiple locations
  static: [{
    root: ['example'], use: true,
  }, {
    root: ['wiki'], use: true,
  }],
}, { port: null })

/** http://localhost:57537/app.css */ 

body {
  font-size: larger;
}

Content-Length: 29
Last-Modified: Thu, 18 Jul 2019 14:34:31 GMT
Cache-Control: max-age=0
Content-Type: text/css; charset=utf-8
Date: Thu, 05 Mar 2020 13:30:57 GMT
Connection: close

Content-Length: 114
Last-Modified: Sat, 28 Dec 2019 18:07:31 GMT
Cache-Control: max-age=0
Content-Type: image/svg+xml
Date: Thu, 05 Mar 2020 13:30:59 GMT
Connection: close

Session

👳‍♂️Explore Session Middleware Configuration

Allows to store data in the .session property of the context. The session is serialised and placed in cookies. When the request contains the cookie, the session will be restored and validated (if signed) against the key.

const { url, app } = await idio({
  session: { use: true, keys:
    ['hello', 'world'], algorithm: 'sha512' },
  async middleware(ctx, next) {
    if (ctx.session.user)
      ctx.body = 'welcome back '
        + ctx.session.user
    else {
      ctx.session.user = 'u'
        + (Math.random() * 1000).toFixed(1)
      ctx.body = 'hello new user'
    }
    await next()
  },
})

// GET /
"hello new user"
/* set-cookie */
[
  {
    name: 'koa:sess',
    value: 'eyJ1c2VyIjoidTg2LjciLCJfZXhwaXJlIjoxNTgzNTAxNDU5ODQzLCJfbWF4QWdlIjo4NjQwMDAwMH0=',
    path: '/',
    expires: 'Fri, 06 Mar 2020 13:30:59 GMT',
    httponly: true
  },
  {
    name: 'koa:sess.sig',
    value: '5hRueSOyLuhp6nZvOi4TcziXNiADlaIhE6fJHruR-I8cGtEVDYCgNe9t3LS0SyV-SEN1kPa8ZwIz-a91GWPw-A',
    path: '/',
    expires: 'Fri, 06 Mar 2020 13:30:59 GMT',
    httponly: true
  }
]
// GET /
"welcome back u86.7"

CORS

👮‍♀️Explore CORS Middleware Configuration

To enable dynamic communication between clients and the server via JavaScript requests from the browser, the server must respond with Access-Control-Allow-Origin header that sets the appropriate allowed Origin. This middleware is easy to use on production and development environments.

const { NODE_ENV } = process.env

const { url, app } = await idio({
  async example(ctx, next) {
    console.log('//', ctx.method,
      ctx.path, 'from', ctx.get('Origin'))

    ctx.body = 'hello world'
    await next()
  },
  cors: {
    use: true,
    origin: NODE_ENV == 'production' ?
      'http://prod.com' : null,
    allowMethods: ['GET', 'POST'],
  },
})

// GET / from https://3rd.party
{
  vary: 'Origin',
  'access-control-allow-origin': 'http://prod.com',
  date: 'Thu, 05 Mar 2020 13:31:01 GMT',
  connection: 'close'
}

// GET / from http://prod.com
{
  vary: 'Origin',
  'access-control-allow-origin': 'http://prod.com',
  date: 'Thu, 05 Mar 2020 13:31:01 GMT',
  connection: 'close'
}

// OPTIONS / from http://prod.com
{
  vary: 'Origin',
  'access-control-allow-origin': 'http://prod.com',
  'access-control-allow-methods': 'GET,POST',
  date: 'Thu, 05 Mar 2020 13:31:01 GMT',
  connection: 'close'
}

Compression

🗜Explore Compression Middleware Configuration

When the body of the response is non-empty, it can be compressed using gzip algorithm. This allows to save data transmitted over the network. The default threshold is 1024 bytes, since below that the benefits of compression are lost as the compressed response might end up being even larger.

const { url, app } = await idio({
  async serve(ctx, next) {
    console.log('//',
      ctx.method, ctx.path)

    ctx.body = packageJson
    await next()
  },
  compress: {
    use: true,
  },
})

// GET /
{
  'content-type': 'application/json; charset=utf-8',
  vary: 'Accept-Encoding',
  'content-encoding': 'gzip',
  date: 'Thu, 05 Mar 2020 13:31:01 GMT',
  connection: 'close',
  'transfer-encoding': 'chunked'
}

File Upload

🖼Explore Form Data Middleware Configuration

Browser will submit forms and send files using multipart/form-data type of request. It will put all fields of the form together and stream them to the server, sending pairs of keys/values as well as files when they were attached. The Form Data middleware is the Multer middleware specifically rewritten for Koa that can handle file uploads.

const { url, app, router, middleware: {
  form,
} } = await idio({
  form: {
    dest: 'example/upload',
  },
})
app.use(router.routes())
router.post('/example',
  form.single('bio'),
  (ctx) => {
    delete ctx.file.stream
    ctx.body = { file: ctx.file,
      body: ctx.request.body }
  }
)

{
  file: {
    fieldname: 'bio',
    originalname: 'bio.txt',
    encoding: '7bit',
    mimetype: 'application/octet-stream',
    destination: 'example/upload',
    filename: '106e5',
    path: 'example/upload/106e5',
    size: 29
  },
  body: { hello: 'world' }
}

Front End

🌐Explore Front End Middleware Configuration

Web applications are always full stack and involve both back-end together with front-end. Whereas all previously described middleware was for the server only, the front-end middleware facilitates browser development, as it allows to serve source code from the node_modules directory and transpile JSX. Modern browsers support modules, but JavaScript needs to be patched to rename imports like

// was
import X from 'package-name'
// becomes
import X from '/node_modules/package-name/src/index.mjs'

This is achieved by resolving the module field from package.json of served packages (with fallback to the main field, but in that case require statements will not work).

const { url, app } = await idio({
  frontend: {
    use: true,
    directory: 'example/frontend',
  },
})

import { render, Component } from 'preact'

class MyComp extends Component {
  render() {
    return (<div className="example">
      Hello World!
    </div>)
  }
}

render(MyComp, document.body)

import { h } from '/node_modules/preact/dist/preact.mjs'
import { render, Component } from '/node_modules/preact/dist/preact.mjs'

class MyComp extends Component {
  render() {
    return (h('div',{className:"example"},
      `Hello World!`
    ))
  }
}

render(MyComp, document.body)

The idea here is to provide a basic mechanism to serve front-end JavaScript code, without inventing any module systems, adapting to CommonJS, or transpiling old features. We simply want to execute our modern code and browsers are more than capable to do that, without us having to run complex build systems on the development code. Our simple JSX parser is not rocket science either and works perfectly well without building ASTs (but check for minor limitations in Wiki).

Additional Middleware

There are some small bits of middleware that can be used in server as well, but which are not essential to its functioning. They are listed in 📖 Wiki.

csrfCheck: Ensures that the csrf token from session matches one in the request.
jsonErrors: Allows to serve errors as JSON, which is useful for APIs.
jsonBody: Parses requests with the application/json content type into ctx.request.body.
logarithm: Record hits in ElasticSearch.
github: Sets up GitHub OAuth routes.

Custom Middleware

When required to add any other middleware in the application not included in the Idio bundle, it can be done in several ways.

Passing the middleware function as part of the MiddlewareConfig. It will be automatically installed to be used by the Application. All middleware will be installed in order it is found in the MiddlewareConfig.

import idio from '@idio/idio'
   
const APIServer = async (port) => {
  const { url } = await idio({
    // 1. Add logging middleware.
    async log(ctx, next) {
      await next()
      console.log(' --> API: %s %s %s', ctx.method, ctx.url, ctx.status)
    },
    // 2. Add always used error middleware.
    async error(ctx, next) {
      try {
        await next()
      } catch (err) {
        ctx.status = 403
        ctx.body = err.message
      }
    },
    // 3. Add validation middleware.
    async validateKey(ctx, next) {
      if (ctx.query.key !== 'app-secret')
        throw new Error('Wrong API key.')
      ctx.body = 'ok'
      await next()
    },
  }, { port })
  return url
}
   
export default APIServer

Started API server at: http://localhost:5005
 --> API: GET / 403
 --> API: GET /?key=app-secret 200

Passing a configuration object as part of the MiddlewareConfig that includes the middlewareConstructor property which will receive the reference to the app. Other properties such as conf and use will be used in the same way as when setting up bundled middleware: setting use to true will result in the middleware being used for every request, and the config will be passed to the constructor.

import rqt from 'rqt'
import idio from '@idio/idio'
import APIServer from './api-server'
   
const ProxyServer = async (port) => {
  // 1. Start the API server.
  const API = await APIServer(5001)
  console.log('API server started at %s', API)
   
  // 2. Start a proxy server to the API.
  const { url } = await idio({
    async log(ctx, next) {
      await next()
      console.log(' --> Proxy: %s %s %s', ctx.method, ctx.url, ctx.status)
    },
    api: {
      use: true,
      async middlewareConstructor(app, config) {
        // e.g., read from a virtual network
        app.context.SECRET = await Promise.resolve('app-secret')
   
        /** @type {import('@typedefs/goa').Middleware} */
        const fn = async (ctx, next) => {
          const { path } = ctx
          const res = await rqt(`${config.API}${path}?key=${ctx.SECRET}`)
          ctx.body = res
          await next()
        }
        return fn
      },
      config: {
        API,
      },
    },
  }, { port })
  return url
}

API server started at http://localhost:5001
Proxy started at http://localhost:5002
 --> API: GET /?key=app-secret 200
 --> Proxy: GET / 200

Router Set-up

After the Application and Router instances are obtained after starting the server as the app and router properties of the returned object, the router can be configured to respond to custom paths. This can be done by assigning configured middleware from the map and standalone middleware, and calling the use method on the Application instance.

import { collect } from 'catchment'
import idio from '@idio/idio'

const Server = async () => {
  const {
    url, router, app, middleware: { pre, post, bodyparser },
  } = await idio({
    // 1. Configure middlewares via middlewareConstructor without installing them.
    pre: {
      middlewareConstructor() {
        return async function(ctx, next) {
          console.log('  <-- %s %s',
            ctx.request.method,
            ctx.request.path,
          )
          await next()
        }
      },
    },
    post: {
      middlewareConstructor() {
        return async function(ctx, next) {
          console.log('  --> %s %s %s',
            ctx.request.method,
            ctx.request.path,
            ctx.response.status,
          )
          await next()
        }
      },
    },
    bodyparser: {
      middlewareConstructor() {
        return async (ctx, next) => {
          let body = await collect(ctx.req)
          if (ctx.is('application/json')) {
            body = JSON.parse(body)
          }
          ctx.request.body = body
          await next()
        }
      },
    },
  }, { port: 5003 })

  // 2. Setup router with the bodyparser and path-specific middleware.
  router.post('/example',
    pre,
    bodyparser,
    async (ctx, next) => {
      ctx.body = {
        ok: true,
        request: ctx.request.body,
      }
      await next()
    },
    post,
  )
  app.use(router.routes())
  return url
}

Page available at: http://localhost:5003
  <-- POST /example
  --> POST /example 200

// server response:
{ ok: true, request: { hello: 'world' } }

Also checkout the Router package that allows to automatically initialise routes from a given directory, and watch for changes in them during development. This means you don't have to refresh the server manually after a change to a route.

const w = await initRoutes(router, 'routes', {
  middleware,
})
if (process.env.NODE_ENV == 'prod') watchRoutes(w)

SSR

Idio supports Server-Side rendering of JSX components (same restrictions apply as for front-end). You can easily mark up your back-end pages using full-scale HTML, or basic placeholders in which you can then render your front-end app.

import idio, { render } from '@idio/idio'

const { url, app, router } = await idio()
router.get('/', (ctx) => {
  ctx.body = render(<html>
    <head>
      <title>Example</title>
    </head>
    <body>
      Hello World!
    </body>
  </html>, {
    addDoctype: true,
    pretty: true,
  })
})
app.use(router.routes())

<!doctype html>
<html>
  <head><title>Example</title></head>
  <body>Hello World!</body>
</html>

NeoLuddite.Dev

This web server integrates with NeoLuddite: the package monetary reward scheme. It's currently in beta, and this section will be relevant when it's open to the public.

Every time you invoke certain functionality in a package somebody has written (e.g., koa-static for static files, koa-session for creation of session), via Idio, your usage will be counted and your balance in Ludds on the neoluddite server will be transferred to the software engineer as a reward for his/her intellectual work. Contact [email protected] for any requests.

const { url, app,
  middleware: { session, form },
  router,
} = await idio({
  // Developers' payment scheme neoluddite.dev
  neoluddite: {
    env: process.env.NODE_ENV,
    key: '0799b7f0-d2c7-4903-a541-10d8092c2911',
    app: 'idio.example',
  },
  // ...
}

The usage will be billed for apps running in production mode, therefore the env variable is needed. Setting the app has no effect but allows to break down statistics by web application on the portal. See the license section for more info.

NeoLudditeOptions: Options for the neoluddite.dev client.

| Name | Type | Description | Default | | -------- | --------------- | -------------------------------------------------------------------------------------------------------- | ------------------------ | | key* | string | The API key received from the app. | - | | env | string | The environment (e.g., dev/staging). The production env must be indicated as prod which is billed. | - | | host | string | The hostname of the server. | https://neoluddite.dev | | app | string | The name of the application. | - |

WebSocket

We've implemented a library to upgrade requests into WebSocket connections. You can read more at the actual package page. Idio simply exports this method via its API. You need to configure it yourself.

import idio, { websocket } from '@idio/idio'

const { url, app, server } = await idio()
// clients stores current connections against ID
const clients = websocket(server, {
  onConnect(clientId) {
    // the value is a function to send messages
    clients[clientId]('intro-event', 'Hello Client!')
  },
})

Copyright & License

GNU Affero General Public License v3.0

Affero GPL means that you're not allowed to use this web server on the web unless you release the source code for your application. This is a restrictive license which has the purpose of defending Open Source work and its creators.

To be able to use the server, just set up a monthly payment on Open Collective for any amount of your choice.

All original work on middleware and Koa are under MIT license. See Goa Page for the list of packages and modules used in compilation of the Goa server, and the package.json file for dependencies of this project (todo: create wiki page w/ licenses table).

Published

Vulnerabilities

Links

Maintainers

Keywords

Readme

@idio/idio

Example Apps

Table Of Contents

API

async idio( middlewareConfig=: !MiddlewareConfig, config=: !Config,): !Idio

Middleware

Static

Session

CORS

Compression

File Upload

Front End

Additional Middleware

Custom Middleware

Router Set-up

SSR

NeoLuddite.Dev

WebSocket

Copyright & License

async idio( `middlewareConfig=: !MiddlewareConfig,` `config=: !Config,`): !Idio