Gammaray application server engine

A scalable application server engine.

Motivation

Throughout my history of working on 3D games and later MMO games as a backend developer, I realized that for client development there is a lot of engines out there that basically solve all the common problems for you:

• The whole rendering pipeline with all VSD/HSR techniques
• Different implementations for the rendering API (OpenGL, WebGL, Direct3D, Metal, ...)
• Collision detection/response with physics
• Animations + state machine

and so on. All you have to do is basically just feeding the engine with textures, 3D models and game logic/scripts.

On the backend side however you have to deal with completely different problems:

• Building the API or protocol
• State handling
• Caching
• Scalability
• Transaction handling which is highly depending on the specific DBMS
• Which DBMS to use? Multiple ones?
• How to address/query for an entity/state? Is a SQL query enough? Do we need an extra search index?
• Microservices?
• Which architecture? Domain-Driven-Design?

and so on. You'll come up with very different solutions depending on the specific performance needs. For non-frequent access the solution is stateless where state is always read from the database and uses a SQL database with transactions. For high frequent access (e.g. real time movements of entities in games) you store in memory (stateful) and store it back to a document based DBMS on a lower frequency. Then the requirements change and the SQL solution needs to be high performance as well, and you start asking yourself why you didn't handle all business entities with one generic solution in the first place. Scaling up the SQL database will increase the cost drastically. And when you start dealing with in memory state handling, it opens up a whole new bag of problems:

• Which concurrency model should be used?
• Classical lock based which will end up in deadlocks as soon as complexity increases
• STM which only makes sense when you don't expect too many transactional collisions
• Isolated mutable state, which requires rewriting the whole existing code since state is accessed in a very different way
• Nowadays, your backend isn't just one physical machine: On which node is the desired state located at? On the one the client is connected to? But what if it is shared state?

So what we need is something similar to the idea of a Game/3D-Engine - but that solves most of the backend problems instead.

Setting up the application server with docker

In case you don't have any docker system on your local computer yet, please refer to the official documentation / tutorials at docker.com or use a third party one like Rancher Desktop. For local development and demo purposes you should use the docker image gammarayapp/gammaray-demo. This can either be pulled through the respective UI of your docker system or through the command line: docker pull gammarayapp/gammaray-demo

Starting one node of the application server is already enough to run any application: docker run --name=gammaray1 -p 8080:8080 --rm gammarayapp/gammaray-demo

For starting another node you can just change the name and use the next free port e.g.: docker run --name=gammaray2 -p 8081:8080 --rm gammarayapp/gammaray-demo

Both nodes should find each other automatically and form a cluster. Nodes can be started and stopped randomly. As long as at least one node stays up, the application server cluster is fully functional. Of course, when you run multiple nodes locally, each node has to have a different port. If you shut down the first node with port 8080, all the scripts like for deployment and opening swagger won't work anymore and need to be changed to one of the nodes ports that are up.

The demo version uses an in-memory database internally. So all data will be lost after the last node of a cluster is shut down.

The CLI (gmr)

npm install -g gammaray-cli

Create a new application:

gmr new <name>

This creates an application with just one simple helloworld GET endpoint.

Deploy the application

The package script "build-deploy" does what the name suggests. So just run "npm run build-deploy" on the command line in the root directory of your just created application or use any other tool of your choice for running the package scripts (most IDEs for node.js support that). This deploys the application to the appserver that is configured in "deploy.json". There is also another npm script called "swagger" which opens the OpenAPI definition using Swagger. And there is an all-in-one npm script called "start" which builds, deploys and opens swagger with just one click. The deployment only communicates with one node of the cluster, which is enough. The whole cluster will automatically know about the deployed app.

Versioning

This CLI, the gammaray-app package (that is included in each application) and the application server itself have a specific version using the major.minor.patch pattern e.g. "1.1.0".

• Major: Breaking changes. Existing features and interfaces may have changed. Apps with a different major version of gammaray-app than the application server or this gammaray-cli package might be incompatible to each other.
• Minor: New features, not affecting any other existing feature or interface. When the application server has a higher number that the app, it is ok since the app just simply will not use those features. But when the app has a higher number than the application server, it might use features that the application server doesn't support.
• Patch: Internal fixes, that do not affect the compatibility to other applications.

Each minor version always contains all previous fixes e.g. "1.2.0" contains all the fixes of "1.1.3".

CLI global vs. local dev dependency

At least right now all the scripts are based on the globally installed CLI to reduce confusion. Feel free to install this CLI locally as a dev dependency and make all your scripts use that instead.

Writing an app for the Gammaray application server

A gammaray application has to be implemented using the "gammaray-app" definitions / SDK. Every logic or behaviour needs to be written inside defined functions from this SDK. There is no manual database access, you just address one entity by its id and a defined function will be called were you can read, create or modify the entity. The entity should only be touched inside the scope of this function call. You syntactically can of course do wild things with it e.g. passing it to other SDK functions or doing something with it asynchronously, but there is no guarantee for consistency anymore. After "touching" an entity the respective function can return specific things to indicate what should be done e.g. nothing since you just read something from the entity or other indicators that you modified it, or it is supposed to be deleted. No transactions or locking is necessary, nor any caching or performance optimization - this is all handled by the application server. You are basically just writing the business entities and logic.

Most topics are documented in the example projects, but some general rules to note: The payload of every request/response must be JSON. In case any other data structure needs to be transferred, it can just be wrapped as a string inside a JSON object. Don't use async - besides that this is not even possible, it is not necessary since the whole design and processing under the hood is already happening asynchronously, and you won't do any manual I/O access.

For better understanding here are some example projects: https://github.com/gammarayapp/example-apps