@project-sunbird/ext-framework-server
v8.0.0
Published
Extensible framework for sunbird extensions on server side
Downloads
46
Readme
Overview
simple, fast, effective way to build pluggable modules using node.js. It can be integrated to the main application as simple as npm modules. You can develop/test/publish modules independently irrespective of the main application.
Sections:
- Setup
- Getting started
- Plugin lifecycle
- Debugging
- Testing
- Publishing plugins
- npm link vs npm install for local modules
- Contributors
Setup
- Download
Apache Cassandra
v3.7 or higher - Download
Elasticsearch
v6.1 - Download
Node.Js
v6.4 or higher - install nodemon globally:
npm i nodemon -g
(optional)
Getting started
To run the demo app with default plugins. Follow the instructions.
- Clone the repo.
- Start the
Cassandra
andElasticsearch
server. - cd
ext-framework/server
and runnpm install
. - create framework build by running
npm run build
, - go to
demo
>app
folder (app) and runnpm install
. - link
ext-framework
library to the demo app. go todemo
>app
and runnpm link ../../server/dist
// Plugins
- go to plugins folder
demo/plugins/hello-world/server
and runnpm install
- run
npm i @project-sunbird/ext-framework-server
. - create build of the plugin by running
npm run build
// link each plugin to demo app
NOTE:
demo/app/index.ts
has Framework configuration to decide which plugin to load from the app. Change this file accordingly to load the plugins
cd demo/app
and runnpm link ../plugins/hello-world/server/dist
- And run
npm run start
. Demo application starts by loading "hello-world" plugin
Writing your first plugin:
In this tutorial, we are going to write simple API endpoint which will respond as "hello world" when we send request. Along this tutorial we will learn about folder structure and conventions to write plugin. Later we will write complex plugins using cassandra
and Elasticsearch
Database.
I believe the setup is ready as instructed above.
- Now go to folder
demo/plugins
and create a new folder called "hello-world" - create a new folder inside
./helo-world
calledserver
- cd
./hello-world/server
& runnpm install typescript --save-dev
. copy thebuild
script command from the below code snippet and add it topackage.json
or
copy the below file and paste it in package.json
and run npm install
{
"name": "hello-world",
"version": "1.0.0",
"description": "",
"main": "dist/index.js",
"types": "dist/index.d.ts",
"scripts": {
"build": "rm -rf ./dist && tsc --noImplicitUseStrict && cp package.json ./dist",
"test": "echo 'go to '../test' folder to execute test cases!'"
},
"dependencies": {
},
"devDependencies": {
"typescript": "^2.7.1"
},
"keywords": [],
"author": "",
"license": "ISC"
}
- since, we are going to write our code in typescript, we need
tsconfig.json
file to configure typescript compiler.
create tsconfig.json file inside current folder and paste the below code:
{
"compilerOptions": {
"module": "commonjs",
"declaration": true,
"outDir": "./dist",
"target": "es6"
},
"exclude": [
"node_modules",
"./**/*.spec.ts"
]
}
- Lets link framework library to plugin using npm link, run
npm link <path-from-root>/ext-framework/server/dist
. - create
server.ts
inside./hello-world/server
copy paste the below code:
export class Server {
public sayHello(req, res, next) {
res.status(200)
.send('Hello world')
}
}
here, we have created a class called Server
and method called sayHello()
which is a express.js
middleware function, which returns 'hello world' when we send the request. This file is called Entry Point
file of the plugin. The class name should be kept as Server
and should be exported as above.
- create an another file inside
./hello-world/server
calledroutes.ts
to mention the routes for our plugin
copy paste the below code:
import { Manifest } from 'ext-framework-server/models/Manifest';
import { frameworkAPI } from 'ext-framework-server/api';
export class Router {
init(app: any, manifest: Manifest, auth?: any) {
const server = frameworkAPI.getPluginInstance(manifest.id);
app.get('/get', (req, res, next) => { server.sayHello(req, res, next) })
}
}
Here, we have imported 'Framework' which is an API of the extensible framework and 'Manifest' is just a type.
Class name should be kept as Router
and it should have a init
method which takes 3 parameters:
- app: express app instance
- manifest:
manifest
of the plugin, explained in below section - auth (optional): auth module instance to authenticate the request.
Inside init method, we will get the plugin instance created by the framework (when it instantiates the plugin during plugin load phase) by passing manifest.id
which is plugin Id and we are calling sayHello()
method when we get the request from the app.
*. Lets create manifest.ts
file inside ./hello-world/server/
which looks like this:
export const manifest = {
"id": "hello-world",
"name": "simple hello world plugin",
"author": "sunil<[email protected]>",
"version": "1.0",
"server": {
"routes": {
"prefix": "/hello"
}
}
}
Manifest should have a unique id, name of the plugin, author name and version. It has server section where we can define prefix for all the endpoint defined in our plugin. Here, we have defined the prefix as /hello
so, our final endpoint for "GET" request would look like GET /hello/get
.
- Lets build the plugin, run
npm run build
. - check the above command has created a
./dist
folder. - Folder structure of our plugin will look like this:
./hello-world
|- server
|-dist
|-node_modules
|-manifest.ts
|-server.ts
|-router.ts
|-package.json
|-tsconfig.json
Lets Intergrate plugin to the app:
Lets go to ext-framework/demo/app/index.ts
where we keep our framework configuration file. It tells framework to load list of plugins declared and other config settings related to server. We have add an entry of our new plugin hello-world
and link our plugin build to the app.
- cd
ext-framework/demo/app
folder and runnpm link <path-from-root>/ext-framework/demo/plugins/hello-world/server/dist
- run
npm link <path-from-root>/ext-framework/server/dist
from the current folder
Now we have linked framework module and our plugin module to the app.
- lets start our app , run
npm run start
it should show the logs like this:
[2018-07-06T16:46:36.140] [INFO] default - loading schema for plugin: core
[2018-07-06T16:46:36.293] [INFO] default - no Cassandra schema change detected for plugin "core"!
[2018-07-06T16:46:36.294] [INFO] default - loading registry schema
[2018-07-06T16:46:36.294] [INFO] default - Framework is initialized!
[2018-07-06T16:46:36.295] [INFO] default - --------loding-plugin-hello-world-------
[2018-07-06T16:46:36.481] [INFO] default - Plugin "hello-world" is registered!
[2018-07-06T16:46:36.640] [INFO] default - --------load-complete-hello-world-------
[2018-07-06T16:46:36.640] [INFO] default - All plugins are loaded!
=====> Application running on port: 4000
- go to browser
http://localhost:4000/hello/get
and hit enter. You should see the response from our plugin as "hello world"
Plugin Lifecycle:
It is important to understand the lifecycle of a plugin to understand deeper workings of framework internals. There are 4 stages:
- Loading
- Activation
- Running
- Closing
1. Loading:
During this phase, the framework tries to read the manifest.ts
file under plugin's home directory. If the manifest.ts
file is not found, the framework fails to load the plugin. When it finds the manifest.ts
file, it will register the plugin in "plugin registry" and update the status of the plugin as REGISTERED
.
When the plugin has dependencies on other plugins, the framework would load all the dependencies mentioned in the manifest.ts
file (in dependencies
section) before loading the actual plugin. when any one of its dependencies are not met or causes an error during loading, it would not load the actual plugin and it wouldn't add an entry into "plugin registry".
Once a plugin is in REGISTERED
state, the framework tries to locate if any schema files are defined in the manifest.ts
file. If the plugin has not defined any schema file, the framework would skip this step. If there are schema files, then it would try to create a schema(tables/index) on the corresponding database provided based on the schema definition. If the schema is already defined in the database, it would not try to re-create the same, keeping it as is.
Once the schema is set in the database it cannot be changed simply by changing the schema definition json and reloading the plugin again. In order to change the already existing schema, the user needs to migrate the old schema to new schema (currenlty, we have to do this manually) and change the schema definition json in the plugin and reload the plugin.
server.ts
is the entry file of a plugin. This file should export class named Server
. Framework loads this class and creates a new instance of it. It passes plugin manifest (from manifest.ts
) object to the plugin through its constructor. The framework holds this runtime instance through out the lifecycle. And it is available to the plugin to access it.
The framework tries to find routes.ts
file under the plugin home directory. If the file is not found, the plugin fails to load. The file should export class named Router
. The framework registers the routes(end point) defined for the plugin with the "prefix" defined in the manifest.ts
file.
To sum it up:
These are the list of task framwork does when loading the plugin:
- Register plugin
- Resolve dependencies
- Create database schema (if any)
- Create runtime plugin instance
- Register plugin routes
2. Activation:
The Framework activates the plugin when it finish loading the list of plugin defined in the configuration file. Before that framework creates an instance (runtime Js Object) of the plugin when it successfully loads. The plugin instances is handled by the framework inorder to communicate with the plugin about its lifecycle events. The plugin should have lifecycle hook methods inorder to take actions when it is called from the framework. E.g when the framework is shuting down, it should inform all the plugins through the lifecycle method to indicate the action so that plugin can perform some task before it gets killed.
Currently there are only 2 lifecycle methods which plugin has to implement:
onStart()
: To indicate plugin is active and available to serve the request.onStop()
: Plugin is being stopped due to maintanence or some other reason and it is no longer available to serve the request.
3. Running:
During this stage, plugin is available to serve the request. Framework doesn't have any control over the functional working of the plugin. Web request specific to the plugin is directed to plugin middleware to handle the request.
4. Closing:
Currently we don't have a way to bring down the individual plugins for maintainence when it is loaded along with other plugins. when we try to terminate the framework, it informs about the event to all the registered plugins through lifecycle hook methods.
Debugging:
Demo app runs in debug
mode enabled. Latest node.js
supports --inspect
flag to run the app in debug mode. To debug the app, start the demo app and open the chrome
developer console and search for node.js
icon next to Elements
tab and click on it. It opens devTool for node.js in a new window.
Testing:
cd server
and run npm run test
. It generates HTML
reporter inside folder server/mochawesome-report
using mochawesome
reporter.
Publishing plugins:
Plugins can be plublished as npm modules.
- Create plugin build (
/dist
folder) usingnpm run build
command. - Add version, description, name of plugin, license in
package.json
. - Copy
package.json
,readme.md
file to thedist
folder. - Login to npm using CLI.
- Publish the package to NPM.
Generate Code Document:
- run
npm run gen-doc
(generatesdocs
folder in root directory)
npm link
vs npm install
for local modules:
To save development time, we have to link the local dependencies such as plugins and framework libraries rather than installing it as a dependency by declaring it in the package.json
. If we declare local dependencies in package.json
, it will install only once and when we make any changes to local modules, we need to re-install the dependencies again by npm install
.
npm link
creates symbolic link to actual module. Any changes to source module is immediately reflected and saves our time.
However, this is not the case when publishing the library. Each plugin should have peerDependencies
section where it should declare ext-framework
as its dependency with its version no.
Note to Contributors:
This project was built using Visual Source Code. Contributors are recommended to use this editor and install the "recommended" extensions from the VScode Extensions
.