kazana

A modular data warehouse system

Kazana is a data warehouse system framework. It provides high level APIs to extract, transform and index data for analysis.

1. Quick start
2. Core philosophies
3. Features
4. How it works

See also: Glossary

Quick start

Create a folder and create a package.json

mkdir myapp
cd myapp
npm init --yes

Add kazana as dependency

npm install --save kazana

Add a "start" script to the package.json

{
  "name": "myapp",
  "scripts": {
    "start": "kazana"
  },
  "dependencies": {
    "kazana": "^8.4.0"
  }
}

Create myapp/index.js with the following content:

module.exports = {
  name: 'myapp',
  version: '1.0.0'
}

In future, that will be enough, but for now the public option must be set, see https://github.com/eHealthAfrica/kazana/issues/195.

So add a public option like this:

module.exports = {
  name: 'myapp',
  version: '1.0.0',
  public: 'public'
}

And create a public/index.html file with this content.

Now start the app

npm start

You need to create a test user at http://localhost:4999/_utils/#/database/_users/new

with this content

{
    "_id": "org.couchdb.user:[email protected]",
    "type": "user",
    "name": "[email protected]",
    "roles": [
        "kazana-id:testadmin",
        "kazana-data-provider",
        "kazana-data-reviewer",
        "kazana-analyst",
        "kazana-admin"
    ],
    "password": "secret",
    "createdAt": "2016-01-01T00:00:00.000Z",
    "fullname": "Test Admin"
}

Now start the app

npm start

Once running, you can open http://localhost:5000. It will show a simple html app that lets you submit data.

Congratulations, you have built your first Kazana app and can already start accepting data submissions!

What’s next?

• Create a custom html app with user authentication
• Transform raw data reports into indicators
• Indices
• Bootstrap users & data
• build a plugin

Core philosophies

1. Quick deployments

   Kazana accepts all kind of data out of the box through its generic REST API and client SDKs.

2. Iterative development

   Transform scripts are not required to accept raw data reports, they simply start integrating existing raw data when they are ready.

3. Tiny core, many plugins

   Instead of a monolithic architecture, Kazana is focussed on a tiny core with great code quality and stability. Context-specific features can be added with plugins.

Features

1. Flexible Raw Data Reports handling Zero-setup, secure raw data warehouse
2. Asynchronous Data Transformation
   Integrate existing and new raw data
3. Indices map & reduce integrated data for analysis
4. Entities
   e.g. involved agencies, locations, projects, etc
5. Custom routes
   e.g. for hook APIs from 3rd party services
6. Events✸
   e.g. send notifications when new data submitted
7. Auditing✸
   Get notified if data is missing
8. Services
   e.g. to constantly pull data from 3rd party services)
9. Plugins
   (reusable) modules with all extension points above
10. User authentication & permissions✸
    Built-in user management and role-based permissions
11. Client APIs with Offline Support✸
    Capture data while offline, sync later

✸ Work in progress

How it works

Kazana is a server built on top of the Hapi framework and uses CouchDB as its data store✸.

A Kazana app is defined by a Kazana Manifest. It’s the app’s main file (index.js by default) and exports an object with pre-defined properties. A minimal manifest looks like this:

module.exports = {
  name: 'myapp'
}

Starting the app will serve a generic app for data submissions at http://localhost:5000. (TODO: https://github.com/eHealthAfrica/kazana/issues/195)

✸ Kazana spawns a PouchDB Server as a CouchDB drop-in replacement by default, unless otherwise configured, so CouchDB is not a hard requirement for development, but recommended for production use.

• System Architecture
• Data Lifecycle

Local setup and Testing

git clone [email protected]:eHealthAfrica/kazana.git
cd kazana
npm install
npm test

CI test with selenium / chrome

npm run test:ci

CI test with Saucelabs

SAUCE_USERNAME=*** SAUCE_ACCESS_KEY=*** TEST_CLIENT="saucelabs:internet explorer:10:Windows 8" npm run test:ci

Credit

Brought to you by eHealth Africa — good tech for hard places.

License

Apache-2.0