This documentation will be in Markdown format and can be used to guide users through the installation, setup, and usage of the project.

Loom Data Visualizer Engine

Overview

Loom is a comprehensive data visualization engine designed to simplify the process of visualizing large datasets using D3.js. It consists of three main components:

1. Loom Library: A powerful npm library for data processing and visualization.
2. Backend: A robust implementation of data engineering that processes data for visualization with D3.js.
3. Frontend: A GitHub Actions-driven implementation using JavaScript, CSS, and HTML, hosted on GitHub for dataset storage and on Netlify for UI.

Additionally, Loom provides downloadable versions of the frontend for various platforms, including macOS (.dmg), Windows (.exe), and Ubuntu (.deb).

Table of Contents

• Installation
  • Loom Library
  • Backend
  • Frontend
• Usage
  • Using the Loom Library
  • Setting Up the Backend
  • Deploying the Frontend
• Self-Hosting
  • Requirements
  • Provisioning
  • One-Click Deployment
• Contributing
• License

Installation

Loom Library

The Loom library can be installed via npm:

npm install loom-data-visualizer

Backend

Clone the backend repository and install the required dependencies:

git clone https://github.com/nturing/loom.git
cd loom
npm install

Usage

Using the Loom Library

Import the Loom library into your project and start using it for data processing and visualization:

const loom = require('loom');

// Example usage
const data = loom.processData(rawData);
loom.visualizeData(data);

Setting Up the Backend

To start the backend server:

node server.js

Configure the backend by editing the config.js file to suit your needs.

Deploying the Frontend

Hosting on Netlify

1. Connect your GitHub repository to Netlify:

   • Log in to Netlify and create a new site.
   • Connect your site to the GitHub repository.

2. Configure build settings:

   • Build command: npm run build
   • Publish directory: dist

3. Deploy the site.

Downloadable Versions

• macOS (.dmg): Download from the release section on GitHub.
• Windows (.exe): Download from the release section on GitHub.
• Ubuntu (.deb): Download from the release section on GitHub.

Self-Hosting

Requirements

• Node.js
• npm
• Git
• Netlify account (for frontend hosting)

Provisioning

1. Clone the repositories:

   git clone https://github.com/nturing/loom.git

2. Install dependencies:

   cd backend
   npm install
   cd ../frontend
   npm install

3. Configure and start the backend:

   cd backend
   node server.js

4. Deploy the frontend on Netlify or serve it locally:

   cd frontend
   npm run build

   Host the dist directory on your preferred web server.

One-Click Deployment

AWS

Click the button below to deploy to AWS:

Azure

Click the button below to deploy to Azure:

DigitalOcean

Click the button below to deploy to DigitalOcean:

Google Cloud Platform

Click the button below to deploy to Google Cloud Platform:

Datasets

Contributing

We welcome contributions from the community! Please read our contributing guidelines for more information on how to get started.

References

Vicgalle

A huge thank you to Victor Gallego https://huggingface.co/vicgalle for creating and sharing the Alpaca GPT-4 Dataset. Your contributions to the open-source community are invaluable and greatly appreciated.

Gutenberg

Special thanks to Project Gutenberg for their incredible Gutenberg Dataset https://raw.githubusercontent.com/nturing/loom/main/data/bow/gutenberg_metadata.csv. Your efforts in making literature freely accessible to everyone are truly commendable.

License

Loom is licensed under the Apache-2.0.

This documentation provides a clear and concise guide for users to install, set up, and use the Loom data visualizer engine. By following these steps, users can provision the software themselves and leverage its powerful data visualization capabilities.