npm package discovery and stats viewer.

Discover Tips

  • General search

    [free text search, go nuts!]

  • Package details

    pkg:[package-name]

  • User packages

    @[username]

Sponsor

Optimize Toolset

I’ve always been into building performant and accessible sites, but lately I’ve been taking it extremely seriously. So much so that I’ve been building a tool to help me optimize and monitor the sites that I build to make sure that I’m making an attempt to offer the best experience to those who visit them. If you’re into performant, accessible and SEO friendly sites, you might like it too! You can check it out at Optimize Toolset.

About

Hi, 👋, I’m Ryan Hefner  and I built this site for me, and you! The goal of this site was to provide an easy way for me to check the stats on my npm packages, both for prioritizing issues and updates, and to give me a little kick in the pants to keep up on stuff.

As I was building it, I realized that I was actually using the tool to build the tool, and figured I might as well put this out there and hopefully others will find it to be a fast and useful way to search and browse npm packages as I have.

If you’re interested in other things I’m working on, follow me on Twitter or check out the open source projects I’ve been publishing on GitHub.

I am also working on a Twitter bot for this site to tweet the most popular, newest, random packages from npm. Please follow that account now and it will start sending out packages soon–ish.

Open Software & Tools

This site wouldn’t be possible without the immense generosity and tireless efforts from the people who make contributions to the world and share their work via open source initiatives. Thank you 🙏

© 2024 – Pkg Stats / Ryan Hefner

@tableland/jeti

v2.0.0

Published

An IPFS and generic plugin framework for inserting & retrieving data with the @tableland/sdk

Downloads

28

Readme

@tableland/jeti

Review Test License: MIT AND Apache-2.0 Version standard-readme compliant

An IPFS and generic plugin framework for inserting & retrieving data with the @tableland/sdk

Table of Contents

Background

The @tableland/jeti library builds on top of the @tableland/sdk, allowing you to easily add IPFS data to Tableland and read the underlying data from Tableland. Simply import the library, connect to the Tableland network, and you are ready to start creating, updating, and reading table data.

JETI (JavaScript Extension for Tableland Integrations) is also designed to be extensible. You can create your own custom plugins/processors that transform data inserted into tables or retrieved from tables. For example, the IPFS plugin processes data and inserts a CID into a cell, and when data is read, it will "see" the CID and fetch/transform the underlying content within the query response. You can do whatever you'd like with an implementation using the creatorProcessor method.

Install

Installation is easy using npm or yarn. An ES bundle is also available for those operating purely in a browser environment.

npm i @tableland/jeti

Usage

Before getting started, you'll need to set up and have an IPFS node running. For example, IPFS Desktop exposes the API on http://127.0.0.1:5001 by default; you'll then need to pass host/port information to connect to the IPFS node, which are endpoints defined in the IPFS HTTP API.

For the full documentation, see the docs page: here.

Set up Tableland database

First, connect and create a table. You'll want to make sure @tableland/sdk is installed as well, and this example shows ethers for setting up a signer with a private key on a local-only Local Tableland network running.

import { Database } from "@tableland/sdk";
import { Wallet, getDefaultProvider } from "ethers";

// Set up signer
const privateKey =
  "59c6995e998f97a5a0044966f0945389dc9e86dae88c7a8412f4603b6b78690d";
const wallet = new Wallet(privateKey);
// To avoid connecting to the browser wallet (locally, port 8545).
// For example: "https://polygon-mumbai.g.alchemy.com/v2/YOUR_ALCHEMY_KEY"
const provider = getDefaultProvider("http://127.0.0.1:8545");
const signer = wallet.connect(provider);

// Connect to database & create table
const db = new Database({ signer });
const { meta: create } = await db
  .prepare(`CREATE TABLE my_table (id integer primary key, val text);`)
  .run();
await create.txn?.wait();
const [tableName] = create.txn?.names ?? [];
console.log(tableName); // my_table_31337_2

Pin to IPFS

In this example, using pinToLocal as a tagged template will insert the proper values in your statement, while simultaneously uploading files to your remote pinning services.

Using the resolve function of pinToLocal fetches the data from IPFS into the result set.

// Existing imports
// For pinning to a local IPFS node
import { pinToLocal, skip } from "@tableland/jeti";

// Database/table setup steps from above

// Set up pinner for string templating
const localPinner = pinToLocal({
  host: "127.0.0.1",
  port: 5001,
  protocol: "http",
});

// Define content and process SQL string
const contentToPin = "Hello world"; // A string, or a file buffer (Uint8Array)
const sql = await localPinner`insert into ${skip(
  tableName
)} (val) values ('${contentToPin}');`; // Converts non-skipped variable to CID
console.log(sql);

// Insert the transformed data with a CID
const { meta: insert } = await db.prepare(sql).all();
await insert.txn?.wait();

When you read the raw results, it'll contain the CID that was inserted, unless to resolve the results with the pinner:

const { results } = await db.prepare(`SELECT * FROM ${tableName}`).all();
console.log(results);
// [
//   {
//     id: 1,
//     val: 'bafybeiabfiu2uipule2sro2maoufk2waokktnsbqp5gvaaod3y44ouft54'
//   }
// ]

const resultsWithCIDsResolved = await localPinner.resolve(results, ["val"]);
console.log(resultsWithCIDsResolved);
// [
//   {
//     id: 1,
//     val: 'Hello world'
//   }
// ]

The pinToProvider follows nearly the same steps, except is assumes you've configured a remote pinning provider on your IPFS node. For example, if you've configured Pinata, you can use the pinToProvider template to pin to Pinata.

Truncate

Table cells have a max limit of 1 KB (1024 bytes), so the IPFS plugin is especially useful for data larger than that. However, if you're okay with lossy data that gets cut at 1024 bytes, the truncate plugin can be used:

import { truncate } from "@tableland/jeti";

// 1025 letter 'a's, i.e., 1025 bytes is one over the limit
const longString = new Array(1026).join("a"); // First value is `undefined`, so it will be skipped

const sql =
  await truncate`INSERT INTO my_table_31337_2 (val) values ('${longString}')`;

Since the input string was one byte over the limit, it will be truncated to 1024 bytes. When you resolve it, the original string will not be returned—an ellipses is appended to represent the truncation occurred:

const detruncated = await truncate.resolve([{ val: `${longString}` }], ["val"]);
console.log(detruncated); // Truncated input at 1024 bytes plus `...`

Create your own

Creating your own processor requires two function to be created: one for transforming inputs before data is created, and one for transforming the data after it is read. For example, if you wanted to add a value to the end of a string before it is inserted, and remove it when it is read, you could do the following:

import { createProcessor } from "@tableland/jeti";

function addValue(value: string): PrepareResult {
  const add = (input: string) => {
    return input + value;
  };

  const remove = (input: string) => {
    return input.replace(value, "");
  };

  return createProcessor(add, remove);
}

Then, set up the processor and insert data:

const processor = addValue(" world");
const originalValueOne = "Hello";
const originalValueTwo = "Hello again";

const sql =
  await processor`INSERT INTO my_table_31337_2 (val) VALUES ('${originalValueOne}'), ('${originalValueTwo}');`;
console.log(sql);
// INSERT INTO my_table_31337_2 VALUES ('Hello world'), ('Hello again world');

The processor is designed to work with the parameter and response types from the Tableland SDK. But, to emulate it's functionality, we can do the following and resolve the results:

const rawResults = [
  {
    id: 1,
    val: "Hello world",
  },
  {
    id: 2,
    val: "Hello again world",
  },
];

const unprocessedResults = await processor.resolve(rawResults, ["val"]);
console.log(unprocessedResults);
// [
//   {
//     id: 1,
//     val: "Hello",
//   },
//   {
//     id: 2,
//     val: "Hello again",
//   },
// ];

Development

Get started by cloning, installing, building, and testing the project:

git clone https://github.com/tablelandnetwork/jeti.git
cd jeti
npm install
npm run build
npm test

Contributing

PRs accepted.

Small note: If editing the README, please conform to the standard-readme specification.

License

MIT AND Apache-2.0, © 2021-2023 Tableland Network Contributors