better-sqlite3-octodb

OctoDB for Node.js, using better-sqlite3

• Full transaction support
• High performance, efficiency, and safety
• Easy-to-use synchronous API (better concurrency than an asynchronous API... yes, you read that correctly)
• Support for user-defined functions, aggregates, virtual tables, and extensions
• 64-bit integers (invisible until you need them)
• Worker thread support (for large/slow queries)

How other libraries compare

| |select 1 row  get() |select 100 rows   all()  |select 100 rows iterate() 1-by-1|insert 1 row run()|insert 100 rows in a transaction| |---|---|---|---|---|---| |better-sqlite3|1x|1x|1x|1x|1x| |sqlite and sqlite3|11.7x slower|2.9x slower|24.4x slower|2.8x slower|15.6x slower|

You can verify these results by running the benchmark yourself.

Installation

npm install better-sqlite3-octodb

You must be using Node.js v14.21.1 or above. Prebuilt binaries are available for LTS versions. If you have trouble installing, check the troubleshooting guide.

Usage

const options = { verbose: console.log };
const uri = 'file:test.db?node=secondary&connect=tcp://127.0.0.1:1234';
const db = require('better-sqlite3-octodb')(uri, options);

db.on('ready', function() {

  const row = db.prepare('SELECT * FROM users').all();
  console.log(row.name, row.email);

});

db.on('sync', function() {

  console.log('the database received updates');

});

You can also check the status with:

let res = db.prepare('PRAGMA sync_status').get();
let status = JSON.parse(res.sync_status);

Though not required, it is generally important to set the WAL pragma for performance reasons.

db.pragma('journal_mode = WAL');

In ES6 module notation:

import Database from 'better-sqlite3';
const db = new Database('foobar.db', options);
db.pragma('journal_mode = WAL');

Why should I use this instead of node-sqlite3?

• node-sqlite3 uses asynchronous APIs for tasks that are either CPU-bound or serialized. That's not only bad design, but it wastes tons of resources. It also causes mutex thrashing which has devastating effects on performance.
• node-sqlite3 exposes low-level (C language) memory management functions. better-sqlite3 does it the JavaScript way, allowing the garbage collector to worry about memory management.
• better-sqlite3 is simpler to use, and it provides nice utilities for some operations that are very difficult or impossible in node-sqlite3.
• better-sqlite3 is much faster than node-sqlite3 in most cases, and just as fast in all other cases.

When is this library not appropriate?

In most cases, if you're attempting something that cannot be reasonably accomplished with better-sqlite3, it probably cannot be reasonably accomplished with SQLite3 in general. For example, if you're executing queries that take one second to complete, and you expect to have many concurrent users executing those queries, no amount of asynchronicity will save you from SQLite3's serialized nature. Fortunately, SQLite3 is very very fast. With proper indexing, we've been able to achieve upward of 2000 queries per second with 5-way-joins in a 60 GB database, where each query was handling 5–50 kilobytes of real data.

If you have a performance problem, the most likely causes are inefficient queries, improper indexing, or a lack of WAL mode—not better-sqlite3 itself. However, there are some cases where better-sqlite3 could be inappropriate:

• If you expect a high volume of concurrent reads each returning many megabytes of data (i.e., videos)
• If you expect a high volume of concurrent writes (i.e., a social media site)
• If your database's size is near the terabyte range

For these situations, you should probably use a full-fledged RDBMS such as PostgreSQL.

Documentation

• API documentation
• Performance (also see benchmark results)
• 64-bit integer support
• Worker thread support
• Unsafe mode (advanced)
• SQLite3 compilation (advanced)
• Contribution rules
• Code of conduct

License

MIT