npm package discovery and stats viewer.

Discover Tips

  • General search

    [free text search, go nuts!]

  • Package details

    pkg:[package-name]

  • User packages

    @[username]

Sponsor

Optimize Toolset

I’ve always been into building performant and accessible sites, but lately I’ve been taking it extremely seriously. So much so that I’ve been building a tool to help me optimize and monitor the sites that I build to make sure that I’m making an attempt to offer the best experience to those who visit them. If you’re into performant, accessible and SEO friendly sites, you might like it too! You can check it out at Optimize Toolset.

About

Hi, 👋, I’m Ryan Hefner  and I built this site for me, and you! The goal of this site was to provide an easy way for me to check the stats on my npm packages, both for prioritizing issues and updates, and to give me a little kick in the pants to keep up on stuff.

As I was building it, I realized that I was actually using the tool to build the tool, and figured I might as well put this out there and hopefully others will find it to be a fast and useful way to search and browse npm packages as I have.

If you’re interested in other things I’m working on, follow me on Twitter or check out the open source projects I’ve been publishing on GitHub.

I am also working on a Twitter bot for this site to tweet the most popular, newest, random packages from npm. Please follow that account now and it will start sending out packages soon–ish.

Open Software & Tools

This site wouldn’t be possible without the immense generosity and tireless efforts from the people who make contributions to the world and share their work via open source initiatives. Thank you 🙏

© 2024 – Pkg Stats / Ryan Hefner

bigint-buffer-browser

v1.0.0

Published

bigint-buffer with native code removed

Downloads

2

Readme

💪🔢 bigint-buffer: Buffer Utilities for TC39 BigInt Proposal

NPM Package Build Status Coverage Status node

bigint-buffer is a utility converts TC39 Proposed BigInts to and from buffers. This utility is necessary because BigInts, as proposed, do not support direct conversion between Buffers (or UInt8Arrays), but rather require conversion from buffers to hexadecimal strings then to BigInts, which is suboptimal. This utility includes N-API bindings, so under node, conversion is performed without generating a hexadecimal string. In the browser, normal string conversion is used.

Why use BigInts?

BigInts are currently a stage 3 proposal, supported in Node 10 and V8 v6.7. BigInts are primitive arbitrary precision integers, overcoming the limitations of the number type in javascript, which only supports up to 53 bits of precision.

In many applications, manipulating 64, 128 or even 256 bit numbers is quite common. For example, database identifiers are often 128 bits, and hashes are often 256 bits (If you're looking for hashing support, try out bigint-hash). Before BigInts, manipulating these numbers safely required either allocating a Buffer or UInt8Arrays, which is quite expensive compared to a number, since Buffers are allocated on the heap.

BigInts solve this problem by introducing a primitive that can hold arbitrary precision integers, reducing memory pressure and allowing the runtime to better optimize arithmetic operations. This results in significant performance improvements - 10x-100x for simple equality comparisons (using === vs Buffer.compare()):

Buffer equality comparison: 11916844±4.23% ops/s 83.91±17.293 ns/op (91 runs)
bigint equality comparison: 798024851±0.29% ops/s 1.25±0.017 ns/op (91 runs)

Before BigInts, you probably used a library such as the widely used bn.js. bn.js fares a little better than a plain Buffer, but is still 5-10x slower than the bigint:

BN equality comparison: 73255774±0.67% ops/s 13.65±0.442 ns/op (89 runs)

bigints are also much better with arithmetic, here are the results compared to BN for multiplying two 128-bit integers, yielding a 4x improvement:

BN multiply: 4763236±0.49% ops/s 209.94±5.111 ns/op (93 runs)
bigint multiply: 15268666±0.92% ops/s 65.49±2.938 ns/op (92 runs)

So what's the problem?

BigInts, unfortunately lack an efficient way to be converted back and forth between buffers. When dealing with serialized data or legacy node code, you'll often want to generate a BigInt from a buffer, or convert a BigInt to a Buffer in order to send a BigInt over the wire.

Currently, the only method to generate a new BigInt is with the BigInt constructor. Unfortunately, it doesn't support Buffers, though it may in the future:

> BigInt(Buffer.from([1]))
SyntaxError: Cannot convert  to a BigInt

Instead, you need to convert the Buffer to a hexadecimal string of the correct format. For example:

> BigInt(`0x${buf.toString('hex')}`);
1n

These conversions are not only quite expensive, but obviate a lot of the performance gains we get from BigInts. For example, on a large buffer, this conversion can take over 100x the time to do a comparison:

bigint from hex string from buffer (huge): 1230607±1.02% ops/s 812.61±40.013 ns/op (89 runs)

And... bigint-buffer helps how?

bigint-buffer introduces four functions for conversion between buffers and bigints. A small example follows:

import {toBigIntBE, toBigIntLE, toBufferBE, toBufferLE} from 'bigint-buffer';

// Get a big endian buffer of the given width
toBufferBE(0xdeadbeefn, 8);
// ↪ <Buffer 00 00 00 00 de ad be ef>

// Get a little endian buffer of the given width
toBufferLE(0xdeadbeefn, 8);
// ↪ <Buffer ef be ad de 00 00 00 00>

// Get a BigInt from a buffer in big endian format
toBigIntBE(Buffer.from('deadbeef', 'hex'));
// ↪ 3735928559n (0xdeadbeefn)

// Get a BigInt from a buffer in little endian format
toBigIntLE(Buffer.from('deadbeef', 'hex'));
// ↪ 4022250974n (0xefbeadd0en)

bigint-buffer uses N-API native bindings to perform the conversion efficiently without generating the immediate hex strings necessary in pure javascript. This results in a significant performance increase, about 2x for small buffer to bigint conversions, and 8x better than BN:

BN to buffer (small): 981703±68.30% ops/s 1018.64±3194.648 ns/op (81 runs)
bigint from hex string from buffer (small): 2804915±5.00% ops/s 356.52±85.371 ns/op (88 runs)
LE bigint-buffer ToBigInt (small): 5932704±1.62% ops/s 168.56±12.971 ns/op (87 runs)

And about 3.3x for bigint to buffer conversions, and 17x better than BN:

BN to buffer (large): 339437±2.85% ops/s 2946.06±385.504 ns/op (81 runs)
BE bigint to hex string to buffer (large): 1714292±1.35% ops/s 583.33±37.995 ns/op (90 runs)
BE bigint-buffer to buffer (large, truncated): 5977218±4.68% ops/s 167.30±37.284 ns/op (87 runs)

You can run the benchmarks by running npm run benchmark.

Typescript Support

bigint-buffer supplies typescript bindings, but BigInts are still not supported in typescript, though a pull request has been opened, so support should be coming soon. If you are using typescript, @calebsander has put up a pull request and the instructions in this post.

Install

Add bigint-buffer to your project with:

npm install bigint-buffer

Documentation

Basic API documentation can be found here. Note that v1.0.0 changes the name of the original functions to meet style guidelines.

Benchmarks

Benchmarks can be run by executing npm run benchmark from the package directory.