base-unicode
v1.0.0
Published
Encodes text and binary data to a subset of Unicode plane 1 (BMP). Runs in browser and Node.js. Character compression. Makes binary data copy-pastable. URL-safe.
Downloads
6,813
Maintainers
Readme
base-unicode
Transcodes string
and Uint8Array
(binary) blob data to and from Unicode.
This algorithm allows for character compression as two bytes are usually represented
by one Unicode character in the alphabet, base-unicode uses.
base-unicode therefore allows for a lossless conversion of binary data to and from Unicode. This is useful for storing binary data in a database, for example but also for shortening binary data for a text representation that can be copy-pasted.
This again allows e.g. for sharing binary and text data in a character compressed form that can be easily copied and pasted, for example as a parameter in a URL or even via twitter.
Setup
As a package for development (Node.js, Browsers):
yarn add base-unicode
# or
npm i base-unicode
Usage
The usage in a Node.js or Browser environment is trivial:
import { encode, decodeToString, decodeToUint8Array } from 'base-unicode'
// encoding + decoding strings
const encoded = encode('Hello, world!') // 1劒碶翚禼誎藝矚h
const decoded = decodeToString(encoded) // Hello, world!
//encoding + decoding binary data
const input = new Uint8Array([0xb, 0xa, 0xb, 0xe]) // a.k.a. [ 11, 10, 11, 14 ]
// you can of course use File, Blob and Buffer as well
const encodedBinary = encode(input) // 0A坘存
const decodedBinary = decodeToUint8Array(encodedBinary) // [ 11, 10, 11, 14 ]
Limitations
The alphabet of base-unicode
is 21091
characters long. It has been carefully
selected to be supported by the majority of system fonts. The default base-unicode
alphabet consists of the following Unicode character ranges (always upper- and lower-case included):
a-z, α-ω, а-я 一-龯
To make sure that the alphabet is URL-safe and doesn't run into invisible character issues, all non-printable control characters and none-URL-safe characters are excluded.
However, some fonts don't support all of these characters. To check if your
system supports copying and pasting text that has been encoded with base-unicde
,
you can simply check the ALPHABET file. If you can spot one character that shows
as a non-renderable square, this algorithm doesn't work on your system.
Test
yarn test