npm package discovery and stats viewer.

Discover Tips

  • General search

    [free text search, go nuts!]

  • Package details

    pkg:[package-name]

  • User packages

    @[username]

Sponsor

Optimize Toolset

I’ve always been into building performant and accessible sites, but lately I’ve been taking it extremely seriously. So much so that I’ve been building a tool to help me optimize and monitor the sites that I build to make sure that I’m making an attempt to offer the best experience to those who visit them. If you’re into performant, accessible and SEO friendly sites, you might like it too! You can check it out at Optimize Toolset.

About

Hi, 👋, I’m Ryan Hefner  and I built this site for me, and you! The goal of this site was to provide an easy way for me to check the stats on my npm packages, both for prioritizing issues and updates, and to give me a little kick in the pants to keep up on stuff.

As I was building it, I realized that I was actually using the tool to build the tool, and figured I might as well put this out there and hopefully others will find it to be a fast and useful way to search and browse npm packages as I have.

If you’re interested in other things I’m working on, follow me on Twitter or check out the open source projects I’ve been publishing on GitHub.

I am also working on a Twitter bot for this site to tweet the most popular, newest, random packages from npm. Please follow that account now and it will start sending out packages soon–ish.

Open Software & Tools

This site wouldn’t be possible without the immense generosity and tireless efforts from the people who make contributions to the world and share their work via open source initiatives. Thank you 🙏

© 2024 – Pkg Stats / Ryan Hefner

@digitallinguistics/transliterate

v0.4.1

Published

A small JavaScript library for transliterating and/or sanitizing strings

Downloads

262

Readme

Transliterate

A small JavaScript library for transliterating and/or sanitizing strings. Tested against a variety of edge cases and unusual inputs.

GitHub Release GitHub issues DOI GitHub license GitHub stars

Overview

This library is useful for linguists and data analysts working with language data. It can be used to convert a string from one writing system to another (a process known as transliteration), or to remove unwanted characters or sequences of characters from a string (a process known as sanitization). This library handles common problems that arise during transliteration and sanitization, including bleeding and feeding issues.

Citation & Attribution

This library is maintained by Daniel W. Hieber. You can cite this library with its DOI using the following model:

Hieber, Daniel W. 2019. digitallinguistics/transliterate. DOI: 10.5281/zenodo.2550468.

Each version of this library is archived on this project's Zenodo page.

Installation

Install with npm or yarn:

npm install @digitallinguistics/transliterate # npm
yarn add @digitallinguistics/transliterate    # yarn

Importing the Library

In the browser, include the library in your HTML (adjust the src to point to the location of the transliterate.js file in your project):

<script src=transliterate.js type=module></script>

In Node, simply import the library:

import { transliterate } from '@digitallinguistics/transliterate';

Basic Usage

The transliterate library exports an object with four methods:

  • transliterate
  • Transliterator
  • sanitize
  • Sanitizer

The sanitize and Sanitizer exports are essentially just aliases for transliterate and Transliterator respectively.

To transliterate a string, use the transliterate method:

// Import the "transliterate" method from the library
import { transliterate } from '@digitallinguistics/transliterate';

// The list of substitutions to make
const substitutions = {
  p: `b`,
  t: `d`,
  k: `g`,
};

// The string to transliterate
const input = `patak`;

// Transliterate the string
const output = transliterate(input, substitutions);

console.log(output); // --> "badag"

To save a set of transliteration rules for reuse on more than one string, use the Transliterator class:

// Import the Transliterator class
import { Transliterator } from '@digitallinguistics/transliterate';

// The list of substitutions to use for transliteration
const substitutions = {
  p: `b`,
  t: `d`,
  k: `g`,
};

// Create a transliterate function that always
// applies the same substitutions
const transliterate = new Transliterator(substitutions);

// The string to transliterate
const input = `patak`;

// Transliterate the string
const output = transliterate(input);

console.log(output); // --> "badag"

View the entire API for this library here.

Working with Substitution Rules

The transliterate library already handles several tricky cases on your behalf. For example, say you have the following substitution rules, and want to use them on the string abc:

Input | Output :----:|:-----: a | b b | c

In this case, you probably intend the output to be bcc. But if you apply the a → b rule before the b → c rule, you get the output ccc. This is called a feeding problem. The transliterate library automatically avoids feeding problems, so that you get the expected result bcc rather than ccc.

Now say that you want to apply the following rules to the string abacad.

Input | Output :----:|:-----: a | b ac | d

You probably intend the output to be abdbd. But if you apply the a → b rule before the ac → d rule, you get the output bbbcbd. This is called a bleeding problem. The transliterate library automatically avoids bleeding problems as well, so that you get the expected result abdbd rather than bbbcbd.

Here are some things to remember about how the transliterate library applies substitutions:

  • Longer substitutions are always made first. If you have substitution rules for both ch and c, the library will first substitute all instances of ch with its replacement, followed by all instances of c.

  • If two substitution inputs are the same length, the substitutions will be applied in the order they were passed to the library. For example, if you have the rules ab → d and bc → e, in that order, the ab → d substitutions will be applied first.

Sometimes the way you want to transliterate a character or sequence of characters will depend on context. For example, you might want a to sometimes become b, and other times become c. In this case you have several options:

  • Update the original text to indicate the difference. For example, you might change all the as that you want to become cs to ɑ or maybe ac or aa or \a, or whatever makes sense for your project.

  • Update the substitution rules to take more context into account. For example, if a becomes b before c and becomes d elsewhere, you could write your rules like this:

    Input | Output :----:|:-----: ab | c a | d

  • Update both the original text and the subsitution rules. For example, you could update the original text to indicate syllable boundaries, and then update your substitution rules to use those boundaries. For instance, the sequence abc could be syllabified as a.bc or ab.c. After updating the original text with syllable boundaries, you could change your rules to target syllable-initial vs. syllable-final b; for example: .b → d (syllable-initial) and b. → e (syllable-final).