Text Essence

Remove non-alphanumeric characters from any Unicode string. Optionally, also remove diacritical marks.

Essence

TextEssence.essence gives you the essence of a string, i.e., only the alphanumerical characters, converted to lower case:

const TextEssence = require('text-essence');
let city = TextEssence.essence('Saint-Étienne');
// 'saintétienne'

Optionally, you can also remove diacritical marks:

const TextEssence = require('text-essence');
let aggressiveTextEssence = new TextEssence({ removeDiacriticalMarks: true });
let city = aggressiveTextEssence.essence('Saint-Étienne');
// 'saintetienne'

Identical

TextEssence.identical lets you check whether two strings are essentially the same:

const TextEssence = require('text-essence');
let sadlyFalse = 'Saint-Étienne' === 'Saint–Étienne';
// false
let happilyTrue = TextEssence.identical('Saint-Étienne', 'Saint–Étienne');
// true

Of course, you have the option to ignore diacritical marks. This should increase recall, while probably harming precision:

const TextEssence = require('text-essence');
let aggressiveTextEssence = new TextEssence({ removeDiacriticalMarks: true });
let sadlyFalse = 'Saint-Étienne' === 'Saint-Etienne';
// false
let happilyTrue = aggressiveTextEssence.identical('Saint-Étienne', 'Saint-Etienne');
// true

Essential Hash

TextEssence.essentialHash gives you the hash of the essence of a string:

const TextEssence = require('text-essence');
let hash = TextEssence.essentialHash('Saint-Étienne');
// 'bb35a91d6f2bd7ba807fdd240cc838bdd3b20fe1a6fdedba60d941fbe8d5c10f'

By default, it uses the sha256 algorithm, but you can pick a different one:

const TextEssence = require('text-essence');
let sha1TextEssence = new TextEssence({ hashAlgorithm: 'sha1' });
let hash = sha1TextEssence.essentialHash('Saint-Étienne');
// '03f938949514a56c863252d3559d0fd92d40720e'