@jetit/id

This ID package is used to generate lexically sorted IDs that are randomly generated. Given the constraints, this package has IDs that have a higher change of collision when used at extreme scales. Care has been taken to avoid this from occuring.

How to install the library

npm install @jetit/id or yarn add @jetit/id

Design Constraints and solution

• Should work in a decentralised Asynchronous environment
• Should not be a generic GUID/UID but rather contain domain specificity or type identification as part of the ID
• Should be random with very very low chances of collision irrespective of it being generated at the same time in the same machine/across machine
• Should be sortable
  • chronologically sortable
  • lexicographically sortable/alphabetically sortable and k-sortable
• Should be verifiable based on checksum. not every ID that can be generated should pass the test
• Should not affect performance. Keys should be discardable
• Should have the following qualities
  • URL Safe
  • 64 bits long
  • Binary keys
• Keys should be opaque - Inference of underlying data should not be possible other than the ID
• Should have no dependencies outside of the ID generation system
• Should function as a database key and make indexing easier
• Should have one or more representations that have a total length of less than 15 characters

The final arrived spec based on these constraints were based on variations to the Snowflake ID that was created by Twitter/Instagram.

• The timestamp is modified to custom Epoch which starts at 2022 Jan when this library is going live and being a 32 bit epoch it can support upto 136 years (till: 2158). IDs are generated at creation time so they need not exist prior to system existence.
• The client ID will be generated by NanoId using the Crockfords base 32 Alphabet and converted to the equivalent binary. This converted to binary gives us 1,048,576 unique variations. Since this is assigned to the clients, the number of competing clients at this time is not an issue. The library can be updated to support additional clients by changing the 64 bit constraint.
• The sequence is 7 bits and is dependent on the Timestamp and the client ID. .
• The checksum bit is calculated by a custom algorithm based on binary addition and XOR Algorithm. There are 16 possible options for the checksum

The resulting data structure looks like || Sign || Timestamp || Client ID || Sequence || Checksum || Type Identifier || |--------|------------|------------|-----------|-----------|-----------------| | 1 bit | 32 bits | 20 bits | 7 bits | 4 bits | 8 bits |

How to use the library

The library exposes the following functions

Generate an ID

const id = generateId() const id = generateId('DECIMAL) const id = generateId('DECIMAL', 'DD')

This function takes a representation and a type identifier as optional input.

Representation The representation can be one of 'BINARY' | 'DECIMAL' | 'HEX' | 'STRING' | 'URLSAFE'.

The string value is generated using the chatAt function and so the final representation can be filled with spaces or non-english characters and is to be used with caution

The decimal representation when used with a type identifier always returns a bigInt.

If no representation is passed, you get the binary representation by default.

Type Identifier The type identifier adds the ability to tie the id generated to a specific type. For example, if you want to generate a personID for a person, you can configure a specific ID, 01 to that type and this type is implemented into the ID. This allows to implement an initial validation from an id perspective at the API layer.

:warning: When it comes to lexical sorting, choose to either use type identifiers throughout the system or not at all. The final resulting value changes drastically based on whether you choose the typeidentifier or not

Validate an ID

const isValid = validateId(id, 'DECIMAL') const isValid = validateId(id, 'DECIMAL' 'DD')

The validateId function takes 2 mandatory parameters and one optional parameter as input

• id: This the ID in the system that needs to be validated
• representation: This is the representation that the ID that is passed in currently has
• type identifier: This is an optional parameter and can be passed in to verify the type if the type was used in generating the ID.

Transform ID Representation

const transform = transformId(id, 'DECIMAL', 'BINARY')

Explain ID

const explanation = explainId(id, 'DECIMAL) The explain ID can be used to get the explanation of the ID along with the components that were involved in generating the ID. An example is displayed below

   {
      id: {
        binary: '100000001010000010110111010100011000001101011100001000000001110000011101',
        decimal: 2372764056026257293312n,
        hex: '80a0b751835c200000',
        string: '\x80 ·Q\x83\\ \x1C\x1D',
        urlsafe: 'gKC3UYNcIBwd'
      },
      typeIdentifier: '1d',
      clientId: 'A46E',
      sequence: 1,
      createdTimestamp: 1662060579000,
      createdTimestampReadable: '2022-09-01T19:29:39.000Z',
      representation: 'URLSAFE'
    }

Generate RRNs

const rrns = {rrn1: generateRRN(), rrn2: generateRRN(), rrn55: generateRRN(55)} The Generate RRNs can be used to generate Retrieval Reference Number for a given STAN (optional defaults to 1). This follows the standard guidelines issued by Card Brands.

   {
      rrn1: 300815000001
      rrn2: 300815000002
      rrn55: 300815000055
   }

Other Gotchas/Features

• When using the hex/binary representation and you have a length restriction, you can chop off and add the type identifier byte as required. This is not possible in the other representations.
• The setup also supports a sign bit if it needs to be ported into another system that does not support unsigned integers
• Client ID can be used to validate the clients on the server side if required

TODO

• Performance tests comparing against other ID generators
• Collision tests across multiple clients