node-structor

Convert binary file data to JavaScript objects

Installation

npm i node-structor

Usage

In the example below we will be reading a list of people from a binary source.

const fs = require('fs');
const Structor = require('node-structor');

const structDef = {
    numPersons: {
        $format: 'byte',
        $ignore: true
    },
    persons: {
        $repeat: 'numPersons',
        $format: {
            firstName: 'string7',
            lastName: 'string7',
            address: {
                city: 'string7',
                street: 'string7',
                number: 'uint16',
                zipCode: 'string7'
            },
            numHobbies: {
                $ignore: true,
                $format: 'byte',
            },
            hobbies: {
                $format: 'string7',
                $repeat: 'numHobbies'
            }
        }
    }
};

let result = Structor.readStruct(structDef, fs.readFileSync('./examples/people.dat'));

console.log(result);

Not that 'string7' is used - this denotes a string that is prepended by the length of that string. Reference.

Running this will log the following:

{
    persons: [
        {
            firstName: 'John',
            lastName: 'A',
            address: {
                city: 'New York',
                street: '1st Ave.',
                number: 1165,
                zipCode: '10065'
            },
            hobbies: [ 'eating', 'coding', 'walking' ]
        },
        {
            firstName: 'Betty',
            lastName: 'B',
            address: {
                city: 'York',
                street: 'Bridge St.',
                number: 1,
                zipCode: 'YO1 6DD'
            },
            hobbies: []
        }
    ]
}

API

readStruct(structDef, buffer, [options])

Reads the binary data from the buffer according to the structure definition. Returns a JavaScript object.

writeStruct(obj, structDef, data, [options])

Writes the an object to a binary buffer according to the structure definition. Returns the number of bytes written.

sizeOf(structDef, [size = 4096])

Calculates the size of the binary data that would be written according to the structure definition. The size parameter is optional and defaults to 4096; it references the size of the buffer that will be allocated to be able to calculate the size of the data.

Types

| Type | Description -------|--------- | byte | Unsigned byte (0 to 255) | uint8 | Unsigned byte (0 to 255) | sbyte | Signed byte (-128 to 127) | int8 | Signed byte (-128 to 127) | uint16 | 16-bit unsigned integer (0 to 65,535) | int16 | 16-bit signed integer (-32,768 to 32,767) | uint32 | 32-bit unsigned integer (0 to 4,294,967,295) | int32 | 32-bit signed integer (-2,147,483,648 to 2,147,483,647) | uint64 | 64-bit unsigned integer (read as BigInt) | int64 | 64-bit signed integer (read as BigInt) | char_* | A string of characters with its length defined by the *. e.g. char_28 | string0| A string of characters terminated by a zero (0) byte. When used with writeStruct, it will write the string with a zero byte at the end. | string7| A string of characters prepended by its 7-bit encoded length | string | Can only be used in conjunction with $format. Read $length amount of bytes as a new string. Can also be used with $encoding to specify the encoding. Default is utf8. | buffer | Can only be used in conjunction with $format. Read $length amount of bytes as a new Buffer.

Note: By default the endianness is little-endian (LE) - But you can explicitly define the endianness e.g. int16be, uint64le, etc.

Directives

$format

Define the format. This can be any of the types mentioned above, or another structure definition.

Examples:

{ 
    someNumber: {
        $format: 'uint16'    // Results in a single number
    },
    anotherNumber: 'uint16'  // Short-hand for the above
}

Some types only work in conjunction with $format, as they require extra information to be on the same level. These are: string and buffer.

Examples:

{
    name: {
        $format: 'string',
        $length: 32,
        $encoding: 'ascii' // $encoding is optional, default is 'utf8'
    }
}

{
    blobData: {
        $format: 'buffer',
        $length: 64000
    }
}

$repeat

Repeats the specified $format. Can be a number or the name of a property containing the value.

Examples:

{
    $format: 'byte',
    $repeat: 2
}

{
    numObjects: 'byte',
    objects: {
        $format: {
            ...
        },
        $repeat: 'numObjects'
    }
}

$foreach

A special form of $repeat. Must be a referenced value pointing to a previously read array combined with an alias.

Examples:

{
    numFiles: 'uint16',
    fileTable: {
        $repeat: 'numFiles',
        $format: {
            name: 'char_24',
            address: 'uint32',
            length: 'uint32'
        }
    },
    files: {
        // Iterate over each item in fileTable as 'file'
        $foreach: 'fileTable file', 
        $format: {
            fileName: {
                $value: 'file.name',
                $format: 'char_24'
            },
            fileContent: {
                $goto: 'file.address',
                $format: 'buffer',
                $length: 'file.length'
            }
        }
    }
}

$switch

Read the next data differently based on a previously read value.

Examples:

{
    type: 'byte',
    shape: {
        $switch: 'type',
        $cases: [
            {
                $case: 1, // when type is 1, assume circle data follows
                $format: {
                    radius: 'uint32'
                }
            },
            {
                $case: 2, // 2 = square data
                $format: {
                    width: 'uint16',
                    height: 'uint16'
                }
            },
            {
                $case: 3, // 2 = polygonal data
                $format: {
                    numPoints: {
                        $ignore: true,
                        $format: 'byte'
                    },
                    points: {
                        $repeat: 'numPoints',
                        $format: 'byte'
                    }
                }
            }
        ]
    }
}
// Which could result in:
{
    type: 1,
    shape: {
        radius: 38892
    }
},
{
    type: 2,
    shape: {
        width: 96,
        height: 128
    }
},
{
    type: 3,
    shape: {
        points: [0, 2, 128, 24, 255, 8]
    }
}

$ignore

Read the data, but don't put the property in the eventual JS object.

Examples:

numObjects: {
    $format: 'byte',
    $ignore: true
}

$goto

Jumps to the specified byte location before reading the value.

Examples:

signature: {
    $goto: 0xf0,
    $format: 'char_2'    
}

$skip

Skips the specified number of bytes before reading the value.

Examples:

startOfHeader: {
    $skip: 255,
    $format: 'uint16'    
}

$length

Can only be used in conjunction with $format: 'string' and must be used when $format: 'buffer'.

Examples:

firstName: {
    $format: 'string',
    $length: 32
}

Note: when $format is 'string', $length is optional. If not present, characters will be read until a zero-byte is encountered.

blobData: {
    $format: 'buffer',
    $length: 64000
}

$encoding

Can only be used in conjunction with $format: 'string'.

Examples:

firstName: {
    $format: 'string',
    $encoding: 'ascii'
}

Note: the default value for $encoding is 'utf8'

$value

A special directive that doesn't read anything from the buffer and thus doesn't move the internal cursor. Used to copy a value from another source.

{
    name: 'string',
    nameCopy: {
        $value: 'name',
        $format: 'uint32'
    }
}

$tell

A special directive that reads the current position of the internal cursor. Must be used in conjunction with $format: '$tell'.

{
    name: 'string',
    currentAddress: {
        $format: '$tell',
        $tell: 'uint16'
    }
}

Referenced values

Every numeric directive supports passing a reference value string instead of a hard-coded integer. This can be a simple name pointing to a sibling value, or a more complex path.

Examples:

{
    nameLength: 'byte',
    myName: {
        $format: 'string',
        $length: 'nameLength'
    }
}

{
    header: {
        config: {
            nameLength: 'byte'
        }
    },
    myName: {
        $format: 'string',
        $length: 'header.config.nameLength'
    }
}

Directives that support this:

• $repeat
• $switch
• $length
• $goto
• $skip
• $value