bireader

A feature rich binary reader and writer that keeps track of your position to quickly create file structures. Includes shared naming conventions, programmable inputs and advanced math for easy data conversions to do low level parsing. Accepts Uint8Array or Buffer.

Supported data types are:

• Bitfields ([u]bit{1-32}{le|be}) 1-32 bit signed or unsigned value in big or little endian order
• Bytes ([u]int8, byte) 8 bit signed or unsigned value
• Shorts ([u]int16, word, short{le|be}) 16 bit signed or unsigned value in big or little endian order
• Half Floats (halffloat, half{le|be}) 16 bit decimal value in big or little endian order
• Integers ([u]int32, long, int, double{le|be}) 32 bit signed or unsigned value in big or little endian order
• Floats (float{le|be}) 32 bit decimal value in big or little endian
• Quadwords ([u]int64, quad, bigint{le|be}) 64 bit signed or unsigned in big or little endian
• Double Floats (doublefloat, dfloat{le|be}) 64 bit decimal value in big or little endian
• Strings (string) Fixed and non-fixed length, UTF, pascal, wide pascal. Includes all TextEncoder types

Installation

npm install bireader

Provides both CommonJS and ES modules.

Example

Import the reader or writer. Create a new parser with the data and start parsing.

Includes presents for quick parsing or programmable functions (examples below).

import {bireader, biwriter} from 'bireader';

//read example - parse a webp file
function parse_webp(data){
  const br = new bireader(data)
  br.hexdump({supressUnicode:true}) //console.log data as hex

  //         0  1  2  3  4  5  6  7  8  9  A  B  C  D  E  F  0123456789ABCDEF
  // 00000  52 49 46 46 98 3a 00 00 57 45 42 50 56 50 38 58  RIFF.:..WEBPVP8X
  // 00010  0a 00 00 00 10 00 00 00 ff 00 00 ff 00 00 41 4c  ..............AL
  // 00020  50 48 26 10 00 00 01 19 45 6d 1b 49 4a 3b cf 0c  PH&.....Em.IJ;..
  // 00030  7f c0 7b 60 88 e8 ff 04 80 a2 82 65 56 d2 d2 86  ..{`.......eV...
  // 00040  24 54 61 d0 83 8f 7f 0e 82 b6 6d e3 f0 a7 bd ed  $Ta.......m.....
  // 00050  87 10 11 13 40 3b 86 8f 26 4b d6 2a b7 6d 24 39  ....@;..&K.*.m$9
  // 00060  52 4f fe 39 7f 3b 62 4e cc ec 9b 17 31 01 0c 24  RO.9.;bN....1..$
  // 00070  49 89 23 e0 01 ab 52 64 e3 23 fc 61 db 76 cc 91  I.#...Rd.#.a.v..
  // 00080  b6 7d fb 51 48 c5 69 db 4c 1b 63 db b6 ed b9 6d  .}.QH.i.L.c....m
  // 00090  db be 87 8d b1 6d db 9e b6 cd a4 d3 ee 24 95 54  .....m.......$.T
  // 000a0  52 b8 8e 65 a9 eb 38 ce ab 52 75 9d 67 ff 75 2f  R..e..8..Ru.g.u/
  // 000b0  77 44 40 94 6d 25 6c 74 91 a8 88 86 58 9b da 6e  [email protected]%lt....X..n

  const header = {}
  header.magic = br.string({length:4})    //RIFF
  header.size = br.uint32le()             //15000
  header.fileSize = header.size + 8       //15008
  header.payload = br.string({length:4})  //WEBP
  header.format = br.string({length:4})   //VP8X
  header.formatChunkSize = br.uint32le()  //10
  switch (header.format){
    case "VP8 ":
        header.formatType = "Lossy"
        var read_size = 0
        header.frame_tag = br.ubit24()
        read_size += 3;
        header.key_frame = header.frame_tag & 0x1;
        header.version = (header.frame_tag >> 1) & 0x7;
        header.show_frame = (header.frame_tag >> 4) & 0x1;
        header.first_part_size = (header.frame_tag >> 5) & 0x7FFFF;
        header.start_code = br.ubit24() //should be 2752925
        header.horizontal_size_code = br.ubit16();
        header.width = header.horizontal_size_code & 0x3FFF;
        header.horizontal_scale = header.horizontal_size_code >> 14;
        header.vertical_size_code = br.ubit16();
        header.height = header.vertical_size_code & 0x3FFF;
        header.vertical_scale = header.vertical_size_code >> 14;
        read_size += 7;
        header.VP8data = br.extract(header.formatChunkSize - read_size, true)
        break;
    case "VP8L":
        header.formatType = "Lossless"
        var read_size = 0
        header.signature = br.ubyte() //should be 47
        read_size += 1;
        header.readWidth =  br.ubit14()
        header.width = header.readWidth+1;
        header.readHeight =  br.ubit14()
        header.height = header.readHeight+1;
        header.alpha_is_used =  br.bit1() 
        header.version_number =  br.ubit3() 
        read_size += 4;
        header.VP8Ldata = br.extract(header.formatChunkSize - read_size, true)
        break;
    case "VP8X":
        header.formatType = "Extended"
        br.big() //switch to Big Endian bit read
        header.rsv = br.bit2()  //Reserved
        header.I = br.bit1()    //ICC profile
        header.L = br.bit1()    //Alpha
        header.E = br.bit1()    //Exif
        header.X = br.bit1()    //XMP
        header.A = br.bit1()    //Animation
        header.R = br.bit1()    //Reserved
        br.little() //return to little
        header.rsv2 = br.ubit24()
        header.widthMinus1 = br.ubit24()
        header.width = header.widthMinus1 + 1
        header.heightMinus1 = br.ubit24()
        header.height = header.heightMinus1 + 1
        if(header.I)
        {
          header.ICCP = br.string({length:4})  // Should be ICCP
          header.ICCPChunkSize = br.uint32()
          header.ICCPData = br.extract(header.ICCPChunkSize, true)
        }
        if(header.L)
        {
          header.ALPH = br.string({length:4})  // Should be ALPH
          header.ALPHChunkSize = br.uint32() //4134
          header.ALPHData = br.extract(header.ALPHChunkSize, true)
        }
        if(header.A)
        {
          header.ANI = br.string({length:4})  // Should be ANIM or ANIF
          header.ANIChunkSize = br.uint32()
          if(header.ANI == "ANIM")
          {
            header.BGColor = br.uint32()
            header.loopCount = br.ushort()
            header.ANIMData = br.extract(header.ANIChunkSize, true)
          } else
          if (header.ANI == "ANIF")
          {
            header.FrameX = br.ubit24()
            header.FrameY = br.ubit24()
            header.readFrameWidth = br.ubit24()
            header.readFrameHeight = br.ubit24()
            header.frameWidth = readFrameWidth + 1
            header.frameHeight = readFrameHeight + 1
            header.duration = br.ubit24()
            header.rsv3 = br.ubit6()
            header.byte.B = br.bit1() //Blending
            header.byte.D = br.bit1() //Disposal
            header.frameData = br.extract(16, true)
            header.ANIFData = br.extract(header.ANIChunkSize, true)
          }
        }
        header.extFormatStr = br.string({length:4})
        header.extChunkSize = br.uint32()
        header.extData = br.extract(header.extChunkSize, true)
        if(header.E)
        {
          header.EXIF = br.string({length:4})  // Should be EXIF
          header.EXIFChunkSize = br.uint32()
          header.EXIFData = br.extract(header.EXIFChunkSize, true)
        }
        if(header.X)
        {
          header.XMP = br.string({length:4})  // Should be XMP
          header.XMPChunkSize = br.uint32()
          header.XMPMetaData = br.extract(header.XMPChunkSize, true)
        }
        break;
    default:
        header.data = br.extract(header.formatChunkSize, true)
        break;
  }
  br.finished()
  return header
}

//write example - write a webp file from read data
function write_webp(data){
  const bw = new biwriter(new Uint8Arry(0x100000)) // Will extends array as we write if needed by default
  bw.string("RIFF",{length:4})
  bw.uint32le(0) //dummy for now, will be final size - 8
  bw.string("WEBP",{length:4})
  switch(data.format){
    case "VP8 ":
      bw.string("VP8 ",{length:4})
      bw.uint32le(data.VP8data.length)
      bw.ubit24(data.key_frame)
      bw.ubit24(data.start_code)
      bw.ubit16(data.horizontal_size_code)
      bw.ubit16(data.vertical_size_code)
      bw.overwrite(data.VP8data ,true)
      break;
    case "VP8L":
      bw.string("VP8L",{length:4})
      bw.uint32le(data.VP8Ldata.length - 4)
      bw.ubyte(47)
      bw.ubit14(data.width - 1)
      bw.ubit14(data.heigth - 1)
      bw.ubit1(data.alpha_is_used)
      bw.bit3(data.version_number)
      bw.overwrite(data.VP8Ldata,true)
      break;
    case "VP8X":
      bw.string("VP8X",{length:4})
      bw.uint32le(10)
      bw.big()
      bw.bit2(0)
      bw.bit1(data.I)
      bw.bit1(data.L)
      bw.bit1(data.E)
      bw.bit1(data.X)
      bw.bit1(data.A)
      bw.bit1(0)
      bw.little()
      bw.ubit24(data.rsv2)
      bw.ubit24(data.width - 1)
      bw.ubit24(data.height - 1)
      if(data.I)
      {
        bw.string(data.ICCP, {length:4})
        bw.uint32(data.ICCPData.length)
        bw.replace(data.ICCPData, true)
      }
      if(data.L)
      {
        bw.string(data.ALPH, {length:4})
        bw.uint32(data.ALPHData.length) 
        bw.replace(data.ALPHData)
      }
      if(data.A)
      {
        bw.string(data.ANI, {length:4})
        bw.uint32(data.ANIChunkSize)
        if(data.ANI == "ANIM")
        {
          bw.uint32(data.BGColor)
          bw.ushort(data.loopCount)
          bw.replace(data.ANIMData)
        } else
        if (data.ANI == "ANIF")
        {
          bw.ubit24(data.FrameX)
          bw.ubit24(data.FrameY)
          bw.ubit24(data.frameWidth - 1)
          bw.ubit24(data.frameHeigh - 1)
          bw.ubit24(data.duration)
          bw.ubit6(data.rsv3)
          bw.bit1(data.byte.B)
          bw.bit1(data.byte.D) 
          bw.replace(data.frameData, true)
          bw.replace(data.ANIFData, true)
        }
      }
      bw.string(data.extFormatStr, {length:4})
      bw.uint32(data.extData.length)
      bw.replace(data.extData, true)
      if(data.E)
      {
        bw.string(data.EXIF, {length:4})
        bw.uint32(data.EXIFData.length)
        bw.replace( data.EXIFData, true)
      }
      if(data.X)
      {
        bw.string(data.XMP, {length:4})
        bw.uint32(data.XMPMetaData.length)
        bw.replace(data.XMPMetaData, true)
      }
      break;
    default:
      break;
  }
  bw.trim() //remove any remaining bytes
  bw.goto(4)
  bw.uint32le(bw.size - 8) //write file size
  return bw.return()
}

Common Functions

Common functions for setup, movement, manipulation and math shared by both.

Bit field

Parse value as a bit field. There are 32 functions from bit1 to bit32 and can be signed or unsigned (with a u at the start) and in little or big endian order (be or le at the end).

Note: Remaining bits are dropped when returning to a byte read. Example, after using bit4() then ubyte(), the read locations drops the remaining 4 bits after bit4() when reading ubyte(). Any bit reading under 8 will always be unsigned.

Byte

Parse value as a byte (aka int8). Can be signed or unsigned (with a u at the start).

Short

Parse value as a int16 (aka short or word). Can be signed or unsigned (with a u at the start) and in little or big endian order (be or le at the end).

Half Float

Parse value as a half float (aka half). Can be in little or big endian order (be or le at the end).

Integer

Parse value as a int32 (aka int, long or double). Can be signed or unsigned (with a u at the start) and in little or big endian order (be or le at the end).

Float

Parse value as a float. Can be in little or big endian order (be or le at the end).

Quadword

Parse value as a int64 (aka quad or bigint). Can be signed or unsigned (with a u at the start) and in little or big endian order (be or le at the end).

Double Float

Parse value as a double float (aka dfloat). Can be in little or big endian order (be or le at the end).

Strings

Parse a string in any format. Be sure to use options object for formatting unless using a preset. Strings with larger than 1 byte character reads can use be or le at the end for little or big endian.

Presents include C or Unicode, Ansi and multiple pascals.

Acknowledgements

This project was born from the desire to have a single library that could both read and write in binary with common named functions. Having been using tools like Binary-parser, QuickBMS and 010 Editor in the past, I wanted something I could translate quickly to a Node app and then use in a web site without having to redo work.

I'm happy to connect and grow this library if others find it useful. Pull requests or bug reports are welcome!

License

MIT