@syncpoint/compact-cache-bundle

ESRI's compact cache V2 bundle files contain pre-rendered raster or vector tiles. This nodejs module provides read-only access to a bundle file in order to read the header, the tile index and the tiles.

ESRI provides a technical description of the compact cache bundle file V2 structure in their github repository. In contrast to the statement in the description the tiles stored in the bundle are not limited to raster tiles.

ESRI's Vector Tile Package (VTPK) contain bundle files as well. The tiles within the bundle files comply with Mapbox's Vector Tile Specification v2.0.

Installation

npm install @syncpoint/compact-cache-bundle --save

Usage

Please download any of the bundle files that are part of the ESRI sample compact cache.

The example below will extract the tiles and write them to the file system. The JPEG file format is hard-coded and should be derived from the file conf.xml:

    <TileImageInfo xsi:type="typens:TileImageInfo">
        <CacheTileFormat>JPEG</CacheTileFormat>
        <CompressionQuality>75</CompressionQuality>
        <Antialiasing>false</Antialiasing>
    </TileImageInfo>

'use strict'

const bundle = require('@syncpoint/compact-cache-bundle')

const file = 'PATH_TO_ESRI_Compact_Cache/L02/R0000C0000.bundle'

/* you don't have to use it the asynchronous way */
const fs = require('fs')
const { promisify } = require('util')
const fsOpen = promisify(fs.open)
const fsClose = promisify(fs.close)
const fsWrite = promisify(fs.write)

    ; (async () => {
        let fd
        try {
            fd = await fsOpen(file, 'r')

            const bundleHeader = await bundle.header(fd)
            console.dir(bundleHeader)

            const records = await bundle.tileIndex(fd)
            console.dir(records)

            const bundleOffset = bundle.offset(file)

            /* please read ESRI's technical specification for details */
            for (let r = 0; r < records.length; r++) {
                let tile = await bundle.tiles(fd, records[r])

                const absRow = bundleOffset.rowOffset + records[r].row
                const absColumn = bundleOffset.columnOffset + records[r].column
                /* please check the tile format upfront */
                const fileName = `${absRow}-${absColumn}.jpeg`

                let outputFd
                try {
                    outputFd = await fsOpen(fileName, 'w')
                    await fsWrite(outputFd, tile)
                    console.log(`created file ${fileName}`)
                }
                catch (error) {
                    console.dir(error)
                }
                finally {
                    await fsClose(outputFd)
                }
            }
        }
        catch (error) {
            console.dir(error)
        }
        finally {
            await fsClose(fd)
        }
    })()

API

@syncpoint/compact-cache-bundle currently provides three functions to access the tiles stored in the bundle files. All functions are asynchronous (return a promise) because they need to access data in the filesystem.

Only the function offset is synchronous.

Header

bundle.header(fileDescriptor) returns an object that contains all header data as specified in ESRI's document.

{ 
  version: 3,
  recordCount: 16384,
  maxTileSize: 43588,
  offsetByteCount: 5,
  slackSpace: 0,
  fileSize: 588369,
  userHeaderOffset: 40,
  userHeaderSize: 131092,
  legacy1: 3,
  legacy2: 16,
  legacy3: 16384,
  legacy4: 5,
  indexSize: 131072
}

Tile Index Records

bundle.tileIndex(fileDescriptor) returns an array of Tile Index Records.

The array contains only records that have a tileSize greater than zero and are augmented with the tile row and column information. Row and column are absolute within the bundle only. Each bundle file is part of a coordinate system and the offset of the bundle is encoded in its name (see Bundle offset).

[ 
  { row: 0, column: 0, tileOffset: 267680, tileSize: 29182 },
  { row: 0, column: 1, tileOffset: 296866, tileSize: 17156 },
  { row: 0, column: 2, tileOffset: 314026, tileSize: 14641 },
  { row: 0, column: 3, tileOffset: 328671, tileSize: 22656 },
  { row: 1, column: 0, tileOffset: 351331, tileSize: 33511 },
  { row: 1, column: 1, tileOffset: 384846, tileSize: 20788 },
  { row: 1, column: 2, tileOffset: 405638, tileSize: 24476 },
  { row: 1, column: 3, tileOffset: 430118, tileSize: 43309 },
  { row: 2, column: 0, tileOffset: 473431, tileSize: 18128 },
  { row: 2, column: 1, tileOffset: 491563, tileSize: 14995 },
  { row: 2, column: 2, tileOffset: 506562, tileSize: 14602 },
  { row: 2, column: 3, tileOffset: 521168, tileSize: 31591 },
  { row: 3, column: 0, tileOffset: 552763, tileSize: 12698 },
  { row: 3, column: 1, tileOffset: 565465, tileSize: 9139 },
  { row: 3, column: 2, tileOffset: 574608, tileSize: 5293 },
  { row: 3, column: 3, tileOffset: 579905, tileSize: 8464 } 
]

Tile

bundle.tile(fileDescriptor, tileIndexRecord)returns a Buffer that contains the tile data. The tileIndexRecord must be obtained by the function tileIndex.

Bundle offset

bundle.offset(bundleFileName) calculates the row and column offset that is encoded in the bundle filename (RrrrrCcccc.bundle). If the tiles should be exported to the filesystem or to a MBTiles database, the bundle offset must be added to the row and column data of the tiles.

    { rowOffset: 0, columnOffset: 0 }

Dependencies

None, just plain nodejs and Javascript.

nodejs version

Please use the current nodejs LTS version.

License

Copyright (c) Syncpoint GmbH. All rights reserved.

Licensed under the MIT License.