gdal-exprtk
v2.0.0
Published
ExprTk.js plugin for gdal-async
Downloads
2
Readme
gdal-exprtk
This is a plugin that adds support for ExprTk.js expressions to gdal-async.
It allows for truly asynchronous background processing performing only O(1) operations on the V8 main thread. Multiple operations run in parallel and never block the event loop.
Requires [email protected]
and [email protected]
.
Installation
To use as a library in a project:
npm install --save gdal-exprtk
Install globally to use the command-line version:
sudo npm install -g gdal-exprtk
Usage
Command-line utility
The command-line utility supports both JS functions and ExprTk expressions. It uses parallel processing whenever possible.
With ExprTk expression:
gdal_calc.js -i AROME_D2m_10.tiff=d -i AROME_T2m_10.tiff=t
-o CLOUDBASE.tiff \
-e -c '125*(t-d)' -f GTiff -t Float64
With JS function:
gdal_calc.js -i AROME_D2m_10.tiff=d -i AROME_T2m_10.tiff=t
-o CLOUDBASE.tiff \
-j -c 'return 125*(t-d);' -f GTiff -t Float64
With multiband input files and automatic variable naming:
gdal_calc.js -i multiband.tif@1 -i multiband.tif@2
-o output.tiff \
-e -c '(a+b)/2' -f GTiff -t Float64
Producing a multiband output file:
gdal_calc.js -i multiband.tif@1=x -i multiband.tif@2=y
-o output.tiff \
-e -c '(x+y)/2' -c '(x-y)/2' -f GTiff -t Float64
With NoData
<->Nan
conversion
If a NoData
value is specified for the output file, then all input NoData
values will be converted to NaN
before invoking the user function and all NaN
values returned from the user function will be written as the NoData
value. This works only if the output data type is a floating point type. [email protected]
supports converting integer types to NaN
, [email protected]
requires that all input files have a floating point type for this to work.
gdal_calc.js -i AROME_D2m_10.tiff=d -i AROME_T2m_10.tiff=t
-o CLOUDBASE.tiff \
-e -c '125*(t-d)' -f GTiff -t Float64 -n -1e-38
Reading a JS function from a file
gdal_calc.js
can use both a default and a named export. The arguments order must be given explicitly.
espy.js
:
module.exports = {};
module.exports.espy = (t, td) => (125 * (t - td));
module.exports.espy.args = ['t', 'td'];
Then:
gdal_calc.js -i AROME_D2m_10.tiff=td -i AROME_T2m_10.tiff=t
-o CLOUDBASE.tiff \
-j -c =./espy.js@espy -f GTiff -t Float64 -n -1e-38
Reading an ExprTk expression from a file
espy.exprtk
:
125 * (t - td)
Then:
gdal_calc.js -i AROME_D2m_10.tiff=td -i AROME_T2m_10.tiff=t
-o CLOUDBASE.tiff \
-e -c =./espy.exprtk -f GTiff -t Float64 -n -1e-38
With calcAsync
import * as gdal from 'gdal-async';
import { Float64 as Float64Expression } from 'exprtk.js';
import { calcAsync } from 'gdal-exprtk';
const T2m = await gdal.openAsync('AROME_T2m_10.tiff'));
const D2m = await gdal.openAsync('AROME_D2m_10.tiff'));
const size = await T2m.rasterSizeAsync;
const filename = `/vsimem/AROME_CLOUDBASE.tiff`;
const dsCloudBase = gdal.open(filename, 'w', 'GTiff',
size.x, size.y, 1, gdal.GDT_Float64);
// Espy's estimation for cloud base height
const espyExpr = new Float64Expression('125 * (T2m - D2m)');
// This is required for the automatic NoData handling
// (it will get converted from/to NaN)
(await cloudBase.bands.getAsync(1)).noDataValue = -1e38;
// Mapping to ExprTk.js variables is by (case-insensitive) name
// and does not depend on the order
await calcAsync({
T2m: await T2m.bands.getAsync(1),
D2m: await D2m.bands.getAsync(1)
}, await cloudBase.bands.getAsync(1), espyExpr, { convertNoData: true });
As a Node.js Streams-compatible Transform
import * as gdal from 'gdal-async';
import { Float64 as Float64Expression } from 'exprtk.js';
import { RasterTransform } from 'gdal-exprtk';
import { finished as _finished } from 'stream';
import { promisify } from 'util';
const finished = promisify(_finished);
// Espy's estimation for cloud base height (lifted condensation level)
// LCL = 125 * (T2m - Td2m)
// where T2m is the temperature at 2m and Td2m is the dew point at 2m
const expr = new Float64Expression('125 * (T2m - D2m)');
const dsT2m = gdal.open('AROME_T2m_10.tiff'));
const dsD2m = gdal.open('AROME_D2m_10.tiff'));
const filename = `/vsimem/AROME_CLOUDBASE.tiff`;
const dsCloudBase = gdal.open(filename, 'w', 'GTiff',
dsT2m.rasterSize.x, dsT2m.rasterSize.y, 1, gdal.GDT_Float64);
// Mapping to ExprTk.js variables is by (case-insensitive) name
// and does not depend on the order
const mux = new gdal.RasterMuxStream({
T2m: dsT2m.bands.get(1).pixels.createReadStream(),
D2m: dsD2m.bands.get(1).pixels.createReadStream()
});
const ws = dsCloudBase.bands.get(1).pixels.createWriteStream();
const espyEstimation = new RasterTransform({ type: Float64Array, expr });
mux.pipe(espyEstimation).pipe(ws);
await finished(ws);
dsCloudBase.close();
API
Table of Contents
RasterTransform
Extends stream.Transform
A raster Transform stream
Applies an ExprTk.js Expression on all data elements.
Input must be a gdal.RasterMuxStream
calcAsync provides a higher-level interface for the same feature
Parameters
options
RasterTransformOptions?options.exr
(Function | Expression) Function to be applied on all data
Examples
const dsT2m = gdal.open('AROME_T2m_10.tiff'));
const dsD2m = gdal.open('AROME_D2m_10.tiff'));
const dsCloudBase = gdal.open('CLOUDBASE.tiff', 'w', 'GTiff',
dsT2m.rasterSize.x, dsT2m.rasterSize.y, 1, gdal.GDT_Float64);
const mux = new gdal.RasterMuxStream({
T2m: dsT2m.bands.get(1).pixels.createReadStream(),
D2m: dsD2m.bands.get(1).pixels.createReadStream()
});
const ws = dsCloudBase.bands.get(1).pixels.createWriteStream();
// Espy's estimation for cloud base height (lifted condensation level)
// LCL = 125 * (T2m - Td2m)
// where T2m is the temperature at 2m and Td2m is the dew point at 2m
const expr = new Float64Expression('125 * (t - td)');
const espyEstimation = new RasterTransform({ type: Float64Array, expr });
mux.pipe(espyEstimation).pipe(ws);
RasterTransformOptions
Extends stream.TransformOptions
Properties
expr
Expression Function to be applied on all data
CalcOptions
Type: object
Properties
convertNoData
boolean?progress_cb
ProgressCb?
ProgressCb
Type: Function
Parameters
complete
number
calcAsync
Compute a new output band as a pixel-wise function of given input bands
This is an alternative implementation of gdal_calc.py
.
It is identical to the one in gdal-async except that it accepts an ExprTK.js expression as function instead of a JS function.
It's main advantage is that it does not solicit the V8's main thread for any
operation that is not O(1) - all computation is performed in background
async threads. The only exception is the convertNoData
option with [email protected]
which is implemented in JS. [email protected]
supports C++ conversion of NoData
to NaN.
It internally uses a RasterTransform which can also be used directly for a finer-grained control over the transformation.
There is no sync version.
Parameters
inputs
Record<string, gdal.RasterBand> An object containing all the input bandsoutput
gdal.RasterBand Output raster bandexpr
Expression ExprTk.js expressionoptions
CalcOptions? Optionsoptions.convertNoData
boolean Input bands will have their NoData pixels converted toNaN and a NaN output value of the given function will be converted to a NoData pixel, provided that the output raster band has itsRasterBand.noDataValue
set (optional, defaultfalse
)options.progress_cb
ProgressCb Progress callback (optional, defaultundefined
)
Examples
const T2m = await gdal.openAsync('TEMP_2M.tiff'));
const D2m = await gdal.openAsync('DEWPOINT_2M.tiff'));
const size = await T2m.rasterSizeAsync
const cloudBase = await gdal.openAsync('CLOUDBASE.tiff', 'w', 'GTiff',
size.x, size.y, 1, gdal.GDT_Float64);
(await cloudBase.bands.getAsync(1)).noDataValue = -1e38
// Espy's estimation for cloud base height
const espyFn = (t, td) => 125 * (t - td);
await calcAsync({
t: await T2m.bands.getAsync(1),
td: await D2m.bands.getAsync(1)
}, cloudBase.bands.getAsync(1), espyFn, { convertNoData: true });
Returns Promise<void>
toPixelFunc
Get a gdal-async
pixel function descriptor for this ExprTk.js
expression.
Every call of this function produces a permanent GDAL descriptor that cannot
be garbage-collected, so it must be called only once per ExprTk.js
expression.
As of GDAL 3.4, GDAL does not allow unregistering a previously registered function.
The returned object can be used across multiple V8 instances (ie worker threads).
gdal-async
does not support multiple V8 instances.
If the V8 instance containing the ExprTk.js
expression is destroyed, further attempts
to read from Datasets referencing the function will produce an exception.
Parameters
expression
Expression
Examples
// This example will register a new GDAL pixel function called sum2
// that requires a VRT dataset with 2 values per pixel
const gdal = require('gdal-async);
const Float64Expression = require('exprtk.js').Float64;
const { toPixelFunc } = require('gdal-exprtk');
const sum2 = new Float64Expression('a + b');
gdal.addPixelFunc('sum2', toPixelFunc(sum2));
Returns gdal.PixelFunction