npm package discovery and stats viewer.

Discover Tips

  • General search

    [free text search, go nuts!]

  • Package details

    pkg:[package-name]

  • User packages

    @[username]

Sponsor

Optimize Toolset

I’ve always been into building performant and accessible sites, but lately I’ve been taking it extremely seriously. So much so that I’ve been building a tool to help me optimize and monitor the sites that I build to make sure that I’m making an attempt to offer the best experience to those who visit them. If you’re into performant, accessible and SEO friendly sites, you might like it too! You can check it out at Optimize Toolset.

About

Hi, 👋, I’m Ryan Hefner  and I built this site for me, and you! The goal of this site was to provide an easy way for me to check the stats on my npm packages, both for prioritizing issues and updates, and to give me a little kick in the pants to keep up on stuff.

As I was building it, I realized that I was actually using the tool to build the tool, and figured I might as well put this out there and hopefully others will find it to be a fast and useful way to search and browse npm packages as I have.

If you’re interested in other things I’m working on, follow me on Twitter or check out the open source projects I’ve been publishing on GitHub.

I am also working on a Twitter bot for this site to tweet the most popular, newest, random packages from npm. Please follow that account now and it will start sending out packages soon–ish.

Open Software & Tools

This site wouldn’t be possible without the immense generosity and tireless efforts from the people who make contributions to the world and share their work via open source initiatives. Thank you 🙏

© 2024 – Pkg Stats / Ryan Hefner

@koopjs/provider-elasticsearch

v3.7.1

Published

Elastic Search Provider for Koop

Downloads

256

Readme

Koop Provider for Elastic Search

This provider allow Koop to fetch and transform data from an Elasticsearch instance.

Install

From command line in the folder containing the project run:

npm install @koopjs/provider-elasticsearch --save

In your Koop instance, register the provider like:

  const esProvider = require('@koopjs/provider-elasticsearch')
  koop.register(esProvider)

The latest version of this provider can be installed with the Koop CLI. See the Koop CLI docs for details on setting up a Koop instance and add providers with the CLI.

Running

To suppress KoopJS warnings from the console output, run with an environment variable of KOOP_WARNINGS="suppress". In powershell, this will look like: $env:KOOP_WARNINGS="suppress" ; node main.js

Elastic Search Connect Environment Variable

Setting an environment variable of KOOP_ENV_KEYS allows saving of usernames and passwords to connect to elastic search without having to store them in the configuration file. The format for this variable is esId,user,password||esId2,user2,password2. The esId is the id given to the connection in the configuration file as shown below in the Basic Config File Structure section.

Command Line

npm start

Sub-Layers

By default, all services only have a single layer, as defined in the index configuration. It is possible to configure one or more sub-layers in the subLayers section of the configuration. Each of these sub layers can be a predefined sub layer or you can create your own.

Basic Config File Structure

The following is a sample configuration file showing most capabilities

{
  "esConnections": {
    "esNameForServiceURLs": {
      "id": "esNameForServiceURLs",
      "protocol": "http://",
      "port": 9200,
      "hosts": [
        "localhost"
      ],
      "shapeIndices": {
        "states": {
          "geometryField": "geometry",
          "geometryType": "Polygon",
          "joinField": "NAME"
        }
      },
      "indices": {
        "myService1": {
          "index": "indexName",
          "allowMultiPoint": false,
          "caching": {
            "enabled": true,
            "seconds": 600
          },
          "geometryField": "geometry.coordinates",
          "geometryType": "geo_point",
          "returnFields": [
            "lastUpdate",
            "createdAt",
            "name"
          ],
          "dateFields": [
            "lastUpdate",
            "createdAt"
          ],
          "idField": "OBJECTID",
          "subLayers": [
            {"name": "geohash"}
          ],
          "maxResults": 1000,
          "maxLayerInfoResults": 1
        },
        "tableService": {
          "index": "indexNoShape",
          "allowMultiPoint": false,
          "isTable": true,
          "returnFields": [
            "state",
            "county",
            "date"
          ],
          "dateFields": [
            "date"
          ],
          "maxResults": 1000
        },
        "joinService": {
          "index": "indexToJoin",
          "allowMultiPoint": false,
          "returnFields": [
            "date",
            "country",
            "state.name"
          ],
          "dateFields": [
            "date"
          ],
          "subLayers": [],
          "shapeIndex": {
            "name": "states",
            "joinField": "state.name"
          },
          "maxResults": 1000
        },
        "polyService": {
          "index": "polygonIndex",
          "allowMultiPoint": false,
          "geometryField": "geometry",
          "geometryType": "MultiPolygon",
          "reversePolygons": true,
          "returnFields": [
            "properties.date",
            "properties.count",
            "properties.state_name"
          ],
          "dateFields": [
            "properties.date"
          ],
          "subLayers": [],
          "maxResults": 1000
        }
      }
    }
  }
}
Configuration Options
  • shapeIndices includes all indices that will be used for their shapes on other services. All fields are mandatory.
  • indices this object has a property for every service to be created. The name of the property will be the service name.

Index Properties

  • index is the name of the ElasticSearch index
  • isTable treat this service as a table, ignoring geometry
  • allowMultiPoint is only important for point services and allows more than one point per feature
  • caching allows local in-memory caching. Only use this for services that have no more documents than your maxResults setting
  • geometryField is the full location of the geometry
  • geometryType can be geo_point, Point, MultiPoint, Polyline, MultiLineString, Polygon and MultiPolygon
  • reversePolygons if the stored polygons do not follow the right-hand rule setting this to true will fix this. Polygons that do not follow the right-hand rule will not be displayed as feature services without setting this to true.
  • returnFields includes all fields that will be returned by the feature service
  • dateFields includes any return fields that should be treated as dates
  • idField if the index includes a field that can be treated as the OBJECTID field, this should be set
  • subLayers any sub-layers to be used. Provided sub layers are geohash_aggregation and geotile_aggregation but custom subLayers may be used here as well.
  • maxResults the maximum features returned from a single request
  • maxLayerInfoResults maximum number of features to return for a layer info request (Normally 1) NOTE: Do not use this on a layer with caching enabled.
  • shapeIndex if the service will join to a shapeIndex for geometry list the name of the index (defined in shapeIndices) and the joinField from this index.
  • vectorLayerID the 0 based index of the subLayer to use for vector output
  • vectorStyle an optional map-box style to use for the vector layer

Sub-Layer Configurations

All subLayer configurations must include a name to map to a registered subLayer class. Other than that an options object contains any other needed information.

  • offset which is used by both provided subLayers is the offset value passed in by esri clients. This value is in meters and is larger for higher zoom levels.
  • aggregationFields can be used to pass in sub-aggregations to elastic search. The fields will be added to the output for display in pop-ups and/or for use in symbology.

GeoHash

precision is between 1-12 for geohash.

{
  "name": "geohash_aggregation",
  "options": {
    "tileConfig": [
      { "precision": 8, "offset": 16 },
      { "precision": 6, "offset": 10000 },
      { "precision": 4, "offset": 32000 },
      { "precision": 2, "offset": 640000 }
    ],
    "aggregationFields": {
      "speed_avg": {
        "avg": { "field":  "speed"}
      },
      "speed_min": {
        "min": { "field": "speed"}
      }
    }
  }
}

GeoTile

precision is between 0-29 for geotile.

{
  "name": "geotile_aggregation",
  "options": {
    "tileConfig": [
      { "precision": 22, "offset": 16 },
      { "precision": 16, "offset": 10000 },
      { "precision": 10, "offset": 32000 },
      { "precision": 4, "offset": 640000 }
    ],
    "aggregationFields": {
      "speed_avg": {
        "avg": { "field":  "speed"}
      },
      "speed_min": {
        "min": { "field": "speed"}
      }
    }
  }
}

BETA - OpenSearch Support

Support for connections to Amazon OpenSearch clusters has been added. This functionality is new and had not been thoroughly tested. Please log any issues encountered when using/testing.

OpenSearch Configuration

The OpenSearch configurations follow the same pattern as elasticsearch connections and can even be included in the same configuration file. Create an osConnections object sibling to the esConnections object within the configuration file. From there the configuration is the same as with elasticsearch connections except for cluster connection information. All OpenSearch connection information is stored within a connectConfig object. An example is given below.

{
  "osConnections": {
    "osCluster1": {
      "id": "osCluster1",
      "connectConfig": {
        "node": "http://localhost:9200"
      },
      "indices": {
        // same as with elastic configuration
      }
    }
  }
}

Additional Index Configurations

mapReturnValues is an object that can contain keys that are field names that in turn have their own keys equal to field values mapped to object values. Example below. __defaultmapping is not required.

{
    "mapReturnValues": {
        "fieldName": {
            "returnedValue1": "mappedValue",
            "__defaultmapping": "defaultValue"
        }
    }
}

IN DEVELOPMENT

mapFieldNames can be used to specify a different return field than what is specified within returnFields

{
    "mapFieldNames": {
        "fieldName": "mappedFieldName"
    }
}