@openstapps/es-mapping-generator
v3.0.0
Published
Tool to convert TypeScript Interfaces to Elasticsearch Mappings
Downloads
10
Keywords
Readme
@openstapps/es-mapping-generator
Tools to convert and validate StAppsCore
What is the ES Mapping Generator for?
The ES Mapping Generator is for converting TypeScript interfaces to Elasticsearch mappings, with support for Integer and Float types, Dates, etc.
Why is this a separate package?
The ES Mapping Generator relies on an outdated version of TypeDoc, that can't be updated. Because of that, it then also relies on an outdated version of TypeScript, resulting in a cascade effect things that can't be updated. Because of that, we decided to isolate it to the best of our abilities.
How to use the Elasticsearch Mapping generator
The mapping generator is intended to be used by the backend directly, but it can also be used to generate these files manually.
Generating the mapping files by hand
To generate the mapping files by hand, you need a local copy of the core-tools and the core, both need to be built first. After that you can run
node lib/cli.js mapping path/to/core path/to/destination ignoredTag1,ignoredTag2,ignoredTag3
If you don't pass in any ignored tags, you will likely be prompted with errors, despite the core being absolutely correct.
This is because there are some tags that are not relevant to Elasticsearch, but the program has no direct way to tell
which ones simply lack an implementation and which ones can be ignored. Currently the ignored tags include
minlength
, pattern
and see
.
Generating the mapping directly from another TypeScript program
This is the more easy way, and it gives you direct access to the generated mapping as a (mostly typesafe) object. To
use it, call generateTemplate
, However you will first need to generate a ProjectReflection of the core you are working
with. If you use the core as a dependency, you can for example use
const map = generateTemplate(
getProjectReflection(resolve('node_modules', '@openstapps', 'core', 'src')),
ignoredTags,
false,
);
to generate the mappings. Note that the result object contains both a list of errors in map.errors
and the actual mapping
in map.template
. You can also specify whether you want the generator to show any errors while generating the mappings
in the console in the last parameter, true
(default) being show errors and false
to suppress them. That said it is very
easy to replace all type: "MISSING_PREMAP"
, type: "PARSE_ERROR"
, type: "TYPE_CONFLICT"
with dynamic: true
(you
can take these exactly like written here and run a replace over the file).
Fixing a generated mapping by hand
If you get errors when generating the mappings, the mappings might not work, however they will still be generated to the programs best efforts. Most small issues can be fixed after the mapping was generated as a temporary solution and without changing anything in the mapper's code. This however requires some understanding of how mappings work.
The output of the program can easily reach 25.000 lines, but you can find errors quickly by searching for MISSING_PREMAP
,
PARSE_ERROR
and TYPE_CONFLICT
. When you reach them you can then manually replace them with your code.
As a last resort you can also replace all errors with dynamic types, this should get the mapping working, but it is NOT RECOMMENDED as a fix other than using it locally.