@mapbox/wikibush
v0.0.5
Published
wikibush -------- Put wikidata dumps into `rbush`. To get around blowing up memory of a single process, this sets up shards and datafiles for seeking on disk rather than storing full feature data in memory. Goal is to allow full wikidata dump to be access
Downloads
10
Maintainers
Keywords
Readme
wikibush
Put wikidata dumps into rbush
. To get around blowing up memory of a single process, this sets up shards and datafiles for seeking on disk rather than storing full feature data in memory. Goal is to allow full wikidata dump to be accessible via a single interface.
Usage example
# stream wikidata dump into wikibush storage
mkdir workdir
cat wikidata-20161107-geo.ldjson.bz2 - | bzip2 -d | wikibush-writer --workdir=workdir --tolerance=0.001
# read features around point back out
wikibush-reader --workdir=workdir --lon=-73.9975 --lat=40.7308