npm package discovery and stats viewer.

Discover Tips

  • General search

    [free text search, go nuts!]

  • Package details

    pkg:[package-name]

  • User packages

    @[username]

Sponsor

Optimize Toolset

I’ve always been into building performant and accessible sites, but lately I’ve been taking it extremely seriously. So much so that I’ve been building a tool to help me optimize and monitor the sites that I build to make sure that I’m making an attempt to offer the best experience to those who visit them. If you’re into performant, accessible and SEO friendly sites, you might like it too! You can check it out at Optimize Toolset.

About

Hi, 👋, I’m Ryan Hefner  and I built this site for me, and you! The goal of this site was to provide an easy way for me to check the stats on my npm packages, both for prioritizing issues and updates, and to give me a little kick in the pants to keep up on stuff.

As I was building it, I realized that I was actually using the tool to build the tool, and figured I might as well put this out there and hopefully others will find it to be a fast and useful way to search and browse npm packages as I have.

If you’re interested in other things I’m working on, follow me on Twitter or check out the open source projects I’ve been publishing on GitHub.

I am also working on a Twitter bot for this site to tweet the most popular, newest, random packages from npm. Please follow that account now and it will start sending out packages soon–ish.

Open Software & Tools

This site wouldn’t be possible without the immense generosity and tireless efforts from the people who make contributions to the world and share their work via open source initiatives. Thank you 🙏

© 2024 – Pkg Stats / Ryan Hefner

movement-data-toolkit

v1.3.0

Published

Tools and utilities for working with Uber Movement data

Downloads

15

Readme

oclif Version License


🚀 Getting Started

Using this toolkit requires Node.js to be installed on your machine, see Downloading and installing Node.js and npm. We recommend using nvm as described in that document, or brew install node for homebrew users. Using the official GUI installer from nodejs.org may lead to permissions issues when installing new commands.

Once node is installed, you can install and run the toolkit as shown in the sections below.

🛠 Usage

$ npm install -g movement-data-toolkit
$ mdt COMMAND
running command...
$ mdt (-v|--version|version)
movement-data-toolkit/1.3.0 darwin-x64 node-v8.16.2
$ mdt --help [COMMAND]
USAGE
  $ mdt COMMAND
...

🕹 Commands

mdt clear-cache

Some commands in this toolkit download files to a temporary directory. These files can be rather large, and it can be helpful to clear this data to reclaim disk space.

USAGE
  $ mdt clear-cache

EXAMPLE
  $ mdt clear-cache

mdt create-geometry-file [CITY] [YEAR]

Creates a geojson file that contains the geometries for each road segment in the specified city, as well as both the corresponding OSM identifiers for each road segment.

USAGE
  $ mdt create-geometry-file [CITY] [YEAR]

ARGUMENTS
  CITY  Slug of the city to download data for. You can find this value in the URL when accessing data on the Movement
        website.

  YEAR  Movement publishes mapping files and road geometries yearly, specify the year you would like to use here

OPTIONS
  -o, --output=output        File to write output to (writes to stdout by default)

  --bbox=bbox                Features outside of this bounding box will be removed from output. Should be of the form
                             minX,minY,maxX,maxY like "-10.0,-10.0,10.0,10.0" (see
                             https://tools.ietf.org/html/rfc7946#section-5)

  --driving-side=LEFT|RIGHT  [default: RIGHT] Road geometries are "twinned" in the output file to visualize speeds in
                             both directions, to do this accurately you should use --driving-side="LEFT" for places like
                             the UK or Australia.

  --format=GEOJSON|CSV       [default: GEOJSON] Format to use for the output file

  --geom-format=GEOJSON|WKT  [default: GEOJSON] Changes format of geometries in output file (only supported with
                             --format=CSV)

  --log-level=INFO|DEBUG     [default: INFO] What types of logs should be emitted?

  --[no-]twin                Whether or not to "twin" road geometries in the output file – this duplicates and offset
                             two-way roads so you can visualize data in both directions. Defaults to true.

DESCRIPTION
  The produced file can help avoid the need to download and manipulate OSM geometries from the web yourself.

EXAMPLE
  $ mdt create-geometry-file cincinnati 2018 > my-output-file.geojson

mdt help [COMMAND]

display help for mdt

USAGE
  $ mdt help [COMMAND]

ARGUMENTS
  COMMAND  command to show help for

OPTIONS
  --all  see all commands in CLI

See code: @oclif/plugin-help

mdt speeds-to-geojson [CITY] [STARTDATE] [ENDDATE]

Downloads and processes Movement speeds data into a geojson file that can be visualized with Kepler.gl or other geospatial visualization tools.

USAGE
  $ mdt speeds-to-geojson [CITY] [STARTDATE] [ENDDATE]

ARGUMENTS
  CITY       Slug of the city to download data for. You can find this value in the URL when accessing data on the
             Movement website.

  STARTDATE  Fetch data starting from this date

  ENDDATE    Fetch data up to and including this date

OPTIONS
  -o, --output=output        File to write output to (writes to stdout by default)

  --bbox=bbox                Features outside of this bounding box will be removed from output. Should be of the form
                             minX,minY,maxX,maxY like "-10.0,-10.0,10.0,10.0" (see
                             https://tools.ietf.org/html/rfc7946#section-5)

  --driving-side=LEFT|RIGHT  [default: RIGHT] Road geometries are "twinned" in the output file to visualize speeds in
                             both directions, to do this accurately you should use --driving-side="LEFT" for places like
                             the UK or Australia.

  --log-level=INFO|DEBUG     [default: INFO] What types of logs should be emitted?

  --[no-]twin                Whether or not to "twin" road geometries in the output file – this duplicates and offset
                             two-way roads so you can visualize data in both directions. Defaults to true.

DESCRIPTION
  Note: Becuase our speeds data CSVs have an hourly granularity, the results produced by this command are technically an 
  average of averages and may differ slightly from results shown in the app at movement.uber.com.

EXAMPLE
  $ mdt speeds-to-geojson cincinnati 2018-01-01 2018-01-31 > my-output-file.geojson

mdt speeds-transform [TYPE] [CITY] [STARTDATE] [ENDDATE]

Downloads and transforms Movement Speeds data, filtering by date range and optional bounding box and producing a CSV with speed values mapped to OSM identifiers.

USAGE
  $ mdt speeds-transform [TYPE] [CITY] [STARTDATE] [ENDDATE]

ARGUMENTS
  TYPE       Either "historical" for the monthly historical time series or "stats" for the quarterly statistics file.

  CITY       Slug of the city to download data for. You can find this value in the URL when accessing data on the
             Movement website.

  STARTDATE  Fetch data starting from this date

  ENDDATE    Fetch data up to and including this date

OPTIONS
  -o, --output=output     File to write output to (writes to stdout by default)

  --bbox=bbox             Features outside of this bounding box will be removed from output. Should be of the form
                          minX,minY,maxX,maxY like "-10.0,-10.0,10.0,10.0" (see
                          https://tools.ietf.org/html/rfc7946#section-5)

  --log-level=INFO|DEBUG  [default: INFO] What types of logs should be emitted?

EXAMPLE
  $ mdt speeds-transform historical cincinnati 2018-01-01 2018-01-31 > filtered.csv

📝 License

The code in this package is licensed under the MIT License, see LICENSE file at the root of this project.

Data specific to Uber Movement – including speeds or travel times – is licensed under the Creative Commons, Attribution Non Commercial License

Data from OpenStreetMap – including road geometries or other OSM metadata and identifiers – is licensed under the Open Data Commons Open Database License (ODbL) by the OpenStreetMap Foundation (OSMF).

⚠️ Troubleshooting

Out of memory Errors (OOM)

Occasionally you may run into an error that looks something like this:

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory

Some of the scripts in this toolkit are processing a lot of data. By default Node.js uses 512mb, but we can increase that limit with another command line flag:

export NODE_OPTIONS="--max_old_space_size=8192" && mdt [command]

The above command would allow the node process to use up to 8gb of memory while it's running.

EACCES permissions errors when installing

The default Node installer from nodejs.org installs Node in a way that can cause permissions issues when you run npm packages globally.

You can either re-install node using a node version manager or change npms global package directory to one you have access to.

👋 Issues & Feedback

Questions? Issues? Feedback? Let us know on Movement's Contact Form.