intercom-to-mongodb

Import all data from an Intercom app into a MongoDB deployment

Install

npm install -g intercom-to-mongodb

CLI

Create a .env file:

INTERCOM_ACCESS_TOKEN=<YOUR_INTERCOM_ACCESS_TOKEN>
MONGODB_URL=<YOUR_MONGODB_URL>

> intercom-to-mongodb --help
Usage: intercom-to-mongodb [options] [command]


Commands:

  users           import all user data
  tags            import data for all tags
  segments        import data for all segments
  conversations   import data for all conversations
  admins          import data for all admins
  events          import all event data

Options:

  -h, --help              output usage information
  -V, --version           output the version number

Import all users

# Also imports all tags, segments, conversations, admins.
DEBUG=* intercom-to-mongodb users;

Import all events

# Must have users imported first.  Have to iterate through user ids and
# fetch event streams individually.  Takes a really long time. Just hit
# ctrl+c to kill it when you have enough data.
DEBUG=* intercom-to-mongodb events;

If you used mongodb://localhost:27017/intercom as MONGODB_URL in your .env file:

> mongo 'mongodb://localhost:27017/intercom'
show collections
admins
conversations
events
segments
tags
users

Todo

• [x] Initial get of 10k users from Intercom API
• [x] Initial get of all admins from Intercom API
• [x] Initial get of all tags from Intercom API
• [x] Initial get of all segments from Intercom API
• [x] Initial get of all conversations from Intercom API
• [x] Use the Scroll API to get all users
• [x] Reshape user location shape and db.users.createIndex( { "coordinates": "2dsphere" } )
• [x] Cast user *_at epoch time fields from Intercom API as ISODate
• [x] $lookup to inline user.tags from tags.name
• [x] $lookup to inline user.segments from tags.segments
• [x] Get all events for all users
• [ ] Support incremental resync from Intercom API
• [ ] For each conversation, get the full conversation from API which has conversations_parts populated. Once completed, the conversation document will be interesting, metadata alone
• [ ] Sentiment analysis on conversations_parts contents, e.g. https://github.com/wooorm/retext-sentiment

License

Apache 2.0