npm package discovery and stats viewer.

Discover Tips

  • General search

    [free text search, go nuts!]

  • Package details

    pkg:[package-name]

  • User packages

    @[username]

Sponsor

Optimize Toolset

I’ve always been into building performant and accessible sites, but lately I’ve been taking it extremely seriously. So much so that I’ve been building a tool to help me optimize and monitor the sites that I build to make sure that I’m making an attempt to offer the best experience to those who visit them. If you’re into performant, accessible and SEO friendly sites, you might like it too! You can check it out at Optimize Toolset.

About

Hi, 👋, I’m Ryan Hefner  and I built this site for me, and you! The goal of this site was to provide an easy way for me to check the stats on my npm packages, both for prioritizing issues and updates, and to give me a little kick in the pants to keep up on stuff.

As I was building it, I realized that I was actually using the tool to build the tool, and figured I might as well put this out there and hopefully others will find it to be a fast and useful way to search and browse npm packages as I have.

If you’re interested in other things I’m working on, follow me on Twitter or check out the open source projects I’ve been publishing on GitHub.

I am also working on a Twitter bot for this site to tweet the most popular, newest, random packages from npm. Please follow that account now and it will start sending out packages soon–ish.

Open Software & Tools

This site wouldn’t be possible without the immense generosity and tireless efforts from the people who make contributions to the world and share their work via open source initiatives. Thank you 🙏

© 2024 – Pkg Stats / Ryan Hefner

flatten-tweet

v0.0.5

Published

Make Twitter v2 API data a bit easier to process

Downloads

13

Readme

flatten-tweet

Install

npm install flatten-tweet

Use

flatten-tweet is designed to be used to transform the data received from the Twitter v2 API to make it easier to work with. It is not a client itself, so you'll need to use something that gives you the raw JSON response (e.g. like twitter-v2).

The value in having a separate function to flatten data is that other libraries and/or projects can import and use it as needed no matter how the data is being fetched. flatten-tweet also exports an object EVERYTHING which you can use to request all expansions and fields, whichever library you might be using.

When new expansions or includes are added changed flatten-tweet will be updated to include them. semver will be used to version the NPM package, with respect to the flatten-tweet JavaScript API (which is pretty simple) but also with respect to the Twitter API. So if a new expansion or field is added you can expect a minor version increment. Any backwards incompatible changes will get a new major version.


const Twitter = require('twitter-v2')
const { flatten, EVERYTHING }  = require('flatten-tweet')

const client = new Twitter(credentials)
const data = await client.get('tweets/search/recent', {
  ...EVERYTHING
  query: 'from:jack'
})

// flatten it!

const flattened = flatten(data)

Huh?

You might be wondering why you would ever want to use flatten-tweet. Let me explain...

Rather than including all the available information for a tweet Twitter's v2 API now includes expansions which allow you to request additional information like user, media, quoted/retweeted tweets, etc. This additional information is not included inline in the tweet objects themselves but as an additional JSON stanza of includes.

The includes stanza cuts down on potential duplication in the response. For example the information about a tweet that is retweeted many times in the tweets contained in the response, is only included once, and everywhere else in the response it is referenced using its tweet id.

But the includes stanza also makes processing retrieved Twitter data a bit more difficult. If you are looking at a tweet and want to know when the tweet author's account was created you can't simply use tweet.author.created_at. You need to get the tweet.author_id and then search through the includes.users list looking for it, and then use that.

To simplify this process the flatten() function included here will take an API response, and will copy the includes into all the tweets that reference them.

So now when you are processing a tweet instead of having to work with a tweet that looks (in abbreviated form) like this:

{
  "data": [
    {
      "id": "21",
      "text": "just setting up my twttr",
      "author_id": "12"
    }
  ],
  "includes": {
    "users": [
      {
        "id": "123",
        "username": "jack",
        "created_at": "2006-03-21T20:50:14.000Z"
      }
    ]
  }
}

after flattening you will have:

{
  "data": [
    {
      "id": "21",
      "text": "just setting up my twttr",
      "author_id": "12",
      "author": {
        "id": "12",
        "username": "jack",
        "created_at": "2006-03-21T20:50:14.000Z",
      }
    }
  ]
}

This JavaScript is a port of the equivalent Python function in twarc.

Test

If you want to run the test suite you'll need to install twitter-v2 and set BEARER_TOKEN in your environment first. Then you can:

npm run test