npm package discovery and stats viewer.

Discover Tips

  • General search

    [free text search, go nuts!]

  • Package details

    pkg:[package-name]

  • User packages

    @[username]

Sponsor

Optimize Toolset

I’ve always been into building performant and accessible sites, but lately I’ve been taking it extremely seriously. So much so that I’ve been building a tool to help me optimize and monitor the sites that I build to make sure that I’m making an attempt to offer the best experience to those who visit them. If you’re into performant, accessible and SEO friendly sites, you might like it too! You can check it out at Optimize Toolset.

About

Hi, 👋, I’m Ryan Hefner  and I built this site for me, and you! The goal of this site was to provide an easy way for me to check the stats on my npm packages, both for prioritizing issues and updates, and to give me a little kick in the pants to keep up on stuff.

As I was building it, I realized that I was actually using the tool to build the tool, and figured I might as well put this out there and hopefully others will find it to be a fast and useful way to search and browse npm packages as I have.

If you’re interested in other things I’m working on, follow me on Twitter or check out the open source projects I’ve been publishing on GitHub.

I am also working on a Twitter bot for this site to tweet the most popular, newest, random packages from npm. Please follow that account now and it will start sending out packages soon–ish.

Open Software & Tools

This site wouldn’t be possible without the immense generosity and tireless efforts from the people who make contributions to the world and share their work via open source initiatives. Thank you 🙏

© 2024 – Pkg Stats / Ryan Hefner

@freetube/yt-comment-scraper

v6.2.0

Published

Scrapes the comments of any YouTube video without YouTube API access. Uses the default YouTube Ajax calls to get the comment data.

Downloads

169

Readme

YouTube Comment Scraper NodeJS Documentation

This NodeJS library scrapes the comments of the YouTube provided HTML comment data without any API usage order by date descending (so most recent first). It is developed for and tailored towards easy usage with FreeTube but can be used with any other project as well.

This library does not require any API keys, with the attached maximum quotas, but instead might take longer to receive the required data.

The library works as long as YouTube keeps its web page layout the same. Therefore, there is no guarantee that this library will work at all times. If this library should not work at some point, please create an issue and let me know so that I can take a look into it. Pull requests are also welcomed in this case.

Installation

npm install @freetube/yt-comment-scraper --save

Usage

Set your instance with the following syntax. Use the second line instead if you're using modules / Typescript

const ytcm = require("@freetube/yt-comment-scraper")

import ytcm from '@freetube/yt-comment-scraper'

getComments(payload)

Returns a list of objects containing comments from the next page of the video.

  • payload (Object) (Required) - An object containing the various options
    • videoId (String) (Required) - The video ID to grab comments from
    • sortByNewest (Boolean) (Optional) - Grabs newest comments when true. Grabs top comments when false
    • continuation (String) (Optional) - The token from a previous call to continue grabbing comments
    • mustSetCookie (Boolean) (Optional) - The flag should be set to true when cookies are not handled by your application (e.g. Electron) already
    • httpsAgent (Object) (Optional) - Allows to specify all kind of different agent data (see NodeJS documentation or 3rd party packages like node-https-proxy-agent for options like proxies)
    const https = require('https');
    const options = {...};  
    const agent = new https.Agent(options);
const payload = {
  videoId: videoId, // Required
  sortByNewest: sortByNewest,
  continuation: continuation,
  mustSetCookie: false,
  httpsAgent: agent
}

ytcm.getComments(payload).then((data) =>{
    console.log(data);
}).catch((error)=>{
    console.log(error);
});

Returned Data

The data is returned as an object with a list of comment objects and a continuation token (if more comments exist)

// The data is a list of objects containing the following attributes:
{
  total: Number | null, // Total number of comments on the video
  comments: [
    {
      commentId: String, // Id of comment
      authorId: String, // Id of user that made the comment
      author: String, // Name of the channel that made the comment
      authorThumb: Array [ // An Array of thumbnails of the channel profile
        {
          width: Number,
          height: Number,
          url: String
        }
      ],
      edited: Boolean, // If the comment has been edited or not
      text: String, // The text content of the comment
      likes: String, // The amount of likes the comment has, numbers > 1000 displayed with 1.9K, 2K...
      time: String, // The time the comment was published. Written as "One day ago"
      numReplies: Number, // The number of replies found for the comment
      isOwner: Boolean, // If the video channel made the comment
      isHearted: Boolean, // If the video channel hearted the comment
      isPinned: Boolean, // If the video channel pinned the comment
      isVerified: Boolean,
      isOfficialArtist: Boolean,
      hasOwnerReplied: Boolean, // If the video channel replied to the comment
      isMember: Boolean, // Whether the user that made the comment is a paid member or not
      memberIconUrl: String | null, // URL of the member icon
      customEmojis: Array [ // An Array of custom emojis used in the comment
        {
          text: String, // the text alias for the emoji
          emojiThumbnails: Array [ // An Array of thumbnails of the custom emoji
            {
              width: Number,
              height: Number,
              url: String
            }
          ]
        }
      ]
      replyToken: String // The continuation token needed for getCommentReplies()
    }
  ],
  continuation: String | null // The continuation token needed to get more comments from getComments()
}

getCommentReplies(payload)

Returns a list of objects containing replies from a given comment.

  • payload (Object) (Required) - An object containing the various options
    • videoId (String) (Required) - The video ID to grab comments from
    • replyToken (String) (Required) - The reply token from a comment object of getComments() or the continuation string from a previous call to getCommentReplies()
    • mustSetCookie (Boolean) (Optional) - The flag should be set to true when cookies are not handled by your application (e.g. Electron) already
    • httpsAgent (Object) (Optional) - Allows to specify all kind of different agent data (see NodeJS documentation or 3rd party packages like node-https-proxy-agent for options like proxies)
const parameters = {videoId: 'someId', replyToken: 'HSDcjasgdajwSdhAsd', mustSetCookie: true, httpsAgent: null};
ytcm.getCommentReplies(parameters).then((data) =>{
    console.log(data);
}).catch((error)=>{
    console.log(error);
});

Returned Data

The data is returned as a list of objects (seen below).

// The data is a list of objects containing the following attributes:
  comments: [
  {
    commentId: String, // Id of comment
    authorId: String, // Id of user that made the comment
    author: String, // Name of the channel that made the comment
    authorThumb: Array [ // An Array of thumbnails of the channel profile
      {
        width: Number,
        height: Number,
        url: String
      }
    ],
    edited: Boolean, // If the comment has been edited or not
    text: String, // The text content of the comment
    likes: String, // The amount of likes the comment has, numbers > 1000 displayed with 1.9K, 2K...
    time: String, // The time the comment was published. Written as "One day ago"
    numReplies: Number, // The number of replies found for the comment
    isOwner: Boolean, // If the video channel made the comment
    isHearted: Boolean, // If the video channel hearted the comment
    isPinned: false,
    isVerified: Boolean,
    isOfficialArtist: Boolean,
    hasOwnerReplied: false,
    isMember: Boolean, // Whether the user that made the comment is a paid member or not
    memberIconUrl: String | null, // URL of the member icon
    customEmojis: Array [ // An Array of custom emojis used in the comment
      {
        text: String, // the text alias for the emoji
        emojiThumbnails: Array [ // An Array of thumbnails of the custom emoji
          {
            width: Number,
            height: Number,
            url: String
          }
        ]
      }
    ]
    replyToken: null
  }],
  continuation: String | null // The continuation token needed (instead of replyToken) to get more replies from getCommentReplies()

Credits

Thanks to egbertbouman for his/her Python project which guided this project through the difficult HTTP calls.