Webshot

Webshot is a lightweight, no-nonsense, and opinionated package that allows you to capture screenshots of specific elements on a webpage. It provides an easy-to-implement way to automate the process of capturing screenshots while handling various options such as user agent, proxy, URL blocking, delays, and clicks before capturing.

Key Features

• Multiple Screenshots: Webshot allows you to capture screenshots of multiple elements on a webpage in a single call. You can specify the CSS selectors of the elements to capture in an array.

• URL Blocking: Webshot allows you to block requests to specific URLs during page load. You don't need to provide complete URLs; just a part of the URL is sufficient. Any URL that includes the specified part will be blocked.

• Element Deletion: Webshot allows you to delete elements before capturing the screenshot. This is useful if you want to remove elements that should not be a part of the screenshot.

• Best Practices Enforcement: Webshot enforces best practices by requiring a user agent and a proxy to be specified. This ensures that the screenshot capture process is performed in a controlled and secure manner.

• Base64 Output: Webshot outputs the captured screenshot as a base64-encoded string. This makes it easy to store, transmit, or process the screenshot data further.

• In-built AWS S3 Support: You simply pass the AWS credentials along with the bucket name and key, and webshot will add the 'screenshotUrl' property to the response.

Installation

To install the Webshot package, run the following command:

npm install webshot-ts

Usage

To use Webshot in your project, import the webshot function from the package and call it with the desired options.

import webshot from "webshot-ts";

const options = {
  elementsToCapture: ["#capture-element-1", "#capture-element-2"],
  urlsToBlockDuringPageLoad: ["example.com", "ads.com"],
  delayBeforeCapture: 2000,
  clicksBeforeCapture: [
    {
      elementToClick: "#click-me",
      delay: 1000,
    },
  ],
  elementsToBeDeletedBeforeCapture: [".useless-element"],
  launchOptions: {
    userAgent:
      "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.82 Safari/537.36",
    proxy: {
      username: "your_username",
      password: "your_password",
      host: "proxy.example.com:8080",
    },
    args: ["--no-sandbox", "--disable-setuid-sandbox"],
    uploadToS3Bucket: true,
    s3Config: {
      bucket: "your-bucket-name",
      key: "your-key-name",
      region: "ap-south-1",
      accessKeyId: "AKIAY72222222222222",
      secretAccessKey: "AKIAY72222222222222",
    },
  },
};

const screenshots = await webshot("https://example.com", options);

console.log(screenshots);

// Output: [
//   {
//     selector: '//div[@id="capture-element-1"]',
//     base64: '<base64-encoded-string-for-element-1>',
//     screenshotUrl: '<s3-url-for-element-1>'
//   },
//   {
//     selector: '//div[@id="capture-element-2"]',
//     base64: '<base64-encoded-string-for-element-2>',
//     screenshotUrl: '<s3-url-for-element-2>'
//   }
// ]

Options

The webshot function accepts an options object with the following properties:

• elementsToCapture (required): An array of CSS selectors of the elements to be captured.
• urlsToBlockDuringPageLoad (optional): An array of URL parts to block during page load. Any URL that includes any of the specified parts will be blocked.
• delayBeforeCapture (optional): The delay (in milliseconds) before capturing the screenshot.
• clicksBeforeCapture (optional): An array of objects representing the elements to click before capturing the screenshot. The clicks occur in the order they are specified in the array. Each object should have the following properties:
  • elementToClick: The CSS selector of the element to click.
  • delay: The delay (in milliseconds) after clicking the element.
• elementsToBeDeletedBeforeCapture (optional): An array of JQuery selectors of elements to delete before capturing the screenshot.
• launchOptions (required): An object containing the launch options for Puppeteer. It should have the following properties:
  • userAgent (required): The user agent to use.
  • proxy (required): An object representing the proxy configuration. It should have the following properties:
    • username: The proxy username.
    • password: The proxy password.
    • host: The proxy host and port (e.g., 'proxy.example.com:8080').
  • args (optional): An array of additional command-line arguments to pass to the browser instance.
  • uploadToS3Bucket (optional): A boolean value indicating whether the screenshot should be uploaded to an S3 bucket.
  • s3Config (optional): An object containing the S3 configuration. It should have the following properties:
    • bucket: The bucket name.
    • key: The key name.
    • region: The region.
    • accessKeyId: The access key ID.
    • secretAccessKey: The secret access key.

Error Handling

The webshot function throws an error in the following cases:

• If the required options are missing or invalid.
• If the screenshot capture fails.

Dependencies

Webshot relies on the following dependencies:

• Puppeteer: A library for controlling a headless Chrome or Chromium browser.
• Proxy Chain: A library for creating anonymous proxies.

Buffer Type

Webshot automatically captures the screenshot in '.png' format. In later releases, we'll support the ability for you to provide other image formats.

Example

Here's an example of how to use Webshot to capture a screenshot of a specific element on a webpage:

import fs from "fs";
import path from "path";
import webshot from "webshot-ts";

const options = {
  elementsToCapture: ["#capture-element-1", "#capture-element-2"],
  urlsToBlockDuringPageLoad: ["example.com", "ads.com"],
  delayBeforeCapture: 2000,
  clicksBeforeCapture: [
    {
      elementToClick: "#click-me",
      delay: 1000,
    },
  ],
  elementsToBeDeletedBeforeCapture: [".useless-element"],
  launchOptions: {
    userAgent:
      "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.82 Safari/537.36",
    proxy: {
      username: "your_username",
      password: "your_password",
      host: "proxy.example.com:8080",
    },
    args: ["--no-sandbox", "--disable-setuid-sandbox"],
  },
};

try {
  const screenshots = await webshot("https://example.com", options);

  // Save the screenshots locally
  for (const screenshot of screenshots) {
    const { selector, base64 } = screenshot;
    const fileName = `screenshot_${selector.replace(/[^a-zA-Z0-9]/g, "_")}.png`;
    const filePath = path.join(__dirname, fileName);

    fs.writeFileSync(filePath, Buffer.from(base64, "base64"));
    console.log(`Screenshot saved in the directory : ${filePath}`);
  }
} catch (error) {
  console.error("Error capturing screenshot:", error);
}

In this example, Webshot navigates to https://example.com, blocks any URLs that include 'example.com' or 'ads.com', clicks on the element specified in clicksBeforeCapture, deletes the first element with the class .useless-element, waits for the specified delay, and then captures screenshots of the elements specified in elementToCapture array. Webshot returns an array of objects, each representing a screenshot with the selector and the base64-encoded string as its properties.