KafkaJSMetrics

A lightweight KafkaJS library that provides simple access to key metrics within your code. KafkaJSMetrics enables custom breakpoint alerts and logging of key metrics, allowing for client health to be monitored within applications.

Table of Contents

• Introduction
• Features
• Getting Started
  • Installation
  • Usage
• Community
• Contributors
• Other

Introduction

Apache Kafka is a distributed event streaming platform that is designed to handle high performance data pipelines in real-time. KafkaJS is a lightweight, user-friendly Apache Kafka client library that runs on Node.js, providing a simple and reliable way to produce and consume messages from Kafka clusters. Event emitters called instrumentation events are built into KafkaJS and provide information about the performance and behavior of a user's Kafka producers and consumers. These events are designed to help developers monitor and troubleshoot their Kafka applications. However, incorporating instrumentation events into code can be tedious and repetitive.

KafkaJSMetrics is a continuously-growing open source product that is designed to simplify and shorten the metric exposing processes. By committing to 100% Javascript alongside KafkaJS, the library leverages already built-in instrumentation events without relying on other third-party applications or requiring any additional installations. The library gives developers simple access to key metrics required to monitor the health of a Kafka instance.

Features

Below are some of the key features from KafkaJSMetrics. For a full explanation of library functionality, visit our docs.

Key features:

• Access to key consumer/producer/admin metrics, including

  • offset lag
  • messages consumed
  • total number of consumer/producer/admin currently connected
  • heartbeat
  • partition assignment
  • see docs for complete metrics

• Custom Breakpoint Alerts and On/Off metrics logging options, including

  • heartbeat monitor
  • request queue size monitor
  • request pending duration monitor
  • offset lag monitor
  • see docs for complete Breakpoint Alert and metrics logging options

  ---

For information about the web browser visualizer display feature that pairs with the kafkaJSMetrics library, visit our main visualizer branch

Getting Started

KafkaJSMetrics works with your KafkaJS client. For information on setting up your KafkaJS client, visit KafkaJS.

• Installation and Usage walk through setting up the KafkaJSMetrics library for use within your KafkaJS Client code

Installation

Install the kafkajs-metrics package from npm

npm install kafkajs-metrics

Usage

KafkaJSMetrics works with your existing KafkaJS client. More information about KafkaJS at kafka.js.org.

1. Require KafkaJSMetrics in your KafkaJS Client file to access metricize function

2. Pass your KafkaJS Client instance into the metricize function (sample KafkaJS Client provided)

   • All consumer, producer, and admin instances created from this client will now have metrics and metrics-associated functionality

3. Run your KafkaJS Client file

node client.js

Turn on logging methods

Visit the docs for a full list of available logging methods

Example:

Prints to console:

Turn on breakpoint alerts

Visit the docs for a full list of available breakpoint alert methods

Example:

Prints to console:

Community

As KafkaJSMetrics continues to grow, we welcome and encourage feedback! We are also always trying to improve our code so we actively welcome all contributions! If you would like to be a part of this product, please follow the steps below.

1. Fork this repo of KafkaJSMetrics and then clone it to your machine

2. Check out into the branch that you want to make contributions in

   1. Contributions to the library:

      git checkout library

3. Create a feature branch

git checkout -b [feature/AmazingFeature]

4. Add all your changes with the add command

git add [all your changes]

5. Commit all your changes

git commit -m '<your comment>'

6. Make sure the library branch is up to date

   1. Check back out to the library branch

   git checkout library

   2. Pull down the latest version

   git pull origin library

   3. Check back out to your feature branch

   git checkout [feature/AmazingFeature]

7. Merge your branch with library

git merge library

8. Resolve any merge conflicts
9. Run unit tests and make sure all tests pass

npm test

10. Push up your branch with the command

git push origin [feature/AmazingFeature]

11. Finally, open a pull request for us to review!

Contributors

• Rebecca Anderson - GitHub | LinkedIn
• Paul Davey - GitHub | LinkedIn
• Ranan Hui - GitHub | LinkedIn
• Amanda Smith - GitHub | LinkedIn
• Josh Tesoro - GitHub | LinkedIn

Other

Important links:

• Full Documentation
• Website
• Check out our latest Medium article

KafkaJSMetrics is licensed under the Mozilla Public License version 2.0 and are available for free. We have no affiliation with and are not endorsed by either The Apache Software Foundation or the developers of KafkaJS.