Graph Network Monitor

The network monitor runs at a regular interval and checks that a number of conditions pass. If any of these tests fails it will issue an alert through Pager Duty.

Alerts are deduplicated by test name and epoch when it was raised to avoid multiple notifications.

Configuration

The monitor uses the network subgragh, deployed contracts and protocol config files as source of information.

To correctly configure the monitor you'll need to:

• Create a config file to describe the tests you want to run. Use --config to pass the path to the file (default: config.yaml).
• Create a providers file to describe the providers you want to use. Use --providers to pass the path to the file (default: providers.yaml).
• Specify the network subgraph using --network-subgraph (default: graphprotocol/graph-network-mainnet).

Running from NPM packages

Install the network monitor as NPM package, using:

npm install -g @graphprotocol/network-monitor

After that, it can be run with the following command:

# Network monitor
graph-network-monitor start ...

Running locally

Clone the repository and run yarn install followed by yarn prepare.

After that, it can be run with the following command:

# Mainnet
ts-node src/index.ts start

# Other network
ts-node src/index.ts start \
  --network-subgraph <NETWORK_SUBGRAPH_NAME_> \
  --config <CONFIG_FILE> \
  --providers <PROVIDERS_FILE>

Running with Docker

Build the docker image and run it using:

# Build the image
docker build -t graph-network-monitor .

# Run on mainnet
docker run \
  --env NETWORK_MONITOR_ETHEREUM=<ETHEREUM_RPC> \
  --env NETWORK_MONITOR_ARBITRUM=<ARBITRUM_RPC> \
  graph-network-monitor

# Run on other network
docker run \
  --env NETWORK_MONITOR_ETHEREUM=<ETHEREUM_RPC> \
  --env NETWORK_MONITOR_ARBITRUM=<ARBITRUM_RPC> \
  --env NETWORK_MONITOR_NETWORK_SUBGRAPH=<NETWORK_SUBGRAPH_NAME> \
  --env NETWORK_MONITOR_CONFIG=<CONFIG_FILE> \
  graph-network-monitor

Usage

Network Monitor

$ graph-network-monitor start --help

Run monitoring tests

Logging
  --metrics-port  Port to serve Prometheus metrics at   [number] [default: 7300]
  --log-level     Log level                          [string] [default: "debug"]

Network Subgraph
  --network-subgraph  Subgraph name to query the network subgraph from
            [string] [required] [default: "graphprotocol/graph-network-mainnet"]

Monitor
  --interval   Number of seconds interval for monitoring [number] [default: 300]
  --config     Config file        [string] [required] [default: "./config.yaml"]
  --providers  Providers config file
                               [string] [required] [default: "./providers.yaml"]
  --dry-run    Don't send alerts if tests fail        [boolean] [default: false]

Options:
  --version  Show version number                                       [boolean]
  --help     Show help                                                 [boolean]

Adding tests

The monitor will execute each configured test sequentially, and raise alerts whenever these tests return a non-ok status.

Each test is a simple function with a defined interface.

Interface

To add a new test, create a function under the /tests folder that receives the following parameters:

  logger: Logger,
  contracts: NetworkContracts,
  testFunctionConfig: TestFunctionConfig,

• You can use logger as a structured logging facility.
• The contracts objects has all the contract interfaces connected to the Ethereum provider using ethers.js
• testFunctionConfig provides an interface with access to:
  • networkSubgraph: the name of the network subgraph to query
  • providers contains credentials to connect to JSON RPC nodes in different networks

This function must return a TestResult with the following interface:

export interface TestResult {
  ok: boolean
  summary?: string
  details?: Record<string, string>
}

• Whenever ok is set to false, the monitor will raise an alert for that test.
• summary should contain a descriptive message about the error. If left empty, the monitor will create a summary based on the function name.
• Use details to store any additional information about how the test failed for debugging purposes.

Enable

Finally, import the function and add it to the monitor.addTests() function located in the /command/monitor.ts file.

Copyright

Copyright © 2021 The Graph Foundation

Licensed under the MIT license.