github_exporter

Export various metrics including insights and traffic metrics about github repositories from the GitHub API, to a Prometheus compatible endpoint.

About rate limits and abuse warnings

github_exporter use GitHub GraphQL API V4 and REST API V3.

Both APIs have a separate rate-limit. The default limit is 5000 requests/hour with an personal access token.

To prevent rate limit or abuse errors I highly recommend to configure a higher scrape interval. If you are still in trouble you also should enable the spread parameter to spread requests across the scrape interval.

More information:

• https://developer.github.com/v3/#rate-limiting
• https://developer.github.com/v4/guides/resource-limitations/

GitHub Token

A Github token is highly recommend.

How to get an token: https://help.github.com/en/github/authenticating-to-github/creating-a-personal-access-token-for-the-command-line#creating-a-token

More information:

• https://developer.github.com/v3/#authentication
• https://developer.github.com/v4/guides/forming-calls/#authenticating-with-graphql

Usage

index.js -t <token> -i 600 [ -s ] [ -l :: ] [ -p 9171 ] [ -o organization ] [ -u user ] [ -r owner/repository ]

Authentication:
      --auth-strategy  GitHub auth strategy  [required] [choices: "token", "oauth-app", "app", "action"] [default: "token"]
  -a, --auth           GitHub auth data (e.g.: token)  [required]

Scape settings:
  -i, --interval  scrape interval  [number] [default: 600]
  -s, --spread    spread request over interval  [boolean] [default: false]
  -S, --scraper   enable or disable scraper  [array] [default: ["collaborators","summarize","rate-limit","contributors","status","traffic-clones","traffic-top-paths","traffic-top-referrers","traffic-views"]]

Scape targets:
  -o, --organization  GitHub organization to scrape. Can be defined multiple times or comma separated list  [array] [default: []]
  -u, --user          GitHub users to scrape. Can be defined multiple times or comma separated list  [array] [default: []]
  -r, --repository    GitHub's repositories to scrape. Can be defined multiple times or comma-separated list. Format: <owner>/<repo>  [array] [default: []]

Bind options:
      --host  address to bind exporter  [default: "::"]
  -p, --port  port to bind exporter  [number] [default: 9171]

Log options:
      --log-level    log level of application  [choices: "error", "warn", "info", "http", "verbose", "debug", "silly"] [default: "info"]
      --log-file     path to log file
      --log-console  log to console  [boolean] [default: true]
      --log-format   log format of application  [default: "cli"]

Options:
      --version  Show version number  [boolean]
      --config   Path to JSON config file
  -h, --help     Show help  [boolean]

Environment variable support. Prefix: GITHUB_EXPORTER, e.g. --auth == GITHUB_EXPORTER_AUTH

for more information, find our manual at https://github.com/jkroepke/github_exporter

.env file config

If case you don't want to define certain options like secrets you can define them in a .env file.

More information about .env file:

• https://github.com/motdotla/dotenv#usage

Authentication

github_exporter supports

• token
• oauth-app
• app
• action

authentication types.

token

Just pass your personal token as an argument

github_exporter --auth ghp_xxx

oauth-app

Pass all properties as JSON object

github_exporter --auth-strategy oauth-app --auth '{"clientType": "oauth-app", "clientId": "1234567890abcdef1234", "clientSecret": "1234567890abcdef1234567890abcdef12345678"}'

app

Pass all properties as JSON object. Authenticate as in app is currently not supported

# Authenticate as OAuth App (client ID/client secret)
github_exporter --auth-strategy app --auth '{"appId": 1, "privateKey": "-----BEGIN PRIVATE KEY-----\n...", "clientId": "1234567890abcdef1234", "clientSecret": "1234567890abcdef1234567890abcdef12345678", "installationId": 123}'

Start the exporter

Docker:

docker run --name github_exporter -d \
    --restart=always -p 9171:9171 \
    -e GITHUB_EXPORTER_AUTH=<secret> \
    -e GITHUB_EXPORTER_ORGANIZATION=org1,org2 \
    -e GITHUB_EXPORTER_USER=user1,user2 \
    -e GITHUB_EXPORTER_REPOSITORY=jkroepke/github_exporter,jkroepke/helm-secrets,jkroepke/2Moons \
    jkroepke/github-exporter

A compose file is available, too.

node package manager

npm install -g github_exporter

github_exporter --token=<GH-Token> --repository=jkroepke/github_exporter

Metrics

# HELP github_repo_collaborator_total total amount of collaborators for given repository
# TYPE github_repo_collaborator_total gauge
github_repo_collaborator_total{owner="jkroepke",repository="jkroepke/github_exporter",affiliation="direct"} 1
# HELP github_repo_scraped Successfully scraped a repository
# TYPE github_repo_scraped gauge
github_repo_scraped{owner="jkroepke",repository="jkroepke/github_exporter"} 1
# HELP github_repo_info Information about given repository
# TYPE github_repo_info gauge
github_repo_info{owner="jkroepke",repository="jkroepke/github_exporter",licence="MIT License",language="JavaScript"} 1
# HELP github_repo_is_disabled Is repository disabled
# TYPE github_repo_is_disabled gauge
github_repo_is_disabled{owner="jkroepke",repository="jkroepke/github_exporter"} 0
# HELP github_repo_is_fork Is repository a fork
# TYPE github_repo_is_fork gauge
github_repo_is_fork{owner="jkroepke",repository="jkroepke/github_exporter"} 0
# HELP github_repo_is_archived Is repository archived
# TYPE github_repo_is_archived gauge
github_repo_is_archived{owner="jkroepke",repository="jkroepke/github_exporter"} 0
# HELP github_repo_created_at repository creation date
# TYPE github_repo_created_at gauge
github_repo_created_at{owner="jkroepke",repository="jkroepke/github_exporter"} 1580644595000
# HELP github_repo_updated_at repository last update date
# TYPE github_repo_updated_at gauge
github_repo_updated_at{owner="jkroepke",repository="jkroepke/github_exporter"} 1667288697000
# HELP github_repo_pushed_at repository last push date
# TYPE github_repo_pushed_at gauge
github_repo_pushed_at{owner="jkroepke",repository="jkroepke/github_exporter"} 1667507246000
# HELP github_repo_has_issues has repository issues enabled
# TYPE github_repo_has_issues gauge
github_repo_has_issues{owner="jkroepke",repository="jkroepke/github_exporter"} 1
# HELP github_repo_has_projects has repository issues enabled
# TYPE github_repo_has_projects gauge
github_repo_has_projects{owner="jkroepke",repository="jkroepke/github_exporter"} 1
# HELP github_repo_has_wiki has repository issues enabled
# TYPE github_repo_has_wiki gauge
github_repo_has_wiki{owner="jkroepke",repository="jkroepke/github_exporter"} 1
# HELP github_repo_is_template is repository a template
# TYPE github_repo_is_template gauge
github_repo_is_template{owner="jkroepke",repository="jkroepke/github_exporter"} 0
# HELP github_repo_issues_total Issues for given repository
# TYPE github_repo_issues_total gauge
github_repo_issues_total{owner="jkroepke",repository="jkroepke/github_exporter",status="open"} 2
# HELP github_repo_pull_request_total Pull requests for given repository
# TYPE github_repo_pull_request_total gauge
github_repo_pull_request_total{owner="jkroepke",repository="jkroepke/github_exporter",status="open"} 1
# HELP github_repo_watchers_total Total number of watchers/subscribers for given repository
# TYPE github_repo_watchers_total gauge
github_repo_watchers_total{owner="jkroepke",repository="jkroepke/github_exporter"} 2
# HELP github_repo_stars_total Total number of Stars for given repository
# TYPE github_repo_stars_total gauge
github_repo_stars_total{owner="jkroepke",repository="jkroepke/github_exporter"} 9
# HELP github_repo_fork_total Total number of forks for given repository
# TYPE github_repo_fork_total gauge
github_repo_fork_total{owner="jkroepke",repository="jkroepke/github_exporter"} 3
# HELP github_repo_commits Total number of commits for given repository
# TYPE github_repo_commits gauge
github_repo_commits{owner="jkroepke",repository="jkroepke/github_exporter"} 0
# HELP github_repo_tags_total Total number of tags for given repository
# TYPE github_repo_tags_total gauge
github_repo_tags_total{owner="jkroepke",repository="jkroepke/github_exporter"} 10
# HELP github_repo_branches_total Total number of branches for given repository
# TYPE github_repo_branches_total gauge
github_repo_branches_total{owner="jkroepke",repository="jkroepke/github_exporter"} 3
# HELP github_repo_packages Total number of packages for given repository
# TYPE github_repo_packages gauge
github_repo_packages{owner="jkroepke",repository="jkroepke/github_exporter"} 0
# HELP github_repo_downloads Total number of releases for given repository
# TYPE github_repo_downloads gauge
github_repo_downloads{owner="jkroepke",repository="jkroepke/helm-secrets",release="v4.1.1",url="https://github.com/jkroepke/helm-secrets/releases/download/v4.1.1/helm-secrets.tar.gz"} 2754
# HELP github_repo_releases Total number of releases for given repository
# TYPE github_repo_releases gauge
github_repo_releases{owner="jkroepke",repository="jkroepke/github_exporter"} 9
# HELP github_repo_disk_usage The number of kilobytes this repository occupies on disk.
# TYPE github_repo_disk_usage gauge
github_repo_disk_usage{owner="jkroepke",repository="jkroepke/github_exporter"} 1053
# HELP github_repo_discussions_total Total number of discussions for given repository
# TYPE github_repo_discussions_total gauge
github_repo_discussions_total{owner="jkroepke",repository="jkroepke/github_exporter"} 0
# HELP github_repo_deployments_total Total number of deployments for given repository
# TYPE github_repo_deployments_total gauge
github_repo_deployments_total{owner="jkroepke",repository="jkroepke/github_exporter"} 0
# HELP github_repo_environments_total Total number of deployments for given repository
# TYPE github_repo_environments_total gauge
github_repo_environments_total{owner="jkroepke",repository="jkroepke/github_exporter"} 0
# HELP github_repo_mentionable_users_total Total number of mentionable users for given repository
# TYPE github_repo_mentionable_users_total gauge
github_repo_mentionable_users_total{owner="jkroepke",repository="jkroepke/github_exporter"} 2
# HELP github_repo_collaborators_total Total number of collaborators for given repository
# TYPE github_repo_collaborators_total gauge
github_repo_collaborators_total{owner="jkroepke",repository="jkroepke/github_exporter"} 1
# HELP github_repo_milestones_total Total number of collaborators for given repository
# TYPE github_repo_milestones_total gauge
github_repo_milestones_total{owner="jkroepke",repository="jkroepke/github_exporter"} 0
# HELP github_repo_milestone_percent Percent of milestone inside the given repository
# TYPE github_repo_milestone_percent gauge
github_repo_milestone_percent{owner="jkroepke",repository="jkroepke/2Moons",title="1.8",number="1"} 0
# HELP github_repo_milestone_state State of milestone inside the given repository
# TYPE github_repo_milestone_state gauge
github_repo_milestone_state{owner="jkroepke",repository="jkroepke/2Moons",title="1.8",number="1"} 0
# HELP github_repo_milestone_issues_total Total issue count of milestone inside the given repository
# TYPE github_repo_milestone_issues_total gauge
github_repo_milestone_issues_total{owner="jkroepke",repository="jkroepke/2Moons",title="1.8",number="1"} 0
# HELP github_repo_vulnerabilities_total vulnerabilities for given repository
# TYPE github_repo_vulnerabilities_total gauge
github_repo_vulnerabilities_total{owner="jkroepke",repository="jkroepke/github_exporter"} 26
# HELP github_repo_languages_size return repo size by langauges for given repository
# TYPE github_repo_languages_size gauge
github_repo_languages_size{owner="jkroepke",repository="jkroepke/github_exporter",language="JavaScript"} 40596
# HELP github_rate_limit_limit GitHub API rate limit limit
# TYPE github_rate_limit_limit gauge
github_rate_limit_limit{api="core"} 5000
# HELP github_rate_limit_remaining GitHub API rate limit remaining
# TYPE github_rate_limit_remaining gauge
github_rate_limit_remaining{api="core"} 2144
# HELP github_rate_limit_reset GitHub API rate limit reset
# TYPE github_rate_limit_reset gauge
github_rate_limit_reset{api="core"} 1667510585
# HELP github_repo_contributors Total number of releases for given repository
# TYPE github_repo_contributors gauge
github_repo_contributors{owner="jkroepke",repository="jkroepke/github_exporter"} 4
# HELP github_repo_status status for the default branch for given repository
# TYPE github_repo_status gauge
# HELP github_repo_traffic_clones Total number of clones for given repository
# TYPE github_repo_traffic_clones gauge
github_repo_traffic_clones{owner="jkroepke",repository="jkroepke/helm-secrets",week="latest"} 21421
# HELP github_repo_traffic_unique_clones Total number of clones for given repository
# TYPE github_repo_traffic_unique_clones gauge
github_repo_traffic_unique_clones{owner="jkroepke",repository="jkroepke/helm-secrets",week="latest"} 10131
# HELP github_repo_traffic_clones_avg Avenge number of clones for given repository
# TYPE github_repo_traffic_clones_avg gauge
github_repo_traffic_clones_avg{owner="jkroepke",repository="jkroepke/helm-secrets"} 59436
# HELP github_repo_traffic_unique_clones_avg Avenge number of clones for given repository
# TYPE github_repo_traffic_unique_clones_avg gauge
github_repo_traffic_unique_clones_avg{owner="jkroepke",repository="jkroepke/helm-secrets"} 27127
# HELP github_repo_traffic_popular_content_views Total views from top 10 content for given repository
# TYPE github_repo_traffic_popular_content_views gauge
github_repo_traffic_popular_content_views{owner="jkroepke",repository="jkroepke/2Moons",path="/jkroepke/2Moons"} 116
# HELP github_repo_traffic_popular_content_unique_vistors Total unique views from top 10 content for given repository
# TYPE github_repo_traffic_popular_content_unique_vistors gauge
github_repo_traffic_popular_content_unique_vistors{owner="jkroepke",repository="jkroepke/2Moons",path="/jkroepke/2Moons"} 78
# HELP github_repo_traffic_referring_sites_views Total views from top 10 referrer for given repository
# TYPE github_repo_traffic_referring_sites_views gauge
github_repo_traffic_referring_sites_views{owner="jkroepke",repository="jkroepke/helm-secrets",referrer="Google"} 2129
# HELP github_repo_traffic_referring_sites_unique_vistors Total unique visitors from top 10 referrers for given repository
# TYPE github_repo_traffic_referring_sites_unique_vistors gauge
github_repo_traffic_referring_sites_unique_vistors{owner="jkroepke",repository="jkroepke/helm-secrets",referrer="Google"} 798
# HELP github_repo_traffic_views Total views from top 10 content for given repository
# TYPE github_repo_traffic_views gauge
github_repo_traffic_views{owner="jkroepke",repository="jkroepke/github_exporter",week="latest"} 41
# HELP github_repo_traffic_unique_vistors Total unique views from top 10 content for given repository
# TYPE github_repo_traffic_unique_vistors gauge
github_repo_traffic_unique_vistors{owner="jkroepke",repository="jkroepke/github_exporter",week="latest"} 5
# HELP github_repo_traffic_views_avg Avenge views from top 10 content for given repository
# TYPE github_repo_traffic_views_avg gauge
github_repo_traffic_views_avg{owner="jkroepke",repository="jkroepke/github_exporter"} 76
# HELP github_repo_traffic_unique_vistors_avg Avenge unique views from top 10 content for given repository
# TYPE github_repo_traffic_unique_vistors_avg gauge
github_repo_traffic_unique_vistors_avg{owner="jkroepke",repository="jkroepke/github_exporter"} 10

Contributing

See CONTRIBUTING.md

Authors

• Jan-Otto Kröpke

Contributors ✨

Thanks goes to these wonderful people (emoji key):

This project follows the all-contributors specification. Contributions of any kind welcome!

License

MIT Licence