Configuring buildbarn RBE to report prometheus metrics to Pushgateway

Buildbarn exposes some prometheus metrics, but not as http endpoints that a prometheus service can scrape.

Configuring buildbarn RBE to report prometheus metrics to Pushgateway

The open source buildbarn RBE provides distributed caching and  execution for Bazel.  As more and  more software development teams adopt bazel, solutions like this  are imperative.

Buildbarn exposes some prometheus metrics, but not as http endpoints that a prometheus service can scrape. Instead it allow for these metrics to be pushed to a prometheus pushgateway. The  pushgateway doesn't turn prometheus into a push-based monitoring system.

It was recently pointed out that my earlier assumptions that Buildbarn doesn't expose prometheus metrics as http endpoints is wrong, as you can see in the following tweet

To configure the buildbarn storage service to send these metrics to pushgateway, please  update the service's configuration file (config/storage.jsonnet) with the following:

...
...
...
global: {
    prometheusPushgateway: {url: '<prometheus-pushgateway-url>',
                            job: '<job-name>',
                            push_interval: '5s' # or whatever
                            },
  },
 ...
 ...

Verify that the data is available at: <prometheus-pushgateway-url>/metrics

For more information checkout the pushgateway protobuf defined in buildbarn storage.