aurora-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Erb, Stephan" <Stephan....@blue-yonder.com>
Subject Re: Using Prometheus with Aurora
Date Wed, 01 Jul 2015 17:06:07 GMT
Hi Brian,

that will come in handy. Thanks for that!

I have thought about using prometheus together with the Mesos and Aurora exporters. Your feature
would even allow us to start both exports on Aurora and let Prometheus discover them automatically.
However, it might be a bit risky to use Aurora/Mesos to host its own monitoring solution...

How have you organized the deployment of your Prometheus components? 

Regards,
Stephan
________________________________________
From: Brian Brazil <brian.brazil@boxever.com>
Sent: Wednesday, July 1, 2015 4:54 PM
To: dev@aurora.apache.org; prometheus-developers
Subject: Using Prometheus with Aurora

Hi,

I've added support to Prometheus to use Twitter serversets for service
discovery, and we're now got this deployed in production. A key point is
that you can configure it to look at a tree of ZNodes in Zookeeper, so that
it can automatically pick up any new jobs announced by the Aurora executor
with no additional configuration required.


The scrape config we have in ansible for this is:

  - job_name: "aurora_job"
    serverset_sd_configs:      - servers:{% for host in
groups["zookeeper"] %}          - "{{ host }}:2181"{% endfor %}
paths:          - "/aurora/serversets"    relabel_configs:      -
source_labels: [ __meta_serverset_path ]        regex:
'^/aurora/serversets/[^/]+/[^/]+/([^/]+)/.*'        target_label: job
      replacement: "${1}"      - source_labels: [ __address__ ]
regex: '(ip-\d+-\d+-\d+-\d+)\..*:(\d+)'        target_label:
__address__        replacement: "${1}:${2}"



The relabel of the addresses is as our mesos slaves have the externally
visible host names so they work nicely in the browser, but which don't
actually exist in DNS so need some munging.

Note that you'll need the Prometheus at head to use this, and that this
should also work for non-Aurora created serversets such as if you're using
Finagle.

Relevant docs:
http://aurora.apache.org/documentation/latest/configuration-reference/#announcer-objects

http://prometheus.io/docs/operating/configuration/#zookeeper-serverset-sd-configurations-serverset_sd_config

Brian

Mime
View raw message