accumulo-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From mwa...@apache.org
Subject [accumulo-website] branch master updated: Created blog post on metrics setup for Grafana (#62)
Date Sat, 24 Mar 2018 16:47:58 GMT
This is an automated email from the ASF dual-hosted git repository.

mwalch pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/accumulo-website.git


The following commit(s) were added to refs/heads/master by this push:
     new e6f707b  Created blog post on metrics setup for Grafana (#62)
e6f707b is described below

commit e6f707b7c38404b31b0ea8da5befabe01437d9e6
Author: Mike Walch <mwalch@apache.org>
AuthorDate: Sat Mar 24 12:47:56 2018 -0400

    Created blog post on metrics setup for Grafana (#62)
    
    * Monitor & metrics docs links to blog post
---
 _docs-2-0/administration/monitoring-metrics.md    |   7 +-
 _posts/blog/2018-03-22-view-metrics-in-grafana.md |  89 ++++++++++++++++++++++
 images/blog/201803_metrics/accumulo-metrics1.png  | Bin 0 -> 151372 bytes
 images/blog/201803_metrics/accumulo-metrics2.png  | Bin 0 -> 30711 bytes
 4 files changed, 95 insertions(+), 1 deletion(-)

diff --git a/_docs-2-0/administration/monitoring-metrics.md b/_docs-2-0/administration/monitoring-metrics.md
index 48e3a9d..d97ec0e 100644
--- a/_docs-2-0/administration/monitoring-metrics.md
+++ b/_docs-2-0/administration/monitoring-metrics.md
@@ -95,8 +95,13 @@ from growing without bound.
 Each server process should have log messages from the Metrics2 library about the sinks that
were created. Be sure to check
 the Accumulo processes log files when debugging missing metrics output.
 
-For additional information on configuring Metrics2, visit the [Javadoc page for Metrics2](https://hadoop.apache.org/docs/current/api/org/apache/hadoop/metrics2/package-summary.html).
+Below are additional resources for configuring Metrics2: 
 
+* [Javadoc page for Metrics2][metrics2-javadoc]
+* [Blog post][grafana-post] about configuring Grafana/InfluxDB to view Accumulo metrics2
+
+[grafana-post]: {{ site.baseurl }}/blog/2018/03/22/view-metrics-in-grafana/
+[metrics2-javadoc]: https://hadoop.apache.org/docs/current/api/org/apache/hadoop/metrics2/package-summary.html
 [tracing]: {{page.docs_baseurl}}/administration/tracing
 [monitor.ssl.keyStore]: {{ page.docs_baseurl }}/administration/properties#monitor_ssl_keyStore
 [monitor.ssl.keyStorePassword]: {{ page.docs_baseurl }}/administration/properties#monitor_ssl_keystorePassword
diff --git a/_posts/blog/2018-03-22-view-metrics-in-grafana.md b/_posts/blog/2018-03-22-view-metrics-in-grafana.md
new file mode 100644
index 0000000..3165709
--- /dev/null
+++ b/_posts/blog/2018-03-22-view-metrics-in-grafana.md
@@ -0,0 +1,89 @@
+---
+title: "View Accumulo metrics in Grafana"
+---
+
+The [Accumulo monitor][monitor] provides the easiest way to view Accumulo metrics but it
has two limitations:
+
+1. The overview page is limited to 10 graphs
+1. These graphs only show metrics for the past two hours
+
+The Accumulo monitor could be improved to be better a visualizaiton tool for metrics but
this doesn't make
+sense when tools like [Grafana], [Graphite], and [Ganglia] exist and Accumulo can report
metrics to these tools
+using [Hadoop Metrics2][metrics2]. While it's easy to configure Accumulo to send metrics,
the hard part is
+setting up and configuring a metrics visualization tool (i.e Graphite, Ganglia, Grafana/InfluxDB)
to collect
+and view these metrics.
+
+To ease this burden, this post describes how to send Accumulo metrics to [InfluxDB], a time
series database, 
+and make them viewable in [Grafana], a visualization tool.
+
+Below is a screenshot of Accumulo metrics in Grafana:
+
+<img src='/images/blog/201803_metrics/accumulo-metrics1.png' width="100%">
+<img src='/images/blog/201803_metrics/accumulo-metrics2.png' width="100%">
+
+## Set up metrics using Uno
+
+[Uno] can setup and configure InfluxDB/Grafana when it sets up Accumulo using the following
command:
+
+```
+uno setup accumulo --with-metrics
+``` 
+
+Metrics from Accumulo will be immediately viewable in Grafana at [http://localhost:3000/](http://localhost:3000/).
+
+## Set up metrics manually
+
+1.  Follow the standard installation instructions for [InfluxDB] and [Grafana]. As for versions,
+    the instructions below were written using InfluxDB v0.9.4.2 and Grafana v2.5.0.
+1.  Add the following to your InfluxDB configuration to configure it accept metrics in Graphite
+    format from Accumulo. The configuration below contains templates that transform the Graphite
+    metrics into a format that is usable in InfluxDB.
+    ```
+    [[graphite]]
+      bind-address = ":2003"
+      enabled = true
+      database = "accumulo_metrics"
+      protocol = "tcp"
+      consistency-level = "one"
+      separator = "_"
+      batch-size = 1000
+      batch-pending = 5
+      batch-timeout = "1s"
+      templates = [
+        "accumulo.*.*.*.*.*.*.* measurement.measurement.measurement.d.e.f.host.measurement",
+        "accumulo.*.*.*.*.*.* measurement.measurement.measurement.d.e.host.measurement",
+        "accumulo.*.*.*.*.* measurement.measurement.measurement.d.host.measurement",
+      ]
+    ```
+1. Configure the Accumulo configuration file `hadoop-metrics2-accumulo.properties` to send
Graphite
+   metrics to InfluxDB. Below is example configuration. Remember to replace `<INFLUXDB_HOST>`
with
+   the actual host.
+    ```
+    *.period=30
+    accumulo.sink.graphite.class=org.apache.hadoop.metrics2.sink.GraphiteSink
+    accumulo.sink.graphite.server_host=<INFLUXDB_HOST>
+    accumulo.sink.graphite.server_port=2003
+    accumulo.sink.graphite.metrics_prefix=accumulo
+    ```
+   Make sure the reporting frequency is set to 30 sec (i.e `*.period=30`). This is required
if you are
+   using the provided Grafana dashboards that is configured in the next step.
+1. Grafana needs to be configured to load dashboard JSON templates from a directory. Before
restarting
+   Grafana, you should copy [this Accumulo dashboard template][dashboard] to the `dashboards/`
directory
+   configured below.
+    ```
+    [dashboards.json]
+    enabled = true
+    path = <GRAFANA_HOME>/dashboards
+    ```
+1.  If you restart Grafana, you will see the Accumulo dashboard configured but all of their
charts will
+    be empty unless you have Accumulo running and configured to send data to InfluxDB. When
you start
+    sending data, you may need to refresh the dashboard page in the browser to start viewing
metrics.
+
+[Uno]: https://github.com/astralway/uno
+[metrics2]: /docs/2.0/administration/monitoring-metrics#hadoop-metrics2
+[Graphite]: https://graphiteapp.org/
+[Ganglia]: http://ganglia.sourceforge.net/
+[monitor]: http://0.0.0.0:4000/features/#screenshots
+[Grafana]: http://grafana.org/
+[InfluxDB]: https://influxdb.com/
+[dashboard]: https://raw.githubusercontent.com/astralway/uno/master/conf/grafana/accumulo-dashboard.json
diff --git a/images/blog/201803_metrics/accumulo-metrics1.png b/images/blog/201803_metrics/accumulo-metrics1.png
new file mode 100644
index 0000000..a46cf86
Binary files /dev/null and b/images/blog/201803_metrics/accumulo-metrics1.png differ
diff --git a/images/blog/201803_metrics/accumulo-metrics2.png b/images/blog/201803_metrics/accumulo-metrics2.png
new file mode 100644
index 0000000..b192909
Binary files /dev/null and b/images/blog/201803_metrics/accumulo-metrics2.png differ

-- 
To stop receiving notification emails like this one, please contact
mwalch@apache.org.

Mime
View raw message