hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sergey Zhemzhitsky <szh.s...@gmail.com>
Subject Gathering counters and metrics of individual hadoop jobs
Date Tue, 13 Sep 2016 12:32:41 GMT
Hello hadoop experts!

I'm looking for a way to gather all the metrics and counters of individual
jobs as well as the whole cluster in the event-driven way to store all this
data within elasticsearch for later troubleshooting and analysis.

Using metric exporters seems to be the right way to follow, but metrics
does not allow to gather individual job counters. Although there are
container
(org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerMetrics)
and mr app metric sources
(org.apache.hadoop.mapreduce.v2.app.metrics.MRAppMetrics) these ones don't
allow to obtain individual job counters.

I also looked at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster#dispatcher's handlers to
register custom org.apache.hadoop.yarn.event.EventHandler for all the
events of interest, but it seems to be hardly possible and not pluggable
right now.

The pretty simple solution I've found is implementing an application that
will poll job history server using its rest api. This way is pretty
straightforward, but it does not allow to gather  changes in an
event-driven way.

So, I'd like to ask whether there is any way to collect metrics and
counters of individual hadoop jobs in an event-driven fashion without
involving black magic of javaagents (java instrumentation api), bytecode
modifications and aop-like functionality to intercept all the executions of
org.apache.hadoop.yarn.event.EventHandler#handle(Event).

Mime
View raw message