hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Marco A. S. Netto" <marco.ne...@gmail.com>
Subject Ganglia + Number of Running Mappers + Reducers + Yarn
Date Wed, 11 Jun 2014 18:58:52 GMT
Dear all,

I've recently installed Hadoop 2.2.0 with Ganglia on a small cluster.

I would like to get information about the progress of the application, such
as running mappers/reducers/tasks, pending mappers/reducers/tasks.

Checking the following file, I've noticed that there are some interesting
variables that could help me:
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/ClusterMetrics.java

However, variables such as runningMaps  and runningReduces don't seem to be
available from Ganglia.

Is it possible to get these variables from Hadoop 2.2.0? Or only 1.x?  Any
help is very welcome.

Thanks in advance,
Marco

------
Here is my hadoop-metrics2.properties  which I believe may not be 100%
correct

---

*.sink.ganglia.class=org.apache.hadoop.metrics2.sink.ganglia.GangliaSink31
*.sink.ganglia.period=10
*.sink.ganglia.supportsparse=true

tasktracker.sink.ganglia.class=org.apache.hadoop.metrics2.sink.ganglia.GangliaSink31
tasktracker.sink.ganglia.period=10
tasktracker.sink.ganglia.servers=IP_GANGLIA_GMETAD:8649

jobtracker.sink.ganglia.class=org.apache.hadoop.metrics2.sink.ganglia.GangliaSink31
jobtracker.sink.ganglia.period=10
jobtracker.sink.ganglia.servers=IP_GANGLIA_GMETAD:8649

maptask.sink.ganglia.class=org.apache.hadoop.metrics2.sink.ganglia.GangliaSink31
maptask.sink.ganglia.period=10
maptask.sink.ganglia.servers=IP_GANGLIA_GMETAD:8649


mapred.sink.ganglia.class=org.apache.hadoop.metrics2.sink.ganglia.GangliaSink31
mapred.sink.ganglia.period=10
mapred.sink.ganglia.servers=IP_GANGLIA_GMETAD:8649


namenode.sink.ganglia.class=org.apache.hadoop.metrics2.sink.ganglia.GangliaSink31
namenode.sink.ganglia.period=10
namenode.sink.ganglia.servers=IP_GANGLIA_GMETAD:8649

datanode.sink.ganglia.class=org.apache.hadoop.metrics2.sink.ganglia.GangliaSink31
datanode.sink.ganglia.period=10
datanode.sink.ganglia.servers=IP_GANGLIA_GMETAD:8649

resourcemanager.sink.ganglia.class=org.apache.hadoop.metrics2.sink.ganglia.GangliaSink31
resourcemanager.sink.ganglia.period=10
resourcemanager.sink.ganglia.servers=IP_GANGLIA_GMETAD:8649
----

Mime
View raw message