hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Marco A. S. Netto" <marco.ne...@gmail.com>
Subject monitor mapper running execution times with hadoop 2.2.0
Date Thu, 12 Jun 2014 22:49:06 GMT
Hi,

I would like to monitor the average execution time of mappers and reducers
or something better to check the hadoop throughput.

I configured the hadoop metrics2 as follows:
*.sink.ganglia.period=10
*.sink.ganglia.supportsparse=true
*.sink.ganglia.servers=GANGLIA_SERVER_IP:8649

mrappmaster.sink.ganglia.class=org.apache.hadoop.metrics2.sink.ganglia.GangliaSink31
resourcemanager.sink.ganglia.class=org.apache.hadoop.metrics2.sink.ganglia.GangliaSink31
mapred.sink.ganglia.class=org.apache.hadoop.metrics2.sink.ganglia.GangliaSink31
namenode.sink.ganglia.class=org.apache.hadoop.metrics2.sink.ganglia.GangliaSink31
datanode.sink.ganglia.class=org.apache.hadoop.metrics2.sink.ganglia.GangliaSink31
nodemanager.sink.ganglia.class=org.apache.hadoop.metrics2.sink.ganglia.GangliaSink31

These two lines seem to be ignored:
maptask.sink.ganglia.class=org.apache.hadoop.metrics2.sink.ganglia.GangliaSink31
reducetask.sink.ganglia.class=org.apache.hadoop.metrics2.sink.ganglia.GangliaSink31


Is there any way to monitor the progress of a hadoop app? Even without
Ganglia?

Kind regards,
Marco

Mime
View raw message