hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From stevens35 <steven...@llnl.gov>
Subject Accessing Job information via the Web UI in Hadoop 2.0
Date Wed, 14 Mar 2012 16:19:16 GMT
HI All,

I'm a little baffled by the new web UI in Hadoop 2.0.  In particular, I 
don't see an obvious way of viewing the task specific counters or the 
task status's that my jobs are setting.  Previously, this was all 
presented cleanly in the TaskTracker UI but now the 
ApplicationManager/HistoryServer obscure this information.

I've found that 
http://host:8088/proxy/<app_id>/mapreduce/jobcounters/<job_id> presents 
the counters over *all* tasks in the job and similarly
http://host:8088/proxy/<app_id>/mapreduce/attempts/<job_id>/m/RUNNING 
presents the current state of the running tasks.  However neither of 
these link to task specific counters, and in the running task display, 
the "STATE" column seems completely unassociated with the task's status.

Is there an easy way to view all this information without writing my own 
webapp around the ApplicationManager/HistoryServer?

I'm also not sure how to view these two web ui's via alternate 
hostnames, as was done automatically in the older versions of Hadoop.  
In my hdfs and yarn configurations, i set the hostname as krackenmaster, 
which uses one network interface.  At the same time, i'd like to view 
these ui's using a second, and more public, hostname such as kracken.com 
that just points to the master node in my cluster.  Would setting up the 
Web Application Proxy (described in 
http://hadoop.apache.org/common/docs/r0.23.1/hadoop-yarn/hadoop-yarn-site/WebApplicationProxy.html)

accomplish this?  As a note, I can view the NameNode UI from kracken.com 
without any problem.

Thanks!
--Keith

Mime
View raw message