hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anfernee Xu <anfernee...@gmail.com>
Subject history server for 2 clusters
Date Fri, 21 Feb 2014 02:08:05 GMT
Hi,

I'm at 2.2.0 release and I have a HDFS cluster which is shared by 2
YARN(MR) cluster, also I have a single shared history server, what I'm
seeing is I can see all job summary for all jobs from history server UI, I
also can see task log for jobs running in one cluster, but if I want to see
log for jobs running in another cluster, it showed me below error

Logs not available for attempt_1392933787561_0024_m_000000_0. Aggregation
may not be complete, Check back later or try the nodemanager at
slc03jvt.mydomain.com:31303

Here's my configuration:

Note: my history server is running on RM node of the MR cluster where I can
see the log.


--------mapred-site.xml
<property>
  <name>mapreduce.jobhistory.address</name>
  <value>slc00dgd:10020</value>
  <description>MapReduce JobHistory Server IPC host:port</description>
</property>

<property>
  <name>mapreduce.jobhistory.webapp.address</name>
  <value>slc00dgd:19888</value>
  <description>MapReduce JobHistory Server Web UI host:port</description>
</property>

----------yarn-site.xml
  <property>
     <name>yarn.log-aggregation-enable</name>
     <value>true</value>
   </property>

   <property>
     <name>yarn.nodemanager.remote-app-log-dir-suffix</name>
     <value>dc</value>
   </property>

Above configuration are almost same for both clusters, the only difference
is "yarn.nodemanager.remote-app-log-dir-suffix", they have different suffix.



-- 
--Anfernee

Mime
View raw message