hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bing Jiang <jiangbinglo...@gmail.com>
Subject Re: What makes a map to fail?
Date Tue, 07 Jan 2014 08:35:46 GMT
<!--yarn-site.xml-->
 <property>
      <description>Whether to enable log aggregation</description>
          <name>yarn.log-aggregation-enable</name>
              <value>true</value>
 </property>

yarn.nodemanager.remote-app-log-dir
Specifies the directory where logs are aggregated.

Create the /var/log/hadoop-yarn directory and set ownership:


sudo -u hdfs hadoop fs -mkdir /var/log/hadoop-yarn
sudo -u hdfs hadoop fs -chown yarn:mapred /var/log/hadoop-yarn



You need to create this directory because it is the parent of
/var/log/hadoop-yarn/apps which is explicitly configured in the
yarn-site.xml.


FYI:
http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/latest/CDH4-Installation-Guide/cdh4ig_topic_11_4.html

2014/1/7 Saeed Adel Mehraban <s.adel07@gmail.com>

> When I click on individual maps logs, it says "Aggregation is not enabled.
> Try the nodemanager at slave1-machine:60933"
> How could I enable aggregation?
>
>
> On Sun, Jan 5, 2014 at 1:21 PM, Harsh J <harsh@cloudera.com> wrote:
>
>> Every failed task typically carries a diagnostic message and a set of
>> logs for you to investigate what caused it to fail. Try visiting the
>> task's logs on the JT UI by clicking through individual failed
>> attempts to find the reason of its failure.
>>
>> On Sun, Jan 5, 2014 at 11:03 PM, Saeed Adel Mehraban <s.adel07@gmail.com>
>> wrote:
>> > Hi all,
>> > My task jobs are failing due to many failed maps. I want to know what
>> makes
>> > a map to fail? Is it something like exceptions or what?
>>
>>
>>
>> --
>> Harsh J
>>
>
>


-- 
Bing Jiang
weibo: http://weibo.com/jiangbinglover
BLOG: www.binospace.com
BLOG: http://blog.sina.com.cn/jiangbinglover
Focus on distributed computing, HDFS/HBase

Mime
View raw message