hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Saeed Adel Mehraban <s.ade...@gmail.com>
Subject Re: What makes a map to fail?
Date Tue, 07 Jan 2014 09:28:32 GMT
Before I get your message, I already have set yarn.log-aggregation-
enable in yarn-site.xml for every node.
But when I click on trackin url of failed job this time, I get "Not found:
job_...."
In slave nodes job history, I can seed previous jobs but not last job!
Is it related to log aggregation?


On Tue, Jan 7, 2014 at 12:35 AM, Bing Jiang <jiangbinglover@gmail.com>wrote:

> <!--yarn-site.xml-->
>  <property>
>       <description>Whether to enable log aggregation</description>
>           <name>yarn.log-aggregation-enable</name>
>               <value>true</value>
>  </property>
>
> yarn.nodemanager.remote-app-log-dir
> Specifies the directory where logs are aggregated.
>
> Create the /var/log/hadoop-yarn directory and set ownership:
>
>
> sudo -u hdfs hadoop fs -mkdir /var/log/hadoop-yarn
> sudo -u hdfs hadoop fs -chown yarn:mapred /var/log/hadoop-yarn
>
>
>
> You need to create this directory because it is the parent of
> /var/log/hadoop-yarn/apps which is explicitly configured in the
> yarn-site.xml.
>
>
> FYI:
> http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/latest/CDH4-Installation-Guide/cdh4ig_topic_11_4.html
>
> 2014/1/7 Saeed Adel Mehraban <s.adel07@gmail.com>
>
>> When I click on individual maps logs, it says "Aggregation is not
>> enabled. Try the nodemanager at slave1-machine:60933"
>> How could I enable aggregation?
>>
>>
>> On Sun, Jan 5, 2014 at 1:21 PM, Harsh J <harsh@cloudera.com> wrote:
>>
>>> Every failed task typically carries a diagnostic message and a set of
>>> logs for you to investigate what caused it to fail. Try visiting the
>>> task's logs on the JT UI by clicking through individual failed
>>> attempts to find the reason of its failure.
>>>
>>> On Sun, Jan 5, 2014 at 11:03 PM, Saeed Adel Mehraban <s.adel07@gmail.com>
>>> wrote:
>>> > Hi all,
>>> > My task jobs are failing due to many failed maps. I want to know what
>>> makes
>>> > a map to fail? Is it something like exceptions or what?
>>>
>>>
>>>
>>> --
>>> Harsh J
>>>
>>
>>
>
>
> --
> Bing Jiang
> weibo: http://weibo.com/jiangbinglover
> BLOG: www.binospace.com
> BLOG: http://blog.sina.com.cn/jiangbinglover
> Focus on distributed computing, HDFS/HBase
>

Mime
View raw message