hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vinod Kumar Vavilapalli <vino...@hortonworks.com>
Subject Re: YARN Logs
Date Wed, 16 Jul 2014 03:23:24 GMT
Adam is right.

"yarn logs" command only works when log-aggregation is enabled. It's not
easy but possible to make it work when aggregation is disabled.

+Vinod
Hortonworks Inc.
http://hortonworks.com/


On Tue, Jul 15, 2014 at 10:03 AM, Brian C. Huffman <
bhuffman@etinternational.com> wrote:

>  Looking at the docs, the yarn.nodemanager.remote-app-log-dir variable
> description says "Where to aggregate logs to."  But I don't want to use log
> aggregation.
>
> Does the 'yarn logs' command look anywhere else if
> yarn.log-aggregation-enable is set to false?
>
> -b
>
>
> On 07/15/2014 12:10 PM, Adam Kawa wrote:
>
> For example, the remote location is configured
> via yarn.nodemanager.remote-app-log-dir and defaults to "/tmp/logs".
> This is why, you see:
> *Logs not available at
> /tmp/logs/hadoop/logs/application_1405396841766_0003.*
>
>  PS.
> The full path is configured via $
> {yarn.nodemanager.remote-app-log-dir}/${username}/${yarn.nodemanager.remote-app-log-dir-suffix}/${application-id}
>
>
>
> 2014-07-15 18:08 GMT+02:00 Adam Kawa <kawa.adam@gmail.com>:
>
>>  IMHO,
>> $ yarn logs looks for aggregated logs at remote location.
>>
>>
>>
>> 2014-07-15 16:49 GMT+02:00 Brian C. Huffman <bhuffman@etinternational.com
>> >:
>>
>> All,
>>>
>>> I am running a small cluster with hadoop-2.2.0 installed on an NFS
>>> shared directory.  Since all nodes can access, I do not want to enable log
>>> aggregation.
>>>
>>> My understanding was that if aggregation wasn't enabled, the 'yarn logs'
>>> command would just look in the $HADOOP_HOME/logs/userlogs dir, but that
>>> isn't happening:
>>> [hadoop@host1 ~]$ yarn logs -applicationId
>>> application_1405396841766_0003
>>> 14/07/15 10:35:58 INFO client.RMProxy: Connecting to ResourceManager at
>>> host1/172.17.1.1:8032
>>> Logs not available at
>>> /tmp/logs/hadoop/logs/application_1405396841766_0003
>>> Log aggregation has not completed or is not enabled.
>>>
>>> Can anyone suggest what I might be doing wrong?
>>>
>>> Thanks,
>>> Brian
>>>
>>>
>>
>
>

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Mime
View raw message