hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Brian C. Huffman" <bhuff...@etinternational.com>
Subject Re: YARN Logs
Date Wed, 16 Jul 2014 13:26:00 GMT
Vinod,

When you say "it's not easy, but possible", what would have to be done 
to make it work?

Thanks,
Brian

On 07/15/2014 11:23 PM, Vinod Kumar Vavilapalli wrote:
> Adam is right.
>
> "yarn logs" command only works when log-aggregation is enabled. It's 
> not easy but possible to make it work when aggregation is disabled.
>
> +Vinod
> Hortonworks Inc.
> http://hortonworks.com/
>
>
> On Tue, Jul 15, 2014 at 10:03 AM, Brian C. Huffman 
> <bhuffman@etinternational.com <mailto:bhuffman@etinternational.com>> 
> wrote:
>
>     Looking at the docs, the yarn.nodemanager.remote-app-log-dir
>     variable description says "Where to aggregate logs to."  But I
>     don't want to use log aggregation.
>
>     Does the 'yarn logs' command look anywhere else if
>     yarn.log-aggregation-enable is set to false?
>
>     -b
>
>
>     On 07/15/2014 12:10 PM, Adam Kawa wrote:
>>     For example, the remote location is configured
>>     via yarn.nodemanager.remote-app-log-dir and defaults to "/tmp/logs".
>>     This is why, you see:
>>     /Logs not available at
>>     */tmp/logs*/hadoop/logs/application_1405396841766_0003./
>>
>>     PS.
>>     The full path is configured via
>>     ${yarn.nodemanager.remote-app-log-dir}/${username}/${yarn.nodemanager.remote-app-log-dir-suffix}/${application-id}
>>
>>
>>
>>     2014-07-15 18:08 GMT+02:00 Adam Kawa <kawa.adam@gmail.com
>>     <mailto:kawa.adam@gmail.com>>:
>>
>>         IMHO,
>>         $ yarn logs looks for aggregated logs at remote location.
>>
>>
>>
>>         2014-07-15 16:49 GMT+02:00 Brian C. Huffman
>>         <bhuffman@etinternational.com
>>         <mailto:bhuffman@etinternational.com>>:
>>
>>             All,
>>
>>             I am running a small cluster with hadoop-2.2.0 installed
>>             on an NFS shared directory.  Since all nodes can access,
>>             I do not want to enable log aggregation.
>>
>>             My understanding was that if aggregation wasn't enabled,
>>             the 'yarn logs' command would just look in the
>>             $HADOOP_HOME/logs/userlogs dir, but that isn't happening:
>>             [hadoop@host1 ~]$ yarn logs -applicationId
>>             application_1405396841766_0003
>>             14/07/15 10:35:58 INFO client.RMProxy: Connecting to
>>             ResourceManager at host1/172.17.1.1:8032
>>             <http://172.17.1.1:8032>
>>             Logs not available at
>>             /tmp/logs/hadoop/logs/application_1405396841766_0003
>>             Log aggregation has not completed or is not enabled.
>>
>>             Can anyone suggest what I might be doing wrong?
>>
>>             Thanks,
>>             Brian
>>
>>
>>
>
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or 
> entity to which it is addressed and may contain information that is 
> confidential, privileged and exempt from disclosure under applicable 
> law. If the reader of this message is not the intended recipient, you 
> are hereby notified that any printing, copying, dissemination, 
> distribution, disclosure or forwarding of this communication is 
> strictly prohibited. If you have received this communication in error, 
> please contact the sender immediately and delete it from your system. 
> Thank You. 


Mime
View raw message