hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jian He <...@hortonworks.com>
Subject Re: Can Yarn AppMaster move Container logs to hdfs?
Date Wed, 12 Feb 2014 19:07:11 GMT
Hi Emmanuel, log aggregation now only aggregates finished apps logs onto
hdfs. there's no way as of now to support running apps. that'll be a to-do
feature in the feature.

Jian



On Mon, Feb 10, 2014 at 11:53 AM, Emmanuel Espina
<espinaemmanuel@gmail.com>wrote:

> Sorry when I said log running I meant LONG running, that is, a service,
> not a batch job
>
>
> 2014-02-10 16:49 GMT-03:00 Emmanuel Espina <espinaemmanuel@gmail.com>:
>
> I'm building a custom YARN app where the App Master is a log running
>> service that can start jobs in containers.
>>
>> The ideal situation for us would be to be able to move all the logs
>> produced by each container to hdfs. The aggregation options that yarn
>> provides does this but only after the app master finishes (and in our case
>> it never finishes).
>>
>> Is there any way of doing this?
>>
>> Thanks
>> Emmanuel
>>
>
>

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Mime
View raw message