hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jagat Singh <jagatsi...@gmail.com>
Subject Re: How to Archive the Task Logs (Stdout, Stderr, Syslogs)
Date Tue, 19 Mar 2013 10:05:58 GMT
Hello,

You should be looking at Flume.

Its made for this

http://flume.apache.org/

Thanks,

Jagat Singh

On Tue, Mar 19, 2013 at 9:03 PM, Christian Schneider <
cschneiderpublic@gmail.com> wrote:

> Hi,
> I found out that these logs are stored directly at the TaskNodes.
>
> We need to have them stored over a long time (some months or better a
> year). What is a good way of doing that?
>
> With my current knowledge I would write a cron job that picks up all the
> files every few minutes.
> But I guess thats not the best approach...
>
> Best Regards,
> Christian.
>

Mime
View raw message