hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Kerzner <mark.kerz...@shmsoft.com>
Subject Re: Best way to collect Hadoop logs across cluster
Date Fri, 26 Apr 2013 00:08:53 GMT
Thank you for all the advice, it was indeed very useful.

Mark


On Thu, Apr 18, 2013 at 11:44 PM, Roman Shaposhnik <rvs@apache.org> wrote:

> On Thu, Apr 18, 2013 at 9:23 PM, Mark Kerzner <mark.kerzner@shmsoft.com>
> wrote:
> > Hi,
> >
> > my clusters are on EC2, and they disappear after the cluster's instances
> are
> > destroyed. What is the best practice to collect the logs for later
> storage?
> >
> > EC2 does exactly that with their EMR, how do they do it?
>
> Apache Flume could be extremely useful for this purpose. You
> can even configure it to deposit log data in realtime into
> S3.
>
> Thanks,
> Roman.
>

Mime
View raw message