hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Roman Shaposhnik <...@apache.org>
Subject Re: Best way to collect Hadoop logs across cluster
Date Fri, 19 Apr 2013 04:44:31 GMT
On Thu, Apr 18, 2013 at 9:23 PM, Mark Kerzner <mark.kerzner@shmsoft.com> wrote:
> Hi,
>
> my clusters are on EC2, and they disappear after the cluster's instances are
> destroyed. What is the best practice to collect the logs for later storage?
>
> EC2 does exactly that with their EMR, how do they do it?

Apache Flume could be extremely useful for this purpose. You
can even configure it to deposit log data in realtime into
S3.

Thanks,
Roman.

Mime
View raw message