incubator-flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Hsieh <...@cloudera.com>
Subject Re: Getting Data from S3
Date Fri, 19 Aug 2011 07:48:39 GMT
Jonathan,

This isn't available out-of-the-box currently but one could create a custom
source that can grab data from s3.

This would be an interesting source plugin. (I've thought about building
something similar that uses an hdfs dir as a source).

Jon.

On Thu, Aug 18, 2011 at 11:24 AM, Jonathan <jonny2112@gmail.com> wrote:

> Hi,
>
> I was wondering if Flume supports retrieving data from an S3 bucket and
> sinking into HDFS. The s3 bucket is getting updated on a regular basis so I
> would like for flume to catch any changes on s3 and move them accordingly.
>  I have seen a lot of documentation on using S3 as a sink but I haven't been
> able to find anything going the other way. Is this even possible?
>
>
> Thanks
> Jonathan
>
>


-- 
// Jonathan Hsieh (shay)
// Software Engineer, Cloudera
// jon@cloudera.com

Mime
View raw message