flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Julian Henry Alcala <zenl...@gmail.com>
Subject Re: Flume (NG?) with S3 Sink?
Date Tue, 28 Aug 2012 21:29:45 GMT
I tried flume-OG with s3 and it was not reliable.  I am interested in what
people are doing with flume-ng and s3 as well.

On Tue, Aug 28, 2012 at 1:47 PM, igor Finkelshteyn <iefinkel@gmail.com>wrote:

> Has no one needed to send logs through Flume NG to s3? This JIRA<https://issues.apache.org/jira/browse/FLUME-1015>
> it sound like it's something that's already been implemented with
> documentation, but there is 0 documentation anywhere (definitely not in the
> user guide), and the JIRA was resolved months ago.
> On Aug 27, 2012, at 3:39 PM, igor Finkelshteyn wrote:
> Hi Folks,
> I'm completely new to flume, but was hoping to use it as a way to collect
> logs on a couple web servers and funnel them to an s3 sink in Avro. I've
> found a tutorial for doing this on Flume OG, but am thinking I should
> probably just work with Flume NG for a new project if that's possible. Is
> there a prebuilt s3 sink for Flume NG that I can use? Can I just use the
> HDFS sink and provide some extra parameters to get it to work with s3? Or
> should I just use Flume OG?I'd also be open to using something else
> entirely that has better s3 support, like FluentD, if that makes more
> sense. Any ideas?
> Thanks!
> Igor

View raw message