flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brock Noland <br...@cloudera.com>
Subject Re: Syslog-ng, Flume (old gen) and metadata
Date Tue, 30 Oct 2012 15:12:37 GMT

I don't actually know the answer to your question, hopefully someone
else will help with that, but I have some hopefully useful comments

On Mon, Oct 29, 2012 at 10:51 AM, Bertrand Dechoux <dechouxb@gmail.com> wrote:
> Hi,
> I would like to deepen my understanding of syslog-ng and flume integration.
> I had initial three questions :
> 1) Does flume timestamp is the same as the received syslog-ng event
> timestamp?
> 2) Are the syslog-ng metadata kept by flume as extra metadata (> syslog-ng
> OSE 3.0)?
> 3) Are the extra metadata (and potientially syslog-ng ones) usable with the
> collector sink template?
> From what I understand :
> 1) Yes, acoording the cookbook. "use the timestamp found in the original
> data,"
> http://archive.cloudera.com/cdh/3/flume/Cookbook/

That is Flume OG which is still available but as far as I am aware no
one is actively taking that forward. Flume NG is actively being worked
on.  The Cloudera link for NG is here:
http://archive.cloudera.com/cdh/3/flume-ng-1.2.0-cdh3u5/ and the
apache webpage for flume is all NG content as far as I am aware:


> 2) No but I haven't found a clear answer about it.
> 3) No but I haven't found a clear answer about it.
> Could someone confirm me that?
> I would believe 2) and 3) be generally asked features for those integrating
> existent syslog-ng system with Flume but maybe it isn't. Or maybe there are
> third party libraries for that? What is the status of Flume-ng with regards
> to the same questions?
> Regards
> Bertrand

Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/

View raw message