flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Otis Gospodnetic <otis.gospodne...@gmail.com>
Subject Re: Flume Jambalaya - A Flume Plugin with Multiple Components
Date Sat, 03 May 2014 00:21:07 GMT
What Wolfgang said :)

I'd be interested in hearing how the File Source is different from or
better than Exec Source with tail -F or
https://issues.apache.org/jira/browse/FLUME-2344 - do you know?

Performance Monitoring * Log Analytics * Search Analytics
Solr & Elasticsearch Support * http://sematext.com/

On Fri, May 2, 2014 at 5:18 PM, Israel Ekpo <israel@aicer.org> wrote:

> Flume Community,
> I created a Flume Plugin with multiple components that complements the
> current version of Apache Flume.
> This was necessary as part of a personal project as I working on.
> It is code named - Flume Jambalaya
> Jambalaya is a standalone Apache Flume plugin that contains a variety of
> sources, interceptors, channels, sinks, serializers and other components
> designed to extend the Flume architecture. It has been released under the
> Apache License version 2.0
> https://github.com/aicer/flume-jambalaya
> It currently contains:
> (a) File Source - This source lets you ingest data by tailing files from a
> specific path
> (b) ElasticSearch HTTP Sink - This sink sends events to an ElasticSearch
> cluster via HTTP with no dependency on the ElasticSearch versions between
> Flume and the Server cluster.
> (c) DateInterceptor - The date interceptor is used for parsing dates from
> fields and using that date or timestamp as the timestamp for the Flume
> event.
> (d) Grok Interceptor - allows you to extract structured data from
> unstructured text and inject them as headers into the event
> Sample configuration files are available here
> https://github.com/aicer/flume-jambalaya/tree/master/sample-configuration-files
> I did not realize that the Flume trunk already has a HTTP Sink for
> ElasticSearch so you can decide whether or not to use the sink that comes
> with it
> I am still testing and integrating the various components.
> Please check it out when you get a chance and send me some feedback
> Thanks.

View raw message