flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ed Judge <ejud...@gmail.com>
Subject Re: Flume/Avro schema manipulation
Date Fri, 29 Aug 2014 15:18:48 GMT
From initial reading of what it does, it look promising.  So, I could ultimately leverage something
like the readLine and toAvro commands 

Thanks,
Ed


On Aug 29, 2014, at 8:25 AM, Gaspar Muñoz <gmunoz@stratio.com> wrote:

> Hi Ed, 
> 
> maybe after read logs from your file, you can apply org.apache.flume.sink.solr.morphline.MorphlineInterceptor$Builder
interceptor and then apply to Avro morphline. 
> 
> Is that what you are looking for?
> 
> Regards.
> 
> 
> 2014-08-29 13:43 GMT+02:00 Ed Judge <ejudgie@gmail.com>:
> I am looking for some good documentation that would explain how I can pull logs from
a file and apply a certain schema to that log file format.  I have gone threw the exercise
of sending a file using a Avro source/sink with the flume_ng avro_client to a flume_ng agent
but my understanding is that this is just using a "default Avro schema”.  I want to be able
build the schema a certain way.  Assuming it is doable, is there anything out that documents
this process?
> 
> Thanks,
> Ed
> 


Mime
View raw message