flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Simeon Zaharici <simeon_zahar...@yahoo.com>
Subject Re: Json over netcat source
Date Thu, 10 Apr 2014 01:17:59 GMT
Hello

This is to be able to easily integrate log centralization clients such as nxlog with flume 

The problem is not related to the splitting, or the size, even a message like { "blah": "blah"
} will result in the exception "org.elasticsearch.common.xcontent.XContentBuilder"

Simeon
On Wednesday, April 9, 2014 9:09:18 PM, Hari Shreedharan <hshreedharan@cloudera.com>
wrote:
 
Why exactly do you want to send JSON over netcat? Why don't you simply use the Avro Source
with Flume SDK? I am not entirely sure about the Elastic search sink. The reason your JSON
is not working is likely because netcat is splitting the events per newline or when it reaches
a fixed size. 


Hari



On Wed, Apr 9, 2014 at 5:58 PM, Simeon Zaharici <simeon_zaharici@yahoo.com> wrote:

Hello
>
>
>No, I did not find any solution to this
>
>
>Simeon
>On Monday, April 7, 2014 6:09:21 AM, Deepak Subhramanian <deepak.subhramanian@gmail.com>
wrote:
> 
>Hi Simeon,
>I am facing the same problem. Did you find any solution ?
>
>
>Thanks, Deepak 
>
>
>
>On Wed, Mar 5, 2014 at 5:52 PM, Simeon Zaharici <simeon_zaharici@yahoo.com> wrote:
>
>Hi guys
>>
>>
>>I would like to send json data to a flume netcat source and then send it to an elasticsearch
sink that uses the 
>>org.apache.flume.sink.elasticsearch.ElasticSearchLogStashEventSerializer.
>>
>>It works to send plain text on the netcat source, however when I send json the event
ends up in elasticsearch as 'org.elasticsearch.common.xcontent.XContentBuilder'.
>>
>>Which would be the easiest way to go about this ? I saw from reading docs that creating
a morphline interceptor could be the way to go but I did not fully understand how that
 works.
>>
>>Thanks a lot
>>
>
>
>
>-- 
>Deepak Subhramanian 
>
>
Mime
View raw message