flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Percy <mpe...@apache.org>
Subject Re: Json over netcat source
Date Thu, 10 Apr 2014 06:48:42 GMT
Not sure either but make sure you're using a compatible version of ElasticSearch.

Sent from my iPhone

> On Apr 9, 2014, at 9:43 PM, Hari Shreedharan <hshreedharan@cloudera.com> wrote:
> 
> Then I really don't know what the issue is. Someone more familiar with elastic search
sink will need to look at it.
> 
> Hari
> 
>> On Wednesday, April 9, 2014, Simeon Zaharici <simeon_zaharici@yahoo.com> wrote:
>> Hello
>> 
>> This is to be able to easily integrate log centralization clients such as nxlog with
flume 
>> 
>> The problem is not related to the splitting, or the size, even a message like { "blah":
"blah" } will result in the exception "org.elasticsearch.common.xcontent.XContentBuilder"
>> 
>> 
>> Simeon
>> On Wednesday, April 9, 2014 9:09:18 PM, Hari Shreedharan <hshreedharan@cloudera.com>
wrote:
>> Why exactly do you want to send JSON over netcat? Why don't you simply use the Avro
Source with Flume SDK? I am not entirely sure about the Elastic search sink. The reason your
JSON is not working is likely because netcat is splitting the events per newline or when it
reaches a fixed size. 
>> 
>> 
>> Hari
>> 
>> 
>> On Wed, Apr 9, 2014 at 5:58 PM, Simeon Zaharici <simeon_zaharici@yahoo.com>
wrote:
>> Hello
>> 
>> No, I did not find any solution to this
>> 
>> Simeon
>> On Monday, April 7, 2014 6:09:21 AM, Deepak Subhramanian <deepak.subhramanian@gmail.com>
wrote:
>> Hi Simeon,
>> I am facing the same problem. Did you find any solution ?
>> 
>> Thanks, Deepak 
>> 
>> 
>> On Wed, Mar 5, 2014 at 5:52 PM, Simeon Zaharici <simeon_zaharici@yahoo.com>
wrote:
>> Hi guys
>> 
>> I would like to send json data to a flume netcat source and then send it to an elasticsearch
sink that uses the
>> org.apache.flume.sink.elasticsearch.ElasticSearchLogStashEventSerializer.
>> 
>> 
>> 
>> 
>> It works to send plain text on the netcat source, however when I send json the event
ends up in elasticsearch as 'org.elasticsearch.common.xcontent.XContentBuilder'.
>> 
>> 
>> Which would be the easiest way to go about this ? I saw from reading docs that creating
a morphline interceptor could be the way to go but I did not fully understand how that
>>  works.
>> 
>> Thanks a lot
>> 
>> 
>> 
>> -- 
>> Deepak Subhramanian

Mime
View raw message