flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From IT CTO <goi....@gmail.com>
Subject Newbe question about Flume-ng - ElasticSearchSink
Date Tue, 08 Sep 2015 15:41:20 GMT
Hi,
I just started learning flume and using it to send events to our HDFS
cluster and log them in elasticsearch.
I am using the latest build

Flume 1.6.0
Source code repository: https://git-wip-us.apache.org/repos/asf/flume.git
Revision: 2561a23240a71ba20bf288c7c2cda88f443c2080
Compiled by hshreedharan on Mon May 11 11:15:44 PDT 2015
>From source with checksum b29e416802ce9ece3269d34233baf43f

I found two issues which I think should be filed as bug:
1) When using
agent.sinks.elastic-sink.serializer =
org.apache.flume.sink.elasticsearch.ElasticSearchDynamicSerializer
the event timestamp is stored as string and not dateOptionalTime if the
agent.sinks.elastic-sink.serializer is not set the code works as expected.

2) Documentation BUG - when using elasticsearchsink a timestamp field need
to be created mainly if we want to use it with kibana (as the documentation
say) so in order for it to work we should use the timestamp interceptor. I
think the documentation should note it in the elasticearchsink section.

Thanks,
Eran


-- 
Eran | "You don't need eyes to see, you need vision" (Faithless)

Mime
View raw message