flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arif,Mubaraka" <arif.mubar...@heb.com>
Subject Flume - elasticsearch sink fails
Date Mon, 02 May 2016 19:44:06 GMT
Hello,

Trying to flume tweets from Tibco-JMS source into elastic search sink.
Running Elasticsearch on our edge nodes independent of Hadoop.
Running cloudera Hadoop 5.4.8

On running the error is  :

May 2, 2:15:14.047 PM

INFO

org.apache.flume.instrumentation.MonitoredCounterGroup

Component type: SINK, name: k1 started

May 2, 2:15:14.050 PM

WARN

org.apache.flume.sink.elasticsearch.client.ElasticSearchTransportClient

[arcapl0021707:9200]

May 2, 2:15:14.052 PM

ERROR

org.apache.flume.lifecycle.LifecycleSupervisor

Unable to start SinkRunner: { policy:org.apache.flume.sink.DefaultSinkProcessor@50482278 counterGroup:{
name:null counters:{} } } - Exception follows.
java.lang.NoSuchMethodError: org.elasticsearch.common.transport.InetSocketTransportAddress.<init>(Ljava/lang/String;I)V
        at org.apache.flume.sink.elasticsearch.client.ElasticSearchTransportClient.configureHostnames(ElasticSearchTransportClient.java:143)
        at org.apache.flume.sink.elasticsearch.client.ElasticSearchTransportClient.<init>(ElasticSearchTransportClient.java:77)
        at org.apache.flume.sink.elasticsearch.client.ElasticSearchClientFactory.getClient(ElasticSearchClientFactory.java:48)
        at org.apache.flume.sink.elasticsearch.ElasticSearchSink.start(ElasticSearchSink.java:357)
        at org.apache.flume.sink.DefaultSinkProcessor.start(DefaultSinkProcessor.java:46)
        at org.apache.flume.SinkRunner.start(SinkRunner.java:79)
        at org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:251)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)



Flume configuration for ElasticSearch is :

#confgiure elastic search sink
test.sinks.k1.type = elasticsearch
test.sinks.k1.hostNames = 172.17.43.207:9200
test.sinks.k1.indexName = TwitterComp_index
test.sinks.k1.indexType = TwitterComp_type
test.sinks.k1.clusterName = twittersearch
test.sinks.k1.batchSize = 500
test.sinks.k1.ttl = 5d
test.sinks.k1.serializer = org.apache.flume.sink.elasticsearch.ElasticSearchDynamicSerializer

Any help is greatly appreciated.

Thanks,
Muby

Mime
View raw message