flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Baris Akgun (Garanti Teknoloji)" <BarisA...@garanti.com.tr>
Subject Flume MorphlineSolrSink
Date Wed, 18 May 2016 07:43:01 GMT
Hi,

I am trying to make real time indexing with using flume 1.5.0 and morphline solr sink. I am
using packed base solr in my CDH.


tier1.sinks = solrSink
tier1.channels = channel1
#
tier1.channels.channel1.type=memory
tier1.channels.channel1.capacity=1000000
#
tier1.sinks.solrSink.type = org.apache.flume.sink.solr.morphline.MorphlineSolrSink
tier1.sinks.solrSink.batchSize = 100
tier1.sinks.solrSink.batchDurationMillis = 10000
tier1.sinks.solrSink.morphlineFile = morphlines.conf
tier1.sinks.solrSink.morphlineId = morphline1
#
tier1.sinks.solrSink.channel = channel1


The flume throw error in sink step.  The error is given below. What is your ideas to solve
the problem?

Thanks.


ERROR
Unable to start SinkRunner: { policy:org.apache.flume.sink.DefaultSinkProcessor@748016d1 counterGroup:{
name:null counters:{} } } - Exception follows.
java.lang.NoSuchMethodError: org.apache.lucene.util.fst.FST$BytesReader.skipBytes(J)V
         at org.apache.lucene.util.fst.FST.readNextRealArc(FST.java:1117)
         at org.apache.lucene.util.fst.FST.readFirstRealTargetArc(FST.java:1030)
         at org.apache.lucene.util.fst.FST.readRootArcs(FST.java:487)
         at org.apache.lucene.util.fst.FST.cacheRootArcs(FST.java:475)
         at org.apache.lucene.util.fst.FST.finish(FST.java:458)
         at org.apache.lucene.util.fst.Builder.finish(Builder.java:464)
         at org.apache.lucene.analysis.synonym.SynonymMap$Builder.build(SynonymMap.java:284)
         at org.apache.lucene.analysis.synonym.FSTSynonymFilterFactory.loadSynonyms(FSTSynonymFilterFactory.java:145)
         at org.apache.lucene.analysis.synonym.FSTSynonymFilterFactory.inform(FSTSynonymFilterFactory.java:112)
         at org.apache.lucene.analysis.synonym.SynonymFilterFactory.inform(SynonymFilterFactory.java:89)
         at org.apache.solr.core.SolrResourceLoader.inform(SolrResourceLoader.java:675)
         at org.apache.solr.schema.IndexSchema.<init>(IndexSchema.java:167)
         at org.apache.solr.schema.IndexSchemaFactory.create(IndexSchemaFactory.java:55)
         at org.apache.solr.schema.IndexSchemaFactory.buildIndexSchema(IndexSchemaFactory.java:69)
         at org.kitesdk.morphline.solr.SolrLocator.getIndexSchema(SolrLocator.java:181)
         at org.kitesdk.morphline.solr.SanitizeUnknownSolrFieldsBuilder$SanitizeUnknownSolrFields.<init>(SanitizeUnknownSolrFieldsBuilder.java:70)
         at org.kitesdk.morphline.solr.SanitizeUnknownSolrFieldsBuilder.build(SanitizeUnknownSolrFieldsBuilder.java:52)
         at org.kitesdk.morphline.base.AbstractCommand.buildCommand(AbstractCommand.java:302)
         at org.kitesdk.morphline.base.AbstractCommand.buildCommandChain(AbstractCommand.java:249)
         at org.kitesdk.morphline.stdlib.Pipe.<init>(Pipe.java:46)
         at org.kitesdk.morphline.stdlib.PipeBuilder.build(PipeBuilder.java:40)
         at org.kitesdk.morphline.base.Compiler.compile(Compiler.java:126)
         at org.kitesdk.morphline.base.Compiler.compile(Compiler.java:55)
         at org.apache.flume.sink.solr.morphline.MorphlineHandlerImpl.configure(MorphlineHandlerImpl.java:101)
         at org.apache.flume.sink.solr.morphline.MorphlineSink.start(MorphlineSink.java:97)
         at org.apache.flume.sink.DefaultSinkProcessor.start(DefaultSinkProcessor.java:46)
         at org.apache.flume.SinkRunner.start(SinkRunner.java:79)
         at org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:251)
         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
         at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
         at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
         at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
         at java.lang.Thread.run(Thread.java:745)





Barış Akgün
Analitik Veri Ambarı ve Büyük Veri Yönetimi
Uzman

Tel

:

Dahili

:

Faks

:



Bu mesaj ve ekleri, mesajda gonderildigi belirtilen kisi/kisilere ozeldir ve gizlidir. Bu
mesajin muhatabi olmamaniza ragmen tarafiniza ulasmis olmasi halinde mesaj iceriginin gizliligi
ve bu gizlilik yukumlulugune uyulmasi zorunlulugu tarafiniz icin de soz konusudur. Mesaj ve
eklerinde yer alan bilgilerin dogrulugu ve guncelligi konusunda gonderenin ya da sirketimizin
herhangi bir sorumlulugu bulunmamaktadir. Sirketimiz mesajin ve bilgilerinin size degisiklige
ugrayarak veya gec ulasmasindan, butunlugunun ve gizliliginin korunamamasindan, virus icermesinden
ve bilgisayar sisteminize verebilecegi herhangi bir zarardan sorumlu tutulamaz.

This message and attachments are confidential and intended solely for the individual(s) stated
in this message. If you received this message although you are not the addressee, you are
responsible to keep the message confidential. The sender has no responsibility for the accuracy
or correctness of the information in the message and its attachments. Our company shall have
no liability for any changes or late receiving, loss of integrity and confidentiality, viruses
and any damages caused in anyway to your computer system.

Mime
View raw message