flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gonzalo Herreros <gherre...@gmail.com>
Subject Re: Flume MorphlineSolrSink
Date Wed, 18 May 2016 07:48:26 GMT
Flume 1.5.0 is pretty old,
Why don't you use version 1.6.0 included in CDH, that will ensure the
library compatibility.

On 18 May 2016 at 08:43, Baris Akgun (Garanti Teknoloji) <
BarisAkgu@garanti.com.tr> wrote:

> Hi,
>
>
>
> I am trying to make real time indexing with using flume 1.5.0 and
> morphline solr sink. I am using packed base solr in my CDH.
>
>
>
>
>
> tier1.sinks = solrSink
>
> tier1.channels = channel1
>
> #
>
> tier1.channels.channel1.type=memory
>
> tier1.channels.channel1.capacity=1000000
>
> #
>
> tier1.sinks.solrSink.type =
> org.apache.flume.sink.solr.morphline.MorphlineSolrSink
>
> tier1.sinks.solrSink.batchSize = 100
>
> tier1.sinks.solrSink.batchDurationMillis = 10000
>
> tier1.sinks.solrSink.morphlineFile = morphlines.conf
>
> tier1.sinks.solrSink.morphlineId = morphline1
>
> #
>
> tier1.sinks.solrSink.channel = channel1
>
>
>
>
>
> The flume throw error in sink step.  The error is given below. What is
> your ideas to solve the problem?
>
>
>
> Thanks.
>
>
>
>
>
> ERROR
>
> Unable to start SinkRunner: {
> policy:org.apache.flume.sink.DefaultSinkProcessor@748016d1 counterGroup:{
> name:null counters:{} } } - Exception follows.
>
> java.lang.NoSuchMethodError:
> org.apache.lucene.util.fst.FST$BytesReader.skipBytes(J)V
>
>          at org.apache.lucene.util.fst.FST.readNextRealArc(FST.java:1117)
>
>          at
> org.apache.lucene.util.fst.FST.readFirstRealTargetArc(FST.java:1030)
>
>          at org.apache.lucene.util.fst.FST.readRootArcs(FST.java:487)
>
>          at org.apache.lucene.util.fst.FST.cacheRootArcs(FST.java:475)
>
>          at org.apache.lucene.util.fst.FST.finish(FST.java:458)
>
>          at org.apache.lucene.util.fst.Builder.finish(Builder.java:464)
>
>          at
> org.apache.lucene.analysis.synonym.SynonymMap$Builder.build(SynonymMap.java:284)
>
>          at
> org.apache.lucene.analysis.synonym.FSTSynonymFilterFactory.loadSynonyms(FSTSynonymFilterFactory.java:145)
>
>          at
> org.apache.lucene.analysis.synonym.FSTSynonymFilterFactory.inform(FSTSynonymFilterFactory.java:112)
>
>          at
> org.apache.lucene.analysis.synonym.SynonymFilterFactory.inform(SynonymFilterFactory.java:89)
>
>          at
> org.apache.solr.core.SolrResourceLoader.inform(SolrResourceLoader.java:675)
>
>          at org.apache.solr.schema.IndexSchema.<init>(IndexSchema.java:167)
>
>          at
> org.apache.solr.schema.IndexSchemaFactory.create(IndexSchemaFactory.java:55)
>
>          at
> org.apache.solr.schema.IndexSchemaFactory.buildIndexSchema(IndexSchemaFactory.java:69)
>
>          at
> org.kitesdk.morphline.solr.SolrLocator.getIndexSchema(SolrLocator.java:181)
>
>          at
> org.kitesdk.morphline.solr.SanitizeUnknownSolrFieldsBuilder$SanitizeUnknownSolrFields.<init>(SanitizeUnknownSolrFieldsBuilder.java:70)
>
>          at
> org.kitesdk.morphline.solr.SanitizeUnknownSolrFieldsBuilder.build(SanitizeUnknownSolrFieldsBuilder.java:52)
>
>          at
> org.kitesdk.morphline.base.AbstractCommand.buildCommand(AbstractCommand.java:302)
>
>          at
> org.kitesdk.morphline.base.AbstractCommand.buildCommandChain(AbstractCommand.java:249)
>
>          at org.kitesdk.morphline.stdlib.Pipe.<init>(Pipe.java:46)
>
>          at
> org.kitesdk.morphline.stdlib.PipeBuilder.build(PipeBuilder.java:40)
>
>          at org.kitesdk.morphline.base.Compiler.compile(Compiler.java:126)
>
>          at org.kitesdk.morphline.base.Compiler.compile(Compiler.java:55)
>
>          at
> org.apache.flume.sink.solr.morphline.MorphlineHandlerImpl.configure(MorphlineHandlerImpl.java:101)
>
>          at
> org.apache.flume.sink.solr.morphline.MorphlineSink.start(MorphlineSink.java:97)
>
>          at
> org.apache.flume.sink.DefaultSinkProcessor.start(DefaultSinkProcessor.java:46)
>
>          at org.apache.flume.SinkRunner.start(SinkRunner.java:79)
>
>          at
> org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:251)
>
>          at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>
>          at
> java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
>
>          at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
>
>          at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
>
>          at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>
>          at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>
>          at java.lang.Thread.run(Thread.java:745)
>
>
>
>
>
>
>
>
>
>
>
> *Barış Akgün*
> Analitik Veri Ambarı ve Büyük Veri Yönetimi
> Uzman
>
> Tel
>
> :
>
> Dahili
>
> :
>
> Faks
>
> :
>
>
> Bu mesaj ve ekleri, mesajda gonderildigi belirtilen kisi/kisilere ozeldir
> ve gizlidir. Bu mesajin muhatabi olmamaniza ragmen tarafiniza ulasmis
> olmasi halinde mesaj iceriginin gizliligi ve bu gizlilik yukumlulugune
> uyulmasi zorunlulugu tarafiniz icin de soz konusudur. Mesaj ve eklerinde
> yer alan bilgilerin dogrulugu ve guncelligi konusunda gonderenin ya da
> sirketimizin herhangi bir sorumlulugu bulunmamaktadir. Sirketimiz mesajin
> ve bilgilerinin size degisiklige ugrayarak veya gec ulasmasindan,
> butunlugunun ve gizliliginin korunamamasindan, virus icermesinden ve
> bilgisayar sisteminize verebilecegi herhangi bir zarardan sorumlu tutulamaz.
>
> This message and attachments are confidential and intended solely for the
> individual(s) stated in this message. If you received this message although
> you are not the addressee, you are responsible to keep the message
> confidential. The sender has no responsibility for the accuracy or
> correctness of the information in the message and its attachments. Our
> company shall have no liability for any changes or late receiving, loss of
> integrity and confidentiality, viruses and any damages caused in anyway to
> your computer system.
>

Mime
View raw message