flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From أنس الليثي <dev.fano...@gmail.com>
Subject Re: Flume MorphlineSolrSink
Date Wed, 18 May 2016 08:27:49 GMT
I am not sure the issue you shared is the same issue you are facing.

*NoClassDefFoundError* is totally different than *NoSuchMethodError*.

I think solving this problem without upgrading is easy.

Could you please tell what is the version of Solr you installed? and also
put a list of the files in the flume lib directory.

Best regards

On 18 May 2016 at 11:11, Baris Akgun (Garanti Teknoloji) <
BarisAkgu@garanti.com.tr> wrote:

> Hi
>
>
>
> You re right. I also find some jira for that problem. (
> https://issues.apache.org/jira/browse/FLUME-2392) The problem is solved
> in 1.6.0 version.
>
>
>
> Unfortunately, we are using oracle bda with cdh 5.4 ,therefore it is not
> easy to upgrade any service (flume, hive etc.) . We depended the oracle for
> upgrading.
>
>
>
> How can we solve that problem without upgrading the flume? It must be some
> other solutions.
>
>
>
> Thanks..
>
>
>
> *From:* Gonzalo Herreros [mailto:gherreros@gmail.com]
> *Sent:* Wednesday, May 18, 2016 10:48 AM
> *To:* user
> *Subject:* Re: Flume MorphlineSolrSink
>
>
>
> Flume 1.5.0 is pretty old,
>
> Why don't you use version 1.6.0 included in CDH, that will ensure the
> library compatibility.
>
>
>
> On 18 May 2016 at 08:43, Baris Akgun (Garanti Teknoloji) <
> BarisAkgu@garanti.com.tr> wrote:
>
> Hi,
>
>
>
> I am trying to make real time indexing with using flume 1.5.0 and
> morphline solr sink. I am using packed base solr in my CDH.
>
>
>
>
>
> tier1.sinks = solrSink
>
> tier1.channels = channel1
>
> #
>
> tier1.channels.channel1.type=memory
>
> tier1.channels.channel1.capacity=1000000
>
> #
>
> tier1.sinks.solrSink.type =
> org.apache.flume.sink.solr.morphline.MorphlineSolrSink
>
> tier1.sinks.solrSink.batchSize = 100
>
> tier1.sinks.solrSink.batchDurationMillis = 10000
>
> tier1.sinks.solrSink.morphlineFile = morphlines.conf
>
> tier1.sinks.solrSink.morphlineId = morphline1
>
> #
>
> tier1.sinks.solrSink.channel = channel1
>
>
>
>
>
> The flume throw error in sink step.  The error is given below. What is
> your ideas to solve the problem?
>
>
>
> Thanks.
>
>
>
>
>
> ERROR
>
> Unable to start SinkRunner: {
> policy:org.apache.flume.sink.DefaultSinkProcessor@748016d1 counterGroup:{
> name:null counters:{} } } - Exception follows.
>
> java.lang.NoSuchMethodError:
> org.apache.lucene.util.fst.FST$BytesReader.skipBytes(J)V
>
>          at org.apache.lucene.util.fst.FST.readNextRealArc(FST.java:1117)
>
>          at
> org.apache.lucene.util.fst.FST.readFirstRealTargetArc(FST.java:1030)
>
>          at org.apache.lucene.util.fst.FST.readRootArcs(FST.java:487)
>
>          at org.apache.lucene.util.fst.FST.cacheRootArcs(FST.java:475)
>
>          at org.apache.lucene.util.fst.FST.finish(FST.java:458)
>
>          at org.apache.lucene.util.fst.Builder.finish(Builder.java:464)
>
>          at
> org.apache.lucene.analysis.synonym.SynonymMap$Builder.build(SynonymMap.java:284)
>
>          at
> org.apache.lucene.analysis.synonym.FSTSynonymFilterFactory.loadSynonyms(FSTSynonymFilterFactory.java:145)
>
>          at
> org.apache.lucene.analysis.synonym.FSTSynonymFilterFactory.inform(FSTSynonymFilterFactory.java:112)
>
>          at
> org.apache.lucene.analysis.synonym.SynonymFilterFactory.inform(SynonymFilterFactory.java:89)
>
>          at
> org.apache.solr.core.SolrResourceLoader.inform(SolrResourceLoader.java:675)
>
>          at org.apache.solr.schema.IndexSchema.<init>(IndexSchema.java:167)
>
>          at
> org.apache.solr.schema.IndexSchemaFactory.create(IndexSchemaFactory.java:55)
>
>          at
> org.apache.solr.schema.IndexSchemaFactory.buildIndexSchema(IndexSchemaFactory.java:69)
>
>          at
> org.kitesdk.morphline.solr.SolrLocator.getIndexSchema(SolrLocator.java:181)
>
>          at
> org.kitesdk.morphline.solr.SanitizeUnknownSolrFieldsBuilder$SanitizeUnknownSolrFields.<init>(SanitizeUnknownSolrFieldsBuilder.java:70)
>
>          at
> org.kitesdk.morphline.solr.SanitizeUnknownSolrFieldsBuilder.build(SanitizeUnknownSolrFieldsBuilder.java:52)
>
>          at
> org.kitesdk.morphline.base.AbstractCommand.buildCommand(AbstractCommand.java:302)
>
>          at
> org.kitesdk.morphline.base.AbstractCommand.buildCommandChain(AbstractCommand.java:249)
>
>          at org.kitesdk.morphline.stdlib.Pipe.<init>(Pipe.java:46)
>
>          at
> org.kitesdk.morphline.stdlib.PipeBuilder.build(PipeBuilder.java:40)
>
>          at org.kitesdk.morphline.base.Compiler.compile(Compiler.java:126)
>
>          at org.kitesdk.morphline.base.Compiler.compile(Compiler.java:55)
>
>          at
> org.apache.flume.sink.solr.morphline.MorphlineHandlerImpl.configure(MorphlineHandlerImpl.java:101)
>
>          at
> org.apache.flume.sink.solr.morphline.MorphlineSink.start(MorphlineSink.java:97)
>
>          at
> org.apache.flume.sink.DefaultSinkProcessor.start(DefaultSinkProcessor.java:46)
>
>          at org.apache.flume.SinkRunner.start(SinkRunner.java:79)
>
>          at
> org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:251)
>
>          at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>
>          at
> java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
>
>          at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
>
>          at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
>
>          at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>
>          at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>
>          at java.lang.Thread.run(Thread.java:745)
>
>
>
>
>
>
>
>
>
>
>
> *Barış Akgün*
> Analitik Veri Ambarı ve Büyük Veri Yönetimi
> Uzman
>
> Tel
>
> :
>
> Dahili
>
> :
>
> Faks
>
> :
>
>
>
> Bu mesaj ve ekleri, mesajda gonderildigi belirtilen kisi/kisilere ozeldir
> ve gizlidir. Bu mesajin muhatabi olmamaniza ragmen tarafiniza ulasmis
> olmasi halinde mesaj iceriginin gizliligi ve bu gizlilik yukumlulugune
> uyulmasi zorunlulugu tarafiniz icin de soz konusudur. Mesaj ve eklerinde
> yer alan bilgilerin dogrulugu ve guncelligi konusunda gonderenin ya da
> sirketimizin herhangi bir sorumlulugu bulunmamaktadir. Sirketimiz mesajin
> ve bilgilerinin size degisiklige ugrayarak veya gec ulasmasindan,
> butunlugunun ve gizliliginin korunamamasindan, virus icermesinden ve
> bilgisayar sisteminize verebilecegi herhangi bir zarardan sorumlu tutulamaz.
>
> This message and attachments are confidential and intended solely for the
> individual(s) stated in this message. If you received this message although
> you are not the addressee, you are responsible to keep the message
> confidential. The sender has no responsibility for the accuracy or
> correctness of the information in the message and its attachments. Our
> company shall have no liability for any changes or late receiving, loss of
> integrity and confidentiality, viruses and any damages caused in anyway to
> your computer system.
>
>
> Bu mesaj ve ekleri, mesajda gonderildigi belirtilen kisi/kisilere ozeldir
> ve gizlidir. Bu mesajin muhatabi olmamaniza ragmen tarafiniza ulasmis
> olmasi halinde mesaj iceriginin gizliligi ve bu gizlilik yukumlulugune
> uyulmasi zorunlulugu tarafiniz icin de soz konusudur. Mesaj ve eklerinde
> yer alan bilgilerin dogrulugu ve guncelligi konusunda gonderenin ya da
> sirketimizin herhangi bir sorumlulugu bulunmamaktadir. Sirketimiz mesajin
> ve bilgilerinin size degisiklige ugrayarak veya gec ulasmasindan,
> butunlugunun ve gizliliginin korunamamasindan, virus icermesinden ve
> bilgisayar sisteminize verebilecegi herhangi bir zarardan sorumlu tutulamaz.
>
> This message and attachments are confidential and intended solely for the
> individual(s) stated in this message. If you received this message although
> you are not the addressee, you are responsible to keep the message
> confidential. The sender has no responsibility for the accuracy or
> correctness of the information in the message and its attachments. Our
> company shall have no liability for any changes or late receiving, loss of
> integrity and confidentiality, viruses and any damages caused in anyway to
> your computer system.
>



-- 
Anas Rabei
Senior Software Developer
Mubasher.info
anas.rabei@mubasher.info

Mime
View raw message