lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gayo.Diallo" <Gayo.Dia...@isped.u-bordeaux2.fr>
Subject RE: SPI class of type org.apache.lucene.codecs.Codec error
Date Thu, 22 Aug 2013 09:42:06 GMT
Hi,

Thank you both for the help.

Here are the contents you asked.

For the final Jar that we use with a thierd client application:
The content of META-INF/services/org.apache.lucene.codecs.Codec is
org.apache.lucene.codecs.lucene40.Lucene40Codec
org.apache.lucene.codecs.lucene3x.Lucene3xCodec
org.apache.lucene.codecs.lucene41.Lucene41Codec
org.apache.lucene.codecs.lucene42.Lucene42Codec

The content of META-INF/services/org.apache.lucene.codecs.PostingsFormat is
org.apache.lucene.codecs.lucene40.Lucene40PostingsFormat
org.apache.lucene.codecs.lucene41.Lucene41PostingsFormat

I do realize that there is no 
org.apache.lucene.codecs.lucene42.Lucene42PostingsFormat. Is that a possible 
reason for the issue?

Please not that lucene is primarely used by a maven project on which depends 
our final project. The content of 
META-INF/services/org.apache.lucene.codecsbothPostingsFormat andCodec of 
that project are similar to those I listed above.
The tow POM are listed here: 1) http://lesimisped.free.fr/servo/pom1IMM.xml 
(it uses 3 lucene jar); 2) final: 
http://lesimisped.free.fr/servo/pom2Final.xml. In both POM we have changed 
the configuration of the transformer after a suggestion from this list.
<transformers><transformer 
implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
<resource> META-INF/services/org.apache.lucene.codecs.Codec 
</resource></transformer>

I have also noticed that into the original lucene.core.jar, the 
/META-INF/services/o.p.l.c.PostingFormat  content is
org.apache.lucene.codecs.lucene40.Lucene40PostingsFormat
org.apache.lucene.codecs.lucene41.Lucene41PostingsFormat

there is no org.apache.lucene.codecs.lucene42.Lucene42PostingsFormat and in 
the repository /org/apache/lucene/codecs/lucene42/ of the jar there is 
nothing similari to  Lucene42PostingsXXX.class 
(http://lesimisped.free.fr/servo/lucene42.png) contrary to 
/org/apache/lucene/codecs/lucene41/ for instance 
(http://lesimisped.free.fr/servo/lucene40.png).

If you need to see any additional file/resource/config please let me know.

Best regards,
Gayo
-----Original Message-----
From: "Uwe Schindler" <uwe@thetaphi.de>
To: <java-user@lucene.apache.org>
Date: Wed, 21 Aug 2013 20:00:18 +0200
Subject: RE: SPI class of type org.apache.lucene.codecs.Codec error


Hi,

Just to check, otherwise we will never find out your problem: In your final 
merged JAR file, what are the contents of the File:
META-INF/services/org.apache.lucene.codecs.Codec
(please also list contents of PostingsFormat (because those are also loaded 
using SPI).

This file lists all Codec and PostingsFormat implementation classes. Without 
these files and without valid class names in it, Lucene cannot find the 
implementation classes. If all Codecs (namely the class files mentioned in 
your mail) are listed in this file Lucene can load them.

The names of codecs have nothing to do with the class names. You can have a 
codec named “FooBar” implemented by “MyCustomCodec.class”. The magic 
to find those classes is in the META-INF/services folder. The codec names 
are registered by the codec itself once it is loaded using the services 
files.

One important thing is also: Keep the whole lucene library complete! When 
“shading” the Uber-JAR, don’t allow the shade plugin to remove 
“unused” classes (because it would remove classes needed by lucene like 
the codec entry points).

If your META-INF/services folder file contents lists correctly all 
implementation classes and the classes are also available in the JAR file, 
then the only reason for this to fail might be crazy custom class loaders 
used in your application. As I have no access to your code, it is impossible 
to diagnose this.

Uwe

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de [http://www.thetaphi.de/]
eMail: uwe@thetaphi.de

From:Gayo Diallo [mailto:Gayo.Diallo@isped.u-bordeaux2.fr]
Sent: Wednesday, August 21, 2013 7:30 PM
To: java-user@lucene.apache.org
Subject: Re: SPI class of type org.apache.lucene.codecs.Codec error

Hi guys,

Thank you both for the help.

@Adriano: we have just tried the solution that worked for you without 
success.

@Duke: we could see in the fat jar in the Meta-Inf/services folder that we 
have org.apache.lucene.codecs.Codec and in org/apache/lucene/codecs we have 
the Codec class. However, the lucene42 folder under codecs does not include 
a classe Lucene42 instead, we have Lucene42Codec.class, 
Lucene42Codec$1.class and Lucene42Codec$2.class.
Is that normal as the error we got says "A SPI class of type 
org.apache.lucene.codecs.Codec with name 'Lucene42' does not exist"? The 
issue does not occurs as far as we are within Eclipse. But when we test the 
same client outside the IDE we got it...

Best regards,
Gayo
Le 21/08/2013 16:14, Adriano Ferreira a écrit :
I had this same problem using a library that used a new codec for lucene.In 
creating the jar maven was replacing the META-INF. I solved the problemwith 
the following configuration in 
pom.xml<configuration><transformers><transformerimplementation="org.apache.maven.plugins.shade.resource.AppendingTransformer"><resource>META-INF/services/org.apache.lucene.codecs.Codec

</ resource></ transformer></ transformers></ configuration>Adriano2013/8/21

Duke DAI <duke.dai.007@gmail.com> [mailto:duke.dai.007@gmail.com]
Hi GD,No idea why maven can't work, or the project structure is a 
littlecomplex?Also sure that lucene 2.9 has no problem with this because 
it does notdepend on SPI.First you need to check the fat jar file and the 
fileMETA-INF/services/o.a.l.codecs.Codec and its counterparts are there.If 
no, then the workaround is to have way to add all META-INF/services/xxxinto 
your fat jar.If yes, the problem may be with classloader, class
loader can't locate theSPI file within its context, I guess it's not this 
case.Hope this help.Best regards,DukeIf not now, when? If not me, who?On 
Tue, Aug 20, 2013 at 11:09 PM, Gayo Diallo <Gayo.Diallo@isped.u-bordeaux2.fr 
[mailto:Gayo.Diallo@isped.u-bordeaux2.fr]> wrote:
Thank you Duke for your help. We have already tried that solution aftera 
kindly advise from this mailing list. But we still having the error.Actually 
the POM of the project is herehttp://lesimisped.free.fr/servo/pom2.xml 
[http://lesimisped.free.fr/servo/pom2.xml]. It reuses a project which 
usesLucene, and the POM is this project 
ishttp://lesimisped.free.fr/servo/pom.xml 
[http://lesimisped.free.fr/servo/pom.xml].With a similar project which uses 
lucene 2.9 we didn't experienced suchan issue.Hope that may help.Best 
regards,GDLe 20/08/2013 16:10, Duke DAI a écrit :The 
linkhttp://maven.apache.org/plugins/maven-shade-plugin/examples/resource-transformers.html#ServicesResourceTransformerwillhelp.Best

regards,DukeIf not now, when? If not me, who?On Mon, Aug 19, 2013 at 8:48 
PM, Amal Kammoun <kammoun.amal25@gmail.com> 
[mailto:kammoun.amal25@gmail.com] <kammoun.amal25@gmail.com> 
[mailto:kammoun.amal25@gmail.com]wrote:Dear All,Please do you have any 
advice regarding the issue we raised  bellow and inthe previous post?Thank 
you for your help.best regards,Amalfind here the two links 
http://lesimisped.free.fr/servo/servicesP1.pnghttp://lesimisped.free.fr/servo/servicesP2.png

[http://lesimisped.free.fr/servo/servicesP1.pnghttp:/lesimisped.free.fr/servo/servicesP2.png]When

we open org.apache.lucene.codecs.Codec, the both have the same 
contentorg.apache.lucene.codecs.lucene40.Lucene40Codecorg.apache.lucene.codecs.lucene3x.Lucene3xCodecorg.apache.lucene.codecs.lucene41.Lucene41Codecorg.apache.lucene.codecs.lucene42.Lucene42CodecThanks

for your help.2013/8/18 Amal Kammoun <kammoun.amal25@gmail.com> 
[mailto:kammoun.amal25@gmail.com] <kammoun.amal25@gmail.com> 
[mailto:kammoun.amal25@gmail.com]Hi Uwe,Please find here the two 
linkshttp://lesimisped.free.fr/servo/servicesP1.pnghttp://lesimisped.free.fr/servo/servicesP2.pngWhen

we open org.apache.lucene.codecs.Codec, the both have the same<
/o:p>contentorg.apache.lucene.codecs.lucene40.Lucene40Codecorg.apache.lucene.codecs.lucene3x.Lucene3xCodecorg.apache.lucene.codecs.lucene41.Lucene41Codecorg.apache.lucene.codecs.lucene42.Lucene42CodecThanks

for your help.Amal2013/8/16 Uwe Schindler <uwe@thetaphi.de> 
[mailto:uwe@thetaphi.de] <uwe@thetaphi.de> [mailto:uwe@thetaphi.de]Hi,There 
is no screen shot attached to your mail. Please put it somewhereinthe web 
and send a link.Uwe-----Uwe SchindlerH.-H.-Meier-Allee 63, D-28213 
Bremen<http://www.thetaphi.de/> [http://www.thetaphi.de/] 
<http://www.thetaphi.de/> [http://www.thetaphi.de/] http://www.thetaphi.de 
[http://www.thetaphi.de/]eMail: uwe@thetaphi.de 
[mailto:uwe@thetaphi.de]From: Amal Kammoun [mailto:kammoun.amal25@gmail.com 
[mailto:kammoun.amal25@gmail.com] <kammoun.amal25@gmail.com> 
[mailto:kammoun.amal25@gmail.com]]Sent: Friday, August 16, 2013 6:08 PMTo: 
java-user@lucene.apache.org [mailto:java-user@lucene.apache.org]Subject: Re: 
SPI class of type org.apache.lucene.codecs.Codec errorThank you,We are using 
Eclipse under Linux, and Java 1.7. Maven Shade is used forassembling the 
project (P1) which depends on another project which usesLucene (P2).  P2 
uses lucene.core, lucene.queryparser,lucene.analyzercommon.Please find 
enclosed a screen-shot of the services of the two Jar.We use to use previous 
versions of lucene (2.x) and we perform the sameprocess of assembling 
without issue.Hope this could help.best regards,Amal2013/8/16 Uwe Schindler 
<uwe@thetaphi.de> [mailto:uwe@thetaphi.de] <uwe@thetaphi.de> 
[mailto:uwe@thetaphi.de]Hi,Maven makes it even simplier to handle this! The 
problem may be (I amnotsure not sure, because I don’t know your setup):It 
seems that you are using the Maven Shade Plugin to merge all JARfilesinto 
one BIG JAR file. During this 
step, you may be missing to merge allthe data correctly in your JAR files. 
Lucene JARs also contain metadataandother resources (in addition to class 
files) in the META-INF folders andthose are generally not always merged by 
all those tools, so those mustbecopied and merged if multiple 
META-INF/services with same name exist.TheMaven-Shade-Plugin can do this for 
you, 
see:http://maven.apache.org/plugins/maven-shade-plugin/examples/resource-transformers.html

[http://maven.apache.org/plugins/maven-shade-plugin/examples/resource-transformers.html]Especially:http://maven.apache.org/plugins/maven-shade-plugin/examples/resource-transformers.html#ServicesResourceTransformer

[http://maven.apache.org/plugins/maven-shade-plugin/examples/resource-transformers.html#ServicesResourceTransformer]It

is recommended to use the ServicesResourceTransformer option.Ideally, you 
should not change or transform JAR files of Lucene at alland not merge them, 
just ship them with your project as is. Please keepthem separate, only for 
special use cases like autostarting double-clickJAR files, merge them 
otherwise management gets crazy.In any case, please check your classpath:- 
Are the *unmodified* lucene-core.jar files in it?- Don't use crazy 
classloader hierarchies. Keep all Lucene code togetherin one classloader (so 
don't place Lucene JAR files outside your webapp,but the code using lucene 
inside a webapp).- If you create uber-JARS (which is a bad idea in general), 
useMaven-Shade plugin and configure it correctly. The Uber-JAR file 
mustcontain a "META-INF/services" folder with some 
org.apache.lucene.index.*files.-----Uwe SchindlerH.-H.-Meier-Allee 63, 
D-28213 Bremenhttp://www.thetaphi.deeMail: uwe@thetaphi.de 
[mailto:uwe@thetaphi.de]-----Original Message-----From: Amal Kammoun 
[mailto:kammoun.amal25@gmail.com [mailto:kammoun.amal25@gmail.com] 
<kammoun.amal25@gmail.com> [mailto:kammoun.amal25@gmail.com]]Sent: Friday, 
August 16, 2013 5:39 PMTo: java-user@lucene.apache.org 
[mailto:java-user@lucene.apache.org]Subject: SPI class of type 
org.apache.lucene.codecs.Codec errorHi,We are working on a project which 
uses Lucene 4.2.1. Actually we arefacingan error message 
"java.lang.ExceptionInInitializerError". We are using Maven for assembling 
theprojectand we have a dependency between two projects. When we do the 
testwith eclipse it works fine. However, whe
n we incorporate our jar in aclientthat is tested outside Eclipse we got 
thejava.lang.ExceptionInInitializerError.We are doing workaround to overcome 
the issue since yesterday, we gotthesame issue with both versions of Lucene 
4.2.0 and 4.2.1.Have you ever experienced such an issue with maven? Are the 
newerLucene versions safer from such an issue?Here is the rest of the 
message errorCaused by: java.lang.IllegalArgumentException: A SPI class of 
typeorg.apache.lucene.codecs.Codec with name 'Lucene42' does not 
exist.Youneed to add the corresponding JAR file supporting this SPI to 
yourclasspath.The current classpath supports the following names.Thank you a 
lot in advance for your support.Best 
regards,---------------------------------------------------------------------To 
unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org 
[mailto:java-user-unsubscribe@lucene.apache.org]For additional commands, 
e-mail: java-user-help@lucene.apache.org 
[mailto:java-user-help@lucene.apache.org]--Dr Gayo DialloPhD, Associate 
Professor in Computer Sciences[image: Université Bordeaux Segalen] 
<http://www.univ-bordeauxsegalen.fr> [http://www.univ-bordeauxsegalen.fr/] 
[image:ISPED] <http://www.isped.u-bordeaux2.fr> 
[http://www.isped.u-bordeaux2.fr/]ISPED, Université Bordeaux SegalenBât. 
NB, 1er étage, Bureau 30146 rue Léo Saignat 33076 Bordeaux 
cedexFranceTél. +33 (0)5 57579540▪ Fax +33 
(0)556240081Gayo.Diallo@isped.u-bordeaux2.fr 
[mailto:Gayo.Diallo@isped.u-bordeaux2.fr]www.gayodiallo.org 
[http://www.gayodiallo.org/]www.univ-bordeauxsegalen.fr 
[http://www.univ-bordeauxsegalen.fr/]---------------------------------------------------------------------To

unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org 
[mailto:java-user-unsubscribe@lucene.apache.org]For additional commands, 
e-mail: java-user-help@lucene.apache.org 
[mailto:java-user-help@lucene.apache.org]

--
Dr Gayo Diallo
PhD, Associate Professor in Computer Sciences
[http://www.univ-bordeauxsegalen.fr/] [http://www.isped.u-bordeaux2.fr/]
ISPED, Université Bordeaux Segalen
Bât. NB, 1er étage, Bureau 30
146 rue Léo Saignat 33076 Bordeaux cedex
France
Tél. +33 (0)5 57579540▪ Fax +33 (0)556240081

Gayo.Diallo@isped.u-bordeaux2.fr [mailto:Gayo.Diallo@isped.u-bordeaux2.fr]
www.gayodiallo.org [http://www.gayodiallo.org/]
www.univ-bordeauxsegalen.fr 
[https://courriel.isped.u-bordeaux2.fr/www.univ-bordeauxsegalen.fr]
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message