lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Schindler" <...@thetaphi.de>
Subject RE: SPI class of type org.apache.lucene.codecs.Codec error
Date Wed, 21 Aug 2013 18:00:18 GMT
Hi,

 

Just to check, otherwise we will never find out your problem: In your final merged JAR file,
what are the contents of the File:

META-INF/services/org.apache.lucene.codecs.Codec

(please also list contents of PostingsFormat (because those are also loaded using SPI).

 

This file lists all Codec and PostingsFormat implementation classes. Without these files and
without valid class names in it, Lucene cannot find the implementation classes. If all Codecs
(namely the class files mentioned in your mail) are listed in this file Lucene can load them.

 

The names of codecs have nothing to do with the class names. You can have a codec named “FooBar”
implemented by “MyCustomCodec.class”. The magic to find those classes is in the META-INF/services
folder. The codec names are registered by the codec itself once it is loaded using the services
files.

 

One important thing is also: Keep the whole lucene library complete! When “shading” the
Uber-JAR, don’t allow the shade plugin to remove “unused” classes (because it would
remove classes needed by lucene like the codec entry points).

 

If your META-INF/services folder file contents lists correctly all implementation classes
and the classes are also available in the JAR file, then the only reason for this to fail
might be crazy custom class loaders used in your application. As I have no access to your
code, it is impossible to diagnose this.

 

Uwe

 

-----

Uwe Schindler

H.-H.-Meier-Allee 63, D-28213 Bremen

 <http://www.thetaphi.de/> http://www.thetaphi.de

eMail: uwe@thetaphi.de

 

From: Gayo Diallo [mailto:Gayo.Diallo@isped.u-bordeaux2.fr] 
Sent: Wednesday, August 21, 2013 7:30 PM
To: java-user@lucene.apache.org
Subject: Re: SPI class of type org.apache.lucene.codecs.Codec error

 

Hi guys,

Thank you both for the help.

@Adriano: we have just tried the solution that worked for you without success.

@Duke: we could see in the fat jar in the Meta-Inf/services folder that we have org.apache.lucene.codecs.Codec
and in org/apache/lucene/codecs we have the Codec class. However, the lucene42 folder under
codecs does not include a classe Lucene42 instead, we have Lucene42Codec.class, Lucene42Codec$1.class
and Lucene42Codec$2.class. 
Is that normal as the error we got says "A SPI class of type org.apache.lucene.codecs.Codec
with name 'Lucene42' does not exist"? The issue does not occurs as far as we are within Eclipse.
But when we test the same client outside the IDE we got it...

Best regards,
Gayo




Le 21/08/2013 16:14, Adriano Ferreira a écrit :

I had this same problem using a library that used a new codec for lucene.
In creating the jar maven was replacing the META-INF. I solved the problem
with the following configuration in pom.xml
 
<configuration>
               <transformers>
                 <transformer
implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
                   <resource>
META-INF/services/org.apache.lucene.codecs.Codec </ resource>
                 </ transformer>
               </ transformers>
</ configuration>
 
Adriano
 
 
 
2013/8/21 Duke DAI  <mailto:duke.dai.007@gmail.com> <duke.dai.007@gmail.com>
 

Hi GD,
 
No idea why maven can't work, or the project structure is a little
complex?
 
Also sure that lucene 2.9 has no problem with this because it does not
depend on SPI.
 
First you need to check the fat jar file and the file
META-INF/services/o.a.l.codecs.Codec and its counterparts are there.
If no, then the workaround is to have way to add all META-INF/services/xxx
into your fat jar.
If yes, the problem may be with classloader, classloader can't locate the
SPI file within its context, I guess it's not this case.
 
Hope this help.
 
Best regards,
Duke
If not now, when? If not me, who?
 
 
On Tue, Aug 20, 2013 at 11:09 PM, Gayo Diallo <
Gayo.Diallo@isped.u-bordeaux2.fr> wrote:
 

 Thank you Duke for your help. We have already tried that solution after
a kindly advise from this mailing list. But we still having the error.
 
Actually the POM of the project is here
http://lesimisped.free.fr/servo/pom2.xml. It reuses a project which uses
Lucene, and the POM is this project is
http://lesimisped.free.fr/servo/pom.xml.
With a similar project which uses lucene 2.9 we didn't experienced such
an issue.
 
Hope that may help.
 
Best regards,
GD
 
Le 20/08/2013 16:10, Duke DAI a écrit :
 
The linkhttp://maven.apache.org/plugins/maven-shade-plugin/examples/resource-transformers.html#ServicesResourceTransformer
will
help.
 
Best regards,
Duke
If not now, when? If not me, who?
 
 
On Mon, Aug 19, 2013 at 8:48 PM, Amal Kammoun  <mailto:kammoun.amal25@gmail.com> <kammoun.amal25@gmail.com>
 <mailto:kammoun.amal25@gmail.com> <kammoun.amal25@gmail.com>wrote:
 
 
 Dear All,
 
Please do you have any advice regarding the issue we raised  bellow and in
the previous post?
 
Thank you for your help.
 
best regards,
Amal
 
 
find here the two links http://lesimisped.free.fr/servo/servicesP1.pnghttp://lesimisped.free.fr/servo/servicesP2.png
<http://lesimisped.free.fr/servo/servicesP1.pnghttp:/lesimisped.free.fr/servo/servicesP2.png>

 
When we open org.apache.lucene.codecs.Codec, the both have the same content
org.apache.lucene.codecs.lucene40.Lucene40Codec
org.apache.lucene.codecs.lucene3x.Lucene3xCodec
org.apache.lucene.codecs.lucene41.Lucene41Codec
org.apache.lucene.codecs.lucene42.Lucene42Codec
 
Thanks for your help.
 
 
 
2013/8/18 Amal Kammoun  <mailto:kammoun.amal25@gmail.com> <kammoun.amal25@gmail.com>
 <mailto:kammoun.amal25@gmail.com> <kammoun.amal25@gmail.com>
 
 Hi Uwe,
 
Please find here the two linkshttp://lesimisped.free.fr/servo/servicesP1.pnghttp://lesimisped.free.fr/servo/servicesP2.png
 
When we open org.apache.lucene.codecs.Codec, the both have the same
 
 content
 
 org.apache.lucene.codecs.lucene40.Lucene40Codec
org.apache.lucene.codecs.lucene3x.Lucene3xCodec
org.apache.lucene.codecs.lucene41.Lucene41Codec
org.apache.lucene.codecs.lucene42.Lucene42Codec
 
Thanks for your help.
Amal
 
 
2013/8/16 Uwe Schindler  <mailto:uwe@thetaphi.de> <uwe@thetaphi.de>  <mailto:uwe@thetaphi.de>
<uwe@thetaphi.de>
 
 Hi,
 
 
 
There is no screen shot attached to your mail. Please put it somewhere
 
 in
 
 the web and send a link.
 
 
 
Uwe
 
 
 
-----
 
Uwe Schindler
 
H.-H.-Meier-Allee 63, D-28213 Bremen
 
  <http://www.thetaphi.de/> <http://www.thetaphi.de/>  <http://www.thetaphi.de/>
<http://www.thetaphi.de/> http://www.thetaphi.de
 
eMail: uwe@thetaphi.de
 
 
 
From: Amal Kammoun [mailto:kammoun.amal25@gmail.com  <mailto:kammoun.amal25@gmail.com>
<kammoun.amal25@gmail.com>]
Sent: Friday, August 16, 2013 6:08 PM
To: java-user@lucene.apache.org
Subject: Re: SPI class of type org.apache.lucene.codecs.Codec error
 
 
 
Thank you,
 
We are using Eclipse under Linux, and Java 1.7. Maven Shade is used for
assembling the project (P1) which depends on another project which uses
Lucene (P2).  P2 uses lucene.core, lucene.queryparser,
lucene.analyzercommon.
 
Please find enclosed a screen-shot of the services of the two Jar.
 
We use to use previous versions of lucene (2.x) and we perform the same
process of assembling without issue.
 
Hope this could help.
 
best regards,
 
Amal
 
 
 
 
 
2013/8/16 Uwe Schindler  <mailto:uwe@thetaphi.de> <uwe@thetaphi.de>  <mailto:uwe@thetaphi.de>
<uwe@thetaphi.de>
 
Hi,
 
Maven makes it even simplier to handle this! The problem may be (I am
 
 not
 
 sure not sure, because I don’t know your setup):
It seems that you are using the Maven Shade Plugin to merge all JAR
 
 files
 
 into one BIG JAR file. During this step, you may be missing to merge all
the data correctly in your JAR files. Lucene JARs also contain metadata
 
 and
 
 other resources (in addition to class files) in the META-INF folders and
those are generally not always merged by all those tools, so those must
 
 be
 
 copied and merged if multiple META-INF/services with same name exist.
 
 The
 
 Maven-Shade-Plugin can do this for you, see:
 
 
 
 
 http://maven.apache.org/plugins/maven-shade-plugin/examples/resource-transformers.html
 
 Especially:
 
 
 http://maven.apache.org/plugins/maven-shade-plugin/examples/resource-transformers.html#ServicesResourceTransformer
 
 It is recommended to use the ServicesResourceTransformer option.
 
Ideally, you should not change or transform JAR files of Lucene at all
and not merge them, just ship them with your project as is. Please keep
them separate, only for special use cases like autostarting double-click
JAR files, merge them otherwise management gets crazy.
 
In any case, please check your classpath:
- Are the *unmodified* lucene-core.jar files in it?
- Don't use crazy classloader hierarchies. Keep all Lucene code together
in one classloader (so don't place Lucene JAR files outside your webapp,
but the code using lucene inside a webapp).
- If you create uber-JARS (which is a bad idea in general), use
Maven-Shade plugin and configure it correctly. The Uber-JAR file must
contain a "META-INF/services" folder with some org.apache.lucene.index.*
files.
 
-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremenhttp://www.thetaphi.de
eMail: uwe@thetaphi.de
 
 
 -----Original Message-----
From: Amal Kammoun [mailto:kammoun.amal25@gmail.com  <mailto:kammoun.amal25@gmail.com>
<kammoun.amal25@gmail.com>]
Sent: Friday, August 16, 2013 5:39 PM
To: java-user@lucene.apache.org
Subject: SPI class of type org.apache.lucene.codecs.Codec error
 
Hi,
 
We are working on a project which uses Lucene 4.2.1. Actually we are
 
 facing
 
 an error message "java.lang.
ExceptionInInitializerError". We are using Maven for assembling the
 
 project
 
 and we have a dependency between two projects. When we do the test
with eclipse it works fine. However, when we incorporate our jar in a
 
 client
 
 that is tested outside Eclipse we got the
 
 java.lang.ExceptionInInitializerError.
 
 We are doing workaround to overcome the issue since yesterday, we got
 
 the
 
 same issue with both versions of Lucene 4.2.0 and 4.2.1.
 
Have you ever experienced such an issue with maven? Are the newer
Lucene versions safer from such an issue?
 
Here is the rest of the message error
Caused by: java.lang.
IllegalArgumentException: A SPI class of type
org.apache.lucene.codecs.Codec with name 'Lucene42' does not exist.
 
  You
 
  need to add the corresponding JAR file supporting this SPI to your
classpath.The current classpath supports the following names.
 
Thank you a lot in advance for your support.
Best regards,
 
 ---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
 
 
 
 
--
 
Dr Gayo Diallo
PhD, Associate Professor in Computer Sciences
[image: Université Bordeaux Segalen]  <http://www.univ-bordeauxsegalen.fr> <http://www.univ-bordeauxsegalen.fr>
[image:
ISPED]  <http://www.isped.u-bordeaux2.fr> <http://www.isped.u-bordeaux2.fr>
 
ISPED, Université Bordeaux Segalen
Bât. NB, 1er étage, Bureau 30
146 rue Léo Saignat 33076 Bordeaux cedex
France
Tél. +33 (0)5 57579540▪ Fax +33 (0)556240081
 
Gayo.Diallo@isped.u-bordeaux2.fr
www.gayodiallo.org
www.univ-bordeauxsegalen.fr
 
 
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
 

 
 

 
 

 

-- 

Dr Gayo Diallo
PhD, Associate Professor in Computer Sciences

 <http://www.univ-bordeauxsegalen.fr> Université Bordeaux Segalen <http://www.isped.u-bordeaux2.fr>
ISPED

ISPED, Université Bordeaux Segalen
Bât. NB, 1er étage, Bureau 30
146 rue Léo Saignat 33076 Bordeaux cedex
France
Tél. +33 (0)5 57579540▪ Fax +33 (0)556240081

 <mailto:Gayo.Diallo@isped.u-bordeaux2.fr> Gayo.Diallo@isped.u-bordeaux2.fr
 <http://www.gayodiallo.org> www.gayodiallo.org
www.univ-bordeauxsegalen.fr


Mime
  • Unnamed multipart/related (inline, None, 0 bytes)
View raw message