lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Schindler" <...@thetaphi.de>
Subject RE: Having some trouble running tests with custom codec
Date Mon, 12 Oct 2015 17:17:33 GMT
Hi,

 

I tested your GitHub project, works without any problems.

 

With ANT 1.8 it also passes without changing the build.xmls (it might be different in 1.9,
because newer ANT versions hide their own libs from passing down to tasks).

 

If I start with fake codec name (I copied the JAR file to my local folder):

 

core>ant -lib hello-1.0.jar -verbose test -Dtests.codec=doesntwork

 

it complains successfully:

 

   [junit4]    > Throwable #1: java.lang.IllegalArgumentException: An SPI class of type
org.apache.lucene.codecs.Codec with name 'doesntwork' does not exist.  You need to add the
corresponding JAR file supporting this SPI to your classpath.  The current classpath supports
the following names: [SimpleText, Asserting, CheapBastard, FastCompressingStoredFields, FastDecompressionCompressingStoredFields,
HighCompressionCompressingStoredFields, DummyCompressingStoredFields, Lucene53, Hello]

   [junit4]    >        at __randomizedtesting.SeedInfo.seed([D2496AE144B72058]:0)

   [junit4]    >        at org.apache.lucene.util.NamedSPILoader.lookup(NamedSPILoader.java:109)

   [junit4]    >        at org.apache.lucene.codecs.Codec.forName(Codec.java:113)

   [junit4]    >        at java.lang.Thread.run(Thread.java:745)

   [junit4] Completed [1/410] on J0 in 0.90s, 0 tests, 1 error <<< FAILURES!

 

It also works if I edit core’s build.xml:

 

  <path id="test.classpath">

    <pathelement location="${common.dir}/build/codecs/classes/java"/>

    <pathelement location="${common.dir}/build/test-framework/classes/java"/>

    <path refid="junit-path"/>

    <pathelement location="${build.dir}/classes/java"/>

    <pathelement location="${build.dir}/classes/test"/>

    <pathelement location="hello-1.0.jar"/>

  </path>

 

And then run:

 

core>ant test -Dtests.codec=doesntwork

 

It still lists the “Hello” codec successfully.

 

For both variants, if I pass "Hello" it passes successfully:

 

core>ant -lib hello-1.0.jar -verbose test -Dtests.codec=Hello

 

All tests pass! Not sure what your problem is.

 

Uwe

 

-----

Uwe Schindler

H.-H.-Meier-Allee 63, D-28213 Bremen

http://www.thetaphi.de

eMail: uwe@thetaphi.de

 

 

> -----Original Message-----

> From: Sigbjørn Lund Olsen [mailto:sigbjorn.lund.olsen@gmail.com]

> Sent: Monday, October 12, 2015 6:08 PM

> To: java-user@lucene.apache.org

> Subject: Re: Having some trouble running tests with custom codec

> 

> I'm mostly interested in replacing the StoredField implementation. I'd have

> to make a Codec for that, right?

> 

> The jar has a META-INF/services/org.apache.lucene.codecs.Codec file with

> the fully qualified domain name for the class. (I assume ".codec.Codec" was a

> typo?)

> 

> The source code is available at:  <https://github.com/sigbjornlo/codecs> https://github.com/sigbjornlo/codecs

> 

> Sigbjørn

> 

> On 12 October 2015 at 17:48, Uwe Schindler < <mailto:uwe@thetaphi.de> uwe@thetaphi.de>
wrote:

> 

> > Hi,

> >

> > Could you check that your JAR file really contains a folder:

> > META-INF/services/

> >

> > With the file:

> > org.apache.lucene.codec.Codec

> >

> > This file should contain your actual Codec's class name.

> >

> > If this is all setup correctly, NamedSPILoader should see your codec

> > and list it as part of the error message.

> > Please also make sure, that your codec is initialized using a unique name.

> >

> > > As to fullfeaturedness: Shouldn't getName() get inherited from Codec

> > > (via FilterCodec), private string name getting set through calling

> > > super("Hello", new FilterCodec())?

> >

> > If you pass your own name in your own constructor, e.g.,

> >

> > super("MyName", ....)

> >

> > then you have to pass "MyName" to the test runner. Because

> > Codec.getName() is final, it is ensured that it is written to the

> > commit (SegmentInfos#write(Directory)), so you should find the name in

> your index.

> >

> > Please note: In most cases you are more interested in writing your own

> > PostingsFormat... Writing your own codec is in most cases not

> > needed... :-)

> >

> > Uwe

> >

> > -----

> > Uwe Schindler

> > H.-H.-Meier-Allee 63, D-28213 Bremen

> >  <http://www.thetaphi.de> http://www.thetaphi.de

> > eMail:  <mailto:uwe@thetaphi.de> uwe@thetaphi.de

> >

> >

> > > -----Original Message-----

> > > From: Sigbjørn Lund Olsen [ <mailto:sigbjorn.lund.olsen@gmail.com> mailto:sigbjorn.lund.olsen@gmail.com]

> > > Sent: Monday, October 12, 2015 4:55 PM

> > > To:  <mailto:java-user@lucene.apache.org> java-user@lucene.apache.org

> > > Subject: Re: Having some trouble running tests with custom codec

> > >

> > > Ok, found that core/build.xml was the place to edit. The jar is now

> > > in

> > the

> > > classpath, but I still get the same error:

> > >

> > > Throwable #1: java.lang.IllegalArgumentException: An SPI class of

> > > type org.apache.lucene.codecs.Codec with name 'Hello' does not

> > > exist.  You

> > need

> > > to add the corresponding JAR file supporting this SPI to your classpath.

> > > The current classpath supports the following names: [SimpleText,

> > Asserting,

> > > CheapBastard, FastCompressingStoredFields,

> > > FastDecompressionCompressingStoredFields,

> > > HighCompressionCompressingStoredFields,

> > > DummyCompressingStoredFields, Lucene53]

> > >

> > > ...which means - I assume - that there is some problem with

> > > HelloCodec

> > not

> > > being loaded or recognized as a Codec.

> > >

> > > Sincerely,

> > > Sigbjørn Lund Olsen

> > >

> > > On 12 October 2015 at 14:07, Sigbjørn Lund Olsen <

> > >  <mailto:sigbjorn.lund.olsen@gmail.com> sigbjorn.lund.olsen@gmail.com>
wrote:

> > >

> > > > Thank you very much for your help :)

> > > >

> > > > I am not too familiar with ant build files. I tried adding

> > > > <pathelement location="/absolute/path/to/hello-1.0.jar"/> to

> > > > junit.classpath, test.classpath, classpath in

> > > > test-framework/build.xml and to classpath in lucene/build.xml, but

> > > > the path does not show up when running with -verbose option. Could

> > > > you point me to the right

> > place

> > > to modify build.xml?

> > > >

> > > > As to fullfeaturedness: Shouldn't getName() get inherited from

> > > > Codec (via FilterCodec), private string name getting set through

> > > > calling super("Hello", new FilterCodec())?

> > > >

> > > > Regards,

> > > > Sigbjørn Lund Olsen

> > > >

> > > > On 12 October 2015 at 09:12, Uwe Schindler < <mailto:uwe@thetaphi.de>
uwe@thetaphi.de>

> wrote:

> > > >

> > > >> Hi,

> > > >>

> > > >> One debugging solution is to use the "-verbose" parameter in the

> > > >> ANT command line. Shortly before starting the test runner it

> > > >> prints the command line of the test runner. Just check that your

> > > >> JAR file is listed as part of java's -classpath / -cp parameter.

> > > >>

> > > >> Uwe

> > > >>

> > > >> -----

> > > >> Uwe Schindler

> > > >> H.-H.-Meier-Allee 63, D-28213 Bremen  <http://www.thetaphi.de>
http://www.thetaphi.de

> > > >> eMail:  <mailto:uwe@thetaphi.de> uwe@thetaphi.de

> > > >>

> > > >>

> > > >> > -----Original Message-----

> > > >> > From: Uwe Schindler [ <mailto:uwe@thetaphi.de> mailto:uwe@thetaphi.de]

> > > >> > Sent: Monday, October 12, 2015 8:47 AM

> > > >> > To:  <mailto:java-user@lucene.apache.org> java-user@lucene.apache.org

> > > >> > Subject: RE: Having some trouble running tests with custom codec

> > > >> >

> > > >> > Hi,

> > > >> >

> > > >> > adding the codec JAR to the ANT classpath does not help, as it
is

> > > >> > not

> > > >> part of

> > > >> > the test classpath:

> > > >> >

> > > >> > > Running *ant -Dtestcase=TestSegmentTermDocs -

> > > >> > Dtests.codec=HelloCodec

> > > >> > > -lib

> > > >> > > /path/to/hello-1.0.jar* test gives me the following error:

> > > >> > >

> > > >> >

> > > >> > * -lib cannot work as it just adds your JAR to ant's own classpath,

> > > >> > not

> > > >> the one

> > > >> > of the test runner!

> > > >> > * -Dtests.codec must be the name your own codec is returning
in

> > > >> getName().

> > > >> > This is not the class name.

> > > >> >

> > > >> > To make this work, you have to pass your codec to test's classpath.

> > > >> This can

> > > >> > only be done with hacking Lucene's build.xml files, because the

> > > >> > test

> > > >> classpath

> > > >> > is isolated from anything outside, especially ANT's classpath.

> > > >> > In addition, your codec must be a full-featured codec with own

> > > >> > name, returned by getName(). This name should not contain the

> > > >> > String "Codec", just the plain name (like "Lucene53", "Hello",

> > > >> > "FooBar123"). This name

> > > >> must

> > > >> > be passed to "-Dtests.codec".

> > > >> >

> > > >> > Uwe

> > > >> >

> > > >> > -----

> > > >> > Uwe Schindler

> > > >> > H.-H.-Meier-Allee 63, D-28213 Bremen  <http://www.thetaphi.de>
http://www.thetaphi.de

> > > >> > eMail:  <mailto:uwe@thetaphi.de> uwe@thetaphi.de

> > > >> >

> > > >> >

> > > >> > > -----Original Message-----

> > > >> > > From: Sigbjørn Lund Olsen [ <mailto:sigbjorn.lund.olsen@gmail.com>
mailto:sigbjorn.lund.olsen@gmail.com]

> > > >> > > Sent: Friday, October 09, 2015 4:08 PM

> > > >> > > To:  <mailto:java-user@lucene.apache.org> java-user@lucene.apache.org

> > > >> > > Subject: Having some trouble running tests with custom codec

> > > >> > >

> > > >> > > As part of my master's thesis I am planning on implementing
a

> > custom

> > > >> > > Lucene codec for compression experiments.

> > > >> > >

> > > >> > > To get started with my prototype, I've tried following the

> > > >> > > instructions here (but using the more recent Lucene 5.3.1):

> > > >> > >

> > > >> > >  <http://opensourceconnections.com/blog/2013/06/05/build-your->
http://opensourceconnections.com/blog/2013/06/05/build-your-

> > > own-

> > > >> > > lucene-codec/

> > > >> > >

> > > >> > >

> > > >> > > However, I am having some trouble getting the codec to load

> when

> > > >> > > running Lucene tests.

> > > >> > >

> > > >> > > I've uploaded a minimal project at

> > > >> > >  <https://github.com/sigbjornlo/codecs> https://github.com/sigbjornlo/codecs

> > > >> > >

> > > >> > > Running *mvn package* generates a jar with a file

> > > >> > > *META-INF/services/org.apache.lucene.codecs.Codec* containing

> a

> > > single

> > > >> > > line:

> > > >> > >

> > > >> > > edu.ntnu.sigbjornlo.codecs.HelloCodec

> > > >> > >

> > > >> > > ...which is where the codec class resides at. It's just
a

> > FilterCodec

> > > >> > > subclass that does nothing extra.

> > > >> > >

> > > >> > > Running *ant -Dtestcase=TestSegmentTermDocs -

> > > >> > Dtests.codec=HelloCodec

> > > >> > > -lib

> > > >> > > /path/to/hello-1.0.jar* test gives me the following error:

> > > >> > >

> > > >> > > java.lang.IllegalArgumentException: An SPI class of type

> > > >> > > org.apache.lucene.codecs.Codec with name 'HelloCodec' does
not

> > > exist.

> > > >> > > You need to add the corresponding JAR file supporting this
SPI to

> > your

> > > >> > classpath.

> > > >> > > The current classpath supports the following names: [SimpleText,

> > > >> > > Asserting, CheapBastard, FastCompressingStoredFields,

> > > >> > > FastDecompressionCompressingStoredFields,

> > > >> > > HighCompressionCompressingStoredFields,

> > > >> > > DummyCompressingStoredFields, Lucene53]

> > > >> > >

> > > >> > > What might I be doing wrong here? Is there any way to make

> > Lucene's

> > > >> > > SPI loader log what it's trying and/or failing to load,
and why?

> > > >> > > (Be adviced: I have also asked this question on StackOverflow:

> > > >> > >  <http://stackoverflow.com/questions/33039973/how-to-configure->
http://stackoverflow.com/questions/33039973/how-to-configure-

> > > lucene-

> > > >> > to

> > > >> > > -

> > > >> > > use-a-custom-codec-from-external-jar

> > > >> > > )

> > > >> > >

> > > >> > > Sincerely,

> > > >> > > Sigbjørn Lund Olsen

> > > >> >

> > > >> >

> > > >> >

> > ---------------------------------------------------------------------

> > > >> > To unsubscribe, e-mail:  <mailto:java-user-unsubscribe@lucene.apache.org>
java-user-unsubscribe@lucene.apache.org

> > > >> > For additional commands, e-mail:  <mailto:java-user-help@lucene.apache.org>
java-user-

 <mailto:java-user-help@lucene.apache.org> > help@lucene.apache.org

> > > >>

> > > >>

> > > >> ---------------------------------------------------------------------

> > > >> To unsubscribe, e-mail:  <mailto:java-user-unsubscribe@lucene.apache.org>
java-user-unsubscribe@lucene.apache.org

> > > >> For additional commands, e-mail:  <mailto:java-user-help@lucene.apache.org>
java-user-help@lucene.apache.org

> > > >>

> > > >>

> > > >

> >

> >

> > ---------------------------------------------------------------------

> > To unsubscribe, e-mail:  <mailto:java-user-unsubscribe@lucene.apache.org>
java-user-unsubscribe@lucene.apache.org

> > For additional commands, e-mail:  <mailto:java-user-help@lucene.apache.org>
java-user-help@lucene.apache.org

> >

> >


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message