lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Schindler" <...@thetaphi.de>
Subject RE: Having some trouble running tests with custom codec
Date Mon, 12 Oct 2015 15:48:11 GMT
Hi,

Could you check that your JAR file really contains a folder:
META-INF/services/

With the file:
org.apache.lucene.codec.Codec

This file should contain your actual Codec's class name.

If this is all setup correctly, NamedSPILoader should see your codec and list it as part of
the error message.
Please also make sure, that your codec is initialized using a unique name.

> As to fullfeaturedness: Shouldn't getName() get inherited from Codec 
> (via FilterCodec), private string name getting set through calling 
> super("Hello", new FilterCodec())?

If you pass your own name in your own constructor, e.g.,

super("MyName", ....)

then you have to pass "MyName" to the test runner. Because Codec.getName() is final, it is
ensured that it is written to the commit (SegmentInfos#write(Directory)), so you should find
the name in your index.

Please note: In most cases you are more interested in writing your own PostingsFormat... Writing
your own codec is in most cases not needed... :-)

Uwe

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de


> -----Original Message-----
> From: Sigbjørn Lund Olsen [mailto:sigbjorn.lund.olsen@gmail.com]
> Sent: Monday, October 12, 2015 4:55 PM
> To: java-user@lucene.apache.org
> Subject: Re: Having some trouble running tests with custom codec
> 
> Ok, found that core/build.xml was the place to edit. The jar is now in the
> classpath, but I still get the same error:
> 
> Throwable #1: java.lang.IllegalArgumentException: An SPI class of type
> org.apache.lucene.codecs.Codec with name 'Hello' does not exist.  You need
> to add the corresponding JAR file supporting this SPI to your classpath.
> The current classpath supports the following names: [SimpleText, Asserting,
> CheapBastard, FastCompressingStoredFields,
> FastDecompressionCompressingStoredFields,
> HighCompressionCompressingStoredFields,
> DummyCompressingStoredFields, Lucene53]
> 
> ...which means - I assume - that there is some problem with HelloCodec not
> being loaded or recognized as a Codec.
> 
> Sincerely,
> Sigbjørn Lund Olsen
> 
> On 12 October 2015 at 14:07, Sigbjørn Lund Olsen <
> sigbjorn.lund.olsen@gmail.com> wrote:
> 
> > Thank you very much for your help :)
> >
> > I am not too familiar with ant build files. I tried adding
> > <pathelement location="/absolute/path/to/hello-1.0.jar"/> to
> > junit.classpath, test.classpath, classpath in test-framework/build.xml
> > and to classpath in lucene/build.xml, but the path does not show up
> > when running with -verbose option. Could you point me to the right place
> to modify build.xml?
> >
> > As to fullfeaturedness: Shouldn't getName() get inherited from Codec
> > (via FilterCodec), private string name getting set through calling
> > super("Hello", new FilterCodec())?
> >
> > Regards,
> > Sigbjørn Lund Olsen
> >
> > On 12 October 2015 at 09:12, Uwe Schindler <uwe@thetaphi.de> wrote:
> >
> >> Hi,
> >>
> >> One debugging solution is to use the "-verbose" parameter in the ANT
> >> command line. Shortly before starting the test runner it prints the
> >> command line of the test runner. Just check that your JAR file is
> >> listed as part of java's -classpath / -cp parameter.
> >>
> >> Uwe
> >>
> >> -----
> >> Uwe Schindler
> >> H.-H.-Meier-Allee 63, D-28213 Bremen
> >> http://www.thetaphi.de
> >> eMail: uwe@thetaphi.de
> >>
> >>
> >> > -----Original Message-----
> >> > From: Uwe Schindler [mailto:uwe@thetaphi.de]
> >> > Sent: Monday, October 12, 2015 8:47 AM
> >> > To: java-user@lucene.apache.org
> >> > Subject: RE: Having some trouble running tests with custom codec
> >> >
> >> > Hi,
> >> >
> >> > adding the codec JAR to the ANT classpath does not help, as it is
> >> > not
> >> part of
> >> > the test classpath:
> >> >
> >> > > Running *ant -Dtestcase=TestSegmentTermDocs -
> >> > Dtests.codec=HelloCodec
> >> > > -lib
> >> > > /path/to/hello-1.0.jar* test gives me the following error:
> >> > >
> >> >
> >> > * -lib cannot work as it just adds your JAR to ant's own classpath,
> >> > not
> >> the one
> >> > of the test runner!
> >> > * -Dtests.codec must be the name your own codec is returning in
> >> getName().
> >> > This is not the class name.
> >> >
> >> > To make this work, you have to pass your codec to test's classpath.
> >> This can
> >> > only be done with hacking Lucene's build.xml files, because the
> >> > test
> >> classpath
> >> > is isolated from anything outside, especially ANT's classpath.
> >> > In addition, your codec must be a full-featured codec with own
> >> > name, returned by getName(). This name should not contain the
> >> > String "Codec", just the plain name (like "Lucene53", "Hello",
> >> > "FooBar123"). This name
> >> must
> >> > be passed to "-Dtests.codec".
> >> >
> >> > Uwe
> >> >
> >> > -----
> >> > Uwe Schindler
> >> > H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de
> >> > eMail: uwe@thetaphi.de
> >> >
> >> >
> >> > > -----Original Message-----
> >> > > From: Sigbjørn Lund Olsen [mailto:sigbjorn.lund.olsen@gmail.com]
> >> > > Sent: Friday, October 09, 2015 4:08 PM
> >> > > To: java-user@lucene.apache.org
> >> > > Subject: Having some trouble running tests with custom codec
> >> > >
> >> > > As part of my master's thesis I am planning on implementing a custom
> >> > > Lucene codec for compression experiments.
> >> > >
> >> > > To get started with my prototype, I've tried following the
> >> > > instructions here (but using the more recent Lucene 5.3.1):
> >> > >
> >> > > http://opensourceconnections.com/blog/2013/06/05/build-your-
> own-
> >> > > lucene-codec/
> >> > >
> >> > >
> >> > > However, I am having some trouble getting the codec to load when
> >> > > running Lucene tests.
> >> > >
> >> > > I've uploaded a minimal project at
> >> > > https://github.com/sigbjornlo/codecs
> >> > >
> >> > > Running *mvn package* generates a jar with a file
> >> > > *META-INF/services/org.apache.lucene.codecs.Codec* containing a
> single
> >> > > line:
> >> > >
> >> > > edu.ntnu.sigbjornlo.codecs.HelloCodec
> >> > >
> >> > > ...which is where the codec class resides at. It's just a FilterCodec
> >> > > subclass that does nothing extra.
> >> > >
> >> > > Running *ant -Dtestcase=TestSegmentTermDocs -
> >> > Dtests.codec=HelloCodec
> >> > > -lib
> >> > > /path/to/hello-1.0.jar* test gives me the following error:
> >> > >
> >> > > java.lang.IllegalArgumentException: An SPI class of type
> >> > > org.apache.lucene.codecs.Codec with name 'HelloCodec' does not
> exist.
> >> > > You need to add the corresponding JAR file supporting this SPI to
your
> >> > classpath.
> >> > > The current classpath supports the following names: [SimpleText,
> >> > > Asserting, CheapBastard, FastCompressingStoredFields,
> >> > > FastDecompressionCompressingStoredFields,
> >> > > HighCompressionCompressingStoredFields,
> >> > > DummyCompressingStoredFields, Lucene53]
> >> > >
> >> > > What might I be doing wrong here? Is there any way to make Lucene's
> >> > > SPI loader log what it's trying and/or failing to load, and why?
> >> > > (Be adviced: I have also asked this question on StackOverflow:
> >> > > http://stackoverflow.com/questions/33039973/how-to-configure-
> lucene-
> >> > to
> >> > > -
> >> > > use-a-custom-codec-from-external-jar
> >> > > )
> >> > >
> >> > > Sincerely,
> >> > > Sigbjørn Lund Olsen
> >> >
> >> >
> >> > ---------------------------------------------------------------------
> >> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >> > For additional commands, e-mail: java-user-help@lucene.apache.org
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >> For additional commands, e-mail: java-user-help@lucene.apache.org
> >>
> >>
> >


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message