lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sigbjørn Lund Olsen <sigbjorn.lund.ol...@gmail.com>
Subject Re: Having some trouble running tests with custom codec
Date Mon, 12 Oct 2015 16:08:10 GMT
I'm mostly interested in replacing the StoredField implementation. I'd have
to make a Codec for that, right?

The jar has a META-INF/services/org.apache.lucene.codecs.Codec file with
the fully qualified domain name for the class. (I assume ".codec.Codec" was
a typo?)

The source code is available at: https://github.com/sigbjornlo/codecs

Sigbjørn

On 12 October 2015 at 17:48, Uwe Schindler <uwe@thetaphi.de> wrote:

> Hi,
>
> Could you check that your JAR file really contains a folder:
> META-INF/services/
>
> With the file:
> org.apache.lucene.codec.Codec
>
> This file should contain your actual Codec's class name.
>
> If this is all setup correctly, NamedSPILoader should see your codec and
> list it as part of the error message.
> Please also make sure, that your codec is initialized using a unique name.
>
> > As to fullfeaturedness: Shouldn't getName() get inherited from Codec
> > (via FilterCodec), private string name getting set through calling
> > super("Hello", new FilterCodec())?
>
> If you pass your own name in your own constructor, e.g.,
>
> super("MyName", ....)
>
> then you have to pass "MyName" to the test runner. Because Codec.getName()
> is final, it is ensured that it is written to the commit
> (SegmentInfos#write(Directory)), so you should find the name in your index.
>
> Please note: In most cases you are more interested in writing your own
> PostingsFormat... Writing your own codec is in most cases not needed... :-)
>
> Uwe
>
> -----
> Uwe Schindler
> H.-H.-Meier-Allee 63, D-28213 Bremen
> http://www.thetaphi.de
> eMail: uwe@thetaphi.de
>
>
> > -----Original Message-----
> > From: Sigbjørn Lund Olsen [mailto:sigbjorn.lund.olsen@gmail.com]
> > Sent: Monday, October 12, 2015 4:55 PM
> > To: java-user@lucene.apache.org
> > Subject: Re: Having some trouble running tests with custom codec
> >
> > Ok, found that core/build.xml was the place to edit. The jar is now in
> the
> > classpath, but I still get the same error:
> >
> > Throwable #1: java.lang.IllegalArgumentException: An SPI class of type
> > org.apache.lucene.codecs.Codec with name 'Hello' does not exist.  You
> need
> > to add the corresponding JAR file supporting this SPI to your classpath.
> > The current classpath supports the following names: [SimpleText,
> Asserting,
> > CheapBastard, FastCompressingStoredFields,
> > FastDecompressionCompressingStoredFields,
> > HighCompressionCompressingStoredFields,
> > DummyCompressingStoredFields, Lucene53]
> >
> > ...which means - I assume - that there is some problem with HelloCodec
> not
> > being loaded or recognized as a Codec.
> >
> > Sincerely,
> > Sigbjørn Lund Olsen
> >
> > On 12 October 2015 at 14:07, Sigbjørn Lund Olsen <
> > sigbjorn.lund.olsen@gmail.com> wrote:
> >
> > > Thank you very much for your help :)
> > >
> > > I am not too familiar with ant build files. I tried adding
> > > <pathelement location="/absolute/path/to/hello-1.0.jar"/> to
> > > junit.classpath, test.classpath, classpath in test-framework/build.xml
> > > and to classpath in lucene/build.xml, but the path does not show up
> > > when running with -verbose option. Could you point me to the right
> place
> > to modify build.xml?
> > >
> > > As to fullfeaturedness: Shouldn't getName() get inherited from Codec
> > > (via FilterCodec), private string name getting set through calling
> > > super("Hello", new FilterCodec())?
> > >
> > > Regards,
> > > Sigbjørn Lund Olsen
> > >
> > > On 12 October 2015 at 09:12, Uwe Schindler <uwe@thetaphi.de> wrote:
> > >
> > >> Hi,
> > >>
> > >> One debugging solution is to use the "-verbose" parameter in the ANT
> > >> command line. Shortly before starting the test runner it prints the
> > >> command line of the test runner. Just check that your JAR file is
> > >> listed as part of java's -classpath / -cp parameter.
> > >>
> > >> Uwe
> > >>
> > >> -----
> > >> Uwe Schindler
> > >> H.-H.-Meier-Allee 63, D-28213 Bremen
> > >> http://www.thetaphi.de
> > >> eMail: uwe@thetaphi.de
> > >>
> > >>
> > >> > -----Original Message-----
> > >> > From: Uwe Schindler [mailto:uwe@thetaphi.de]
> > >> > Sent: Monday, October 12, 2015 8:47 AM
> > >> > To: java-user@lucene.apache.org
> > >> > Subject: RE: Having some trouble running tests with custom codec
> > >> >
> > >> > Hi,
> > >> >
> > >> > adding the codec JAR to the ANT classpath does not help, as it is
> > >> > not
> > >> part of
> > >> > the test classpath:
> > >> >
> > >> > > Running *ant -Dtestcase=TestSegmentTermDocs -
> > >> > Dtests.codec=HelloCodec
> > >> > > -lib
> > >> > > /path/to/hello-1.0.jar* test gives me the following error:
> > >> > >
> > >> >
> > >> > * -lib cannot work as it just adds your JAR to ant's own classpath,
> > >> > not
> > >> the one
> > >> > of the test runner!
> > >> > * -Dtests.codec must be the name your own codec is returning in
> > >> getName().
> > >> > This is not the class name.
> > >> >
> > >> > To make this work, you have to pass your codec to test's classpath.
> > >> This can
> > >> > only be done with hacking Lucene's build.xml files, because the
> > >> > test
> > >> classpath
> > >> > is isolated from anything outside, especially ANT's classpath.
> > >> > In addition, your codec must be a full-featured codec with own
> > >> > name, returned by getName(). This name should not contain the
> > >> > String "Codec", just the plain name (like "Lucene53", "Hello",
> > >> > "FooBar123"). This name
> > >> must
> > >> > be passed to "-Dtests.codec".
> > >> >
> > >> > Uwe
> > >> >
> > >> > -----
> > >> > Uwe Schindler
> > >> > H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de
> > >> > eMail: uwe@thetaphi.de
> > >> >
> > >> >
> > >> > > -----Original Message-----
> > >> > > From: Sigbjørn Lund Olsen [mailto:sigbjorn.lund.olsen@gmail.com]
> > >> > > Sent: Friday, October 09, 2015 4:08 PM
> > >> > > To: java-user@lucene.apache.org
> > >> > > Subject: Having some trouble running tests with custom codec
> > >> > >
> > >> > > As part of my master's thesis I am planning on implementing a
> custom
> > >> > > Lucene codec for compression experiments.
> > >> > >
> > >> > > To get started with my prototype, I've tried following the
> > >> > > instructions here (but using the more recent Lucene 5.3.1):
> > >> > >
> > >> > > http://opensourceconnections.com/blog/2013/06/05/build-your-
> > own-
> > >> > > lucene-codec/
> > >> > >
> > >> > >
> > >> > > However, I am having some trouble getting the codec to load when
> > >> > > running Lucene tests.
> > >> > >
> > >> > > I've uploaded a minimal project at
> > >> > > https://github.com/sigbjornlo/codecs
> > >> > >
> > >> > > Running *mvn package* generates a jar with a file
> > >> > > *META-INF/services/org.apache.lucene.codecs.Codec* containing
a
> > single
> > >> > > line:
> > >> > >
> > >> > > edu.ntnu.sigbjornlo.codecs.HelloCodec
> > >> > >
> > >> > > ...which is where the codec class resides at. It's just a
> FilterCodec
> > >> > > subclass that does nothing extra.
> > >> > >
> > >> > > Running *ant -Dtestcase=TestSegmentTermDocs -
> > >> > Dtests.codec=HelloCodec
> > >> > > -lib
> > >> > > /path/to/hello-1.0.jar* test gives me the following error:
> > >> > >
> > >> > > java.lang.IllegalArgumentException: An SPI class of type
> > >> > > org.apache.lucene.codecs.Codec with name 'HelloCodec' does not
> > exist.
> > >> > > You need to add the corresponding JAR file supporting this SPI
to
> your
> > >> > classpath.
> > >> > > The current classpath supports the following names: [SimpleText,
> > >> > > Asserting, CheapBastard, FastCompressingStoredFields,
> > >> > > FastDecompressionCompressingStoredFields,
> > >> > > HighCompressionCompressingStoredFields,
> > >> > > DummyCompressingStoredFields, Lucene53]
> > >> > >
> > >> > > What might I be doing wrong here? Is there any way to make
> Lucene's
> > >> > > SPI loader log what it's trying and/or failing to load, and why?
> > >> > > (Be adviced: I have also asked this question on StackOverflow:
> > >> > > http://stackoverflow.com/questions/33039973/how-to-configure-
> > lucene-
> > >> > to
> > >> > > -
> > >> > > use-a-custom-codec-from-external-jar
> > >> > > )
> > >> > >
> > >> > > Sincerely,
> > >> > > Sigbjørn Lund Olsen
> > >> >
> > >> >
> > >> >
> ---------------------------------------------------------------------
> > >> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > >> > For additional commands, e-mail: java-user-help@lucene.apache.org
> > >>
> > >>
> > >> ---------------------------------------------------------------------
> > >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > >> For additional commands, e-mail: java-user-help@lucene.apache.org
> > >>
> > >>
> > >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message