lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Trejkaz <trej...@trypticon.org>
Subject Re: Upgrading Lucene 4 index to 5 doesn't update it - for just some indices
Date Mon, 06 Jul 2015 10:32:49 GMT
> The problem are not the class names, the problem are the names as
> written to Index. And in a Lucene 4 index there is written e.g., "Lucene47"
> as Codec and Postingsformat. The lookup by name is done by this name.
> IndexReader/Writer does Codec.forName("Lucene47"). For this to work,
> every codec has this name in the codec. On SPI discovery, it will of course
> load both of your codecs (the "Lucene47" one from Lucene 4.10 and the
> one from backwards-codecs.jar in Lucene 5.x). Depending on which one is
> earlier in classpath, it will load ony one of those.

I think it can only load one, because the Codec.forName() Lucene 5
would be calling would only look up:
    META-INF/services/org.apache.lucene5.codecs.Codec

Whereas 4.10.4's codecs are only present in the alternate version:
    META-INF/services/org.apache.lucene4.codecs.Codec

So each service shouldn't even see the files from the other version.
But I got worried now and wrote a test program. After copying in
Lucene 4, renaming, copying in 5 with backward-codecs, renaming (doing
this from home where I don't have a copy of the scratch area I'm
working with)...

        System.out.println("Looking for Lucene 4:");
        org.apache.lucene4.codecs.Codec codec4 =
            org.apache.lucene4.codecs.Codec.forName("Lucene46");
        System.out.println("Ended up using: " + codec4.getClass());

        System.out.println("Looking for Lucene 5:");
        org.apache.lucene5.codecs.Codec codec5 =
            org.apache.lucene5.codecs.Codec.forName("Lucene46");
        System.out.println("Ended up using: " + codec5.getClass());


This prints:

Looking for Lucene 4:
  Found: class org.apache.lucene4.codecs.lucene40.Lucene40PostingsFormat
with name: Lucene40
  Found: class org.apache.lucene4.codecs.lucene41.Lucene41PostingsFormat
with name: Lucene41
  Found: class org.apache.lucene4.codecs.lucene42.Lucene42DocValuesFormat
with name: Lucene42
  Found: class org.apache.lucene4.codecs.lucene45.Lucene45DocValuesFormat
with name: Lucene45
  Found: class org.apache.lucene4.codecs.lucene49.Lucene49DocValuesFormat
with name: Lucene49
  Found: class org.apache.lucene4.codecs.lucene410.Lucene410DocValuesFormat
with name: Lucene410
  Found: class org.apache.lucene4.codecs.lucene40.Lucene40Codec with
name: Lucene40
  Found: class org.apache.lucene4.codecs.lucene3x.Lucene3xCodec with
name: Lucene3x
  Found: class org.apache.lucene4.codecs.lucene41.Lucene41Codec with
name: Lucene41
  Found: class org.apache.lucene4.codecs.lucene42.Lucene42Codec with
name: Lucene42
  Found: class org.apache.lucene4.codecs.lucene45.Lucene45Codec with
name: Lucene45
  Found: class org.apache.lucene4.codecs.lucene46.Lucene46Codec with
name: Lucene46
  Found: class org.apache.lucene4.codecs.lucene49.Lucene49Codec with
name: Lucene49
  Found: class org.apache.lucene4.codecs.lucene410.Lucene410Codec with
name: Lucene410
Ended up using: class org.apache.lucene4.codecs.lucene46.Lucene46Codec
Looking for Lucene 5:
  Found: class org.apache.lucene5.codecs.lucene50.Lucene50PostingsFormat
with name: Lucene50
  Found: class org.apache.lucene5.codecs.lucene40.Lucene40PostingsFormat
with name: Lucene40
  Found: class org.apache.lucene5.codecs.lucene41.Lucene41PostingsFormat
with name: Lucene41
  Found: class org.apache.lucene5.codecs.lucene50.Lucene50DocValuesFormat
with name: Lucene50
  Found: class org.apache.lucene5.codecs.lucene42.Lucene42DocValuesFormat
with name: Lucene42
  Found: class org.apache.lucene5.codecs.lucene45.Lucene45DocValuesFormat
with name: Lucene45
  Found: class org.apache.lucene5.codecs.lucene49.Lucene49DocValuesFormat
with name: Lucene49
  Found: class org.apache.lucene5.codecs.lucene410.Lucene410DocValuesFormat
with name: Lucene410
  Found: class org.apache.lucene5.codecs.lucene50.Lucene50Codec with
name: Lucene50
  Found: class org.apache.lucene5.codecs.lucene40.Lucene40Codec with
name: Lucene40
  Found: class org.apache.lucene5.codecs.lucene41.Lucene41Codec with
name: Lucene41
  Found: class org.apache.lucene5.codecs.lucene42.Lucene42Codec with
name: Lucene42
  Found: class org.apache.lucene5.codecs.lucene45.Lucene45Codec with
name: Lucene45
  Found: class org.apache.lucene5.codecs.lucene46.Lucene46Codec with
name: Lucene46
  Found: class org.apache.lucene5.codecs.lucene49.Lucene49Codec with
name: Lucene49
  Found: class org.apache.lucene5.codecs.lucene410.Lucene410Codec with
name: Lucene410
Ended up using: class org.apache.lucene5.codecs.lucene46.Lucene46Codec

So when the Lucene 5 copy of NamedSPILoader is looking for services,
indeed it doesn't see the Lucene 4 copy of the SPIs, because they have
the wrong service name. Conversely, Lucene 4 doesn't see Lucene 5's.

I guess, though, just to be doubly sure, I could modify each
NamedSPILoader such that they will fail fast if they ever discover
more than one implementation with the same name, since if that did
somehow happen, nothing good would come of it.

That said, the class loader trick is still tempting, because one of
the things I am disliking with the current approach is that if you
start typing something like IndexReader, autocomplete in the IDE will
find 4 copies of it. The class loader approach will certainly fix
that, if I wedge the entire jar inside the main jar. :)

TX

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message