lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: Lucene 6.1.0 index upgrade
Date Thu, 09 Nov 2017 22:07:31 GMT
Maybe try the addIndexes method that takes CodecReader[] instead?  It will
be somewhat slower, since it fully reads and writes the index, but it
should work across major versions?

Mike McCandless

http://blog.mikemccandless.com

On Thu, Nov 9, 2017 at 3:59 PM, Krishnamurthy, Kannan <
Kannan.Krishnamurthy@cengage.com> wrote:

> Greetings.
>
>
> We are currently using Lucene 6.1.0, our typical index size is ~ 200GB and
> we are planning to move to Lucene 7.0.0.
>
>
> Our daily indexing workflow currently uses IndexWriter.addIndexes() to add
> an index(this will be on Lucene 7.0.0) to a copy of the of the 200GB index.
> Further it calls IndexWriter.forceMerge(1) to create a single segment. The
> addIndexes() now doesn't support different versions of index.
>
> We are facing an issue in upgrading to Lucene 7.0.0 due to this
> limitation. We couldn't afford to do full reindex as there about 50
> different indexes ~ 200GB each.
>
>
> We tried using the LuceneUpgrade tool to upgrade our index to latest
> version, however the IndexCreatedVersionMajor is still LUCENE_6_1_0.major,
> This is preventing us calling IndexWriter.addIndexes() post upgrade.
>
>
> I am wondering what are our options to upgrade without doing a full
> reindex and to continue to use IndexWriter.addIndexes() post upgrade.
>
> The code snippet below shows the different api calls we make.
>
>
>
> @Test
> public void addIndexWith7SegmentToDirectoryOnUpgradedSegment() throws
> IOException {
> IndexUpgrader upgrader = new IndexUpgrader(lucene610Dir); // lucene 6.1.0
> directory 200 GB
> upgrader.upgrade();
>
> IndexWriter w = new IndexWriter(lucene610Dir, newIndexWriterConfig());
> exception.expect(IllegalArgumentException.class);
> exception.expectMessage(
> CoreMatchers.containsString("Cannot use addIndexes(Directory) with
> indexes that have been created by a different Lucene version"));
> w.addIndexes(lucene700Dir); // lucene 7.0.0 daily index
> w.forceMerge(1);
> w.close();
> }
>
> Let me know if you need more details.
>
>
> Many thanks
>
> Kannan.
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message