Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
MIME-Version: 1.0
In-Reply-To: <BY2PR0201MB0726ACCFF1DA583E7CF18BD699570@BY2PR0201MB0726.namprd02.prod.outlook.com>
References: <BY2PR0201MB0726ACCFF1DA583E7CF18BD699570@BY2PR0201MB0726.namprd02.prod.outlook.com>
From: Michael McCandless <lucene@mikemccandless.com>
Date: Thu, 9 Nov 2017 17:07:31 -0500
Message-ID: <CAL8PwkbY_smxM+bg3PLDiJbpE9eBAOUUqA0jL2_EP4exHEiM5w@mail.gmail.com>
Subject: Re: Lucene 6.1.0 index upgrade
To: Lucene Users <java-user@lucene.apache.org>, Kannan.Krishnamurthy@cengage.com
Content-Type: multipart/alternative; boundary="94eb2c14a21e43121c055d940a72"
archived-at: Thu, 09 Nov 2017 22:08:17 -0000

--94eb2c14a21e43121c055d940a72
Content-Type: text/plain; charset="UTF-8"

Maybe try the addIndexes method that takes CodecReader[] instead?  It will
be somewhat slower, since it fully reads and writes the index, but it
should work across major versions?

Mike McCandless

http://blog.mikemccandless.com

On Thu, Nov 9, 2017 at 3:59 PM, Krishnamurthy, Kannan <
Kannan.Krishnamurthy@cengage.com> wrote:

> Greetings.
>
>
> We are currently using Lucene 6.1.0, our typical index size is ~ 200GB and
> we are planning to move to Lucene 7.0.0.
>
>
> Our daily indexing workflow currently uses IndexWriter.addIndexes() to add
> an index(this will be on Lucene 7.0.0) to a copy of the of the 200GB index.
> Further it calls IndexWriter.forceMerge(1) to create a single segment. The
> addIndexes() now doesn't support different versions of index.
>
> We are facing an issue in upgrading to Lucene 7.0.0 due to this
> limitation. We couldn't afford to do full reindex as there about 50
> different indexes ~ 200GB each.
>
>
> We tried using the LuceneUpgrade tool to upgrade our index to latest
> version, however the IndexCreatedVersionMajor is still LUCENE_6_1_0.major,
> This is preventing us calling IndexWriter.addIndexes() post upgrade.
>
>
> I am wondering what are our options to upgrade without doing a full
> reindex and to continue to use IndexWriter.addIndexes() post upgrade.
>
> The code snippet below shows the different api calls we make.
>
>
>
> @Test
> public void addIndexWith7SegmentToDirectoryOnUpgradedSegment() throws
> IOException {
> IndexUpgrader upgrader = new IndexUpgrader(lucene610Dir); // lucene 6.1.0
> directory 200 GB
> upgrader.upgrade();
>
> IndexWriter w = new IndexWriter(lucene610Dir, newIndexWriterConfig());
> exception.expect(IllegalArgumentException.class);
> exception.expectMessage(
> CoreMatchers.containsString("Cannot use addIndexes(Directory) with
> indexes that have been created by a different Lucene version"));
> w.addIndexes(lucene700Dir); // lucene 7.0.0 daily index
> w.forceMerge(1);
> w.close();
> }
>
> Let me know if you need more details.
>
>
> Many thanks
>
> Kannan.
>
>

--94eb2c14a21e43121c055d940a72--