lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Krishnamurthy, Kannan" <Kannan.Krishnamur...@cengage.com>
Subject Lucene 6.1.0 index upgrade
Date Thu, 09 Nov 2017 20:59:48 GMT
Greetings.


We are currently using Lucene 6.1.0, our typical index size is ~ 200GB and we are planning
to move to Lucene 7.0.0.


Our daily indexing workflow currently uses IndexWriter.addIndexes() to add an index(this will
be on Lucene 7.0.0) to a copy of the of the 200GB index. Further it calls IndexWriter.forceMerge(1)
to create a single segment. The addIndexes() now doesn't support different versions of index.

We are facing an issue in upgrading to Lucene 7.0.0 due to this limitation. We couldn't afford
to do full reindex as there about 50 different indexes ~ 200GB each.


We tried using the LuceneUpgrade tool to upgrade our index to latest version, however the
IndexCreatedVersionMajor is still LUCENE_6_1_0.major, This is preventing us calling IndexWriter.addIndexes()
post upgrade.


I am wondering what are our options to upgrade without doing a full reindex and to continue
to use IndexWriter.addIndexes() post upgrade.

The code snippet below shows the different api calls we make.



@Test
public void addIndexWith7SegmentToDirectoryOnUpgradedSegment() throws IOException {
IndexUpgrader upgrader = new IndexUpgrader(lucene610Dir); // lucene 6.1.0 directory 200 GB
upgrader.upgrade();

IndexWriter w = new IndexWriter(lucene610Dir, newIndexWriterConfig());
exception.expect(IllegalArgumentException.class);
exception.expectMessage(
CoreMatchers.containsString("Cannot use addIndexes(Directory) with indexes that have been
created by a different Lucene version"));
w.addIndexes(lucene700Dir); // lucene 7.0.0 daily index
w.forceMerge(1);
w.close();
}

Let me know if you need more details.


Many thanks

Kannan.


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message