lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Barry Forrest" <>
Subject Optimizing index takes too long
Date Sun, 11 Nov 2007 23:16:28 GMT

Optimizing my index of 1.5 million documents takes days and days.

I have a collection of 10 million documents that I am trying to index
with Lucene.  I've divided the collection into chunks of about 1.5 - 2
million documents each.  Indexing 1.5 documents is fast enough (about
12 hours), but this results in an index directory containing about
35000 files.  Optimizing this index takes several days, which is a bit
too long for my purposes.  Each sub-index is about 150G.

What can I do to make this process faster?

Thanks for your help,

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message