lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ian Lea <>
Subject Re: Large index merging/optimization?
Date Wed, 15 Jun 2011 15:44:34 GMT
Waits of several hours on a 4Gb index sounds very unlikely.  Are you
sure there isn't something else going on that is blocking things?
What version of lucene?  Decent, error-free, hardware?

As for optimize, I'd skip it altogether, or schedule it occasionally
when there is no or low activity on the index.


On Wed, Jun 15, 2011 at 1:59 PM, sol myr <> wrote:
> Hi,
> Our Lucene index grew to about 4 GB .
> Unfortunately it brought up a performance problem of slow file merging.
> We have:
> 1. A writer thread: once an Hour it looks for modified documents, and
> updates the Lucene index.
> Usually there are only few modifications, but sometimes we switch the
> entire content and re-index everything.
> 2. The default Lucene Merge thread (ConcurrentMergeScheduler)
> Usually it works great. But every several hours the
> 'ConcurrentMergeScheduler' thread gets stuck (for hours - I'm guessing
> it got to the point where it needs to merge large files).
> During this, our Writer thread is stuck (waiting on a lock), so users
> will see stale data.
> My questions please:
> 1. Is there any configuration that would either speed up file merging,
> or allow IndexWriter to write simultaneously?
> 2. And when do I call 'optimize'?
> Won't it be another very operation, that holds the 'write' lock and
> prevents updates?
> Thanks:)
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message