lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jason Rutherglen <>
Subject Re: is there a way to control when merges happen?
Date Fri, 15 May 2009 20:48:54 GMT
Hi Dan,

You are looking to throttle the merging?  I'd recommend setting
ConcurrentMergeScheduler.setMaxThreadCount(1).  This way IW.addDocument
doesn't wait while a merge occurs (like SerialMergeScheduler) however it
should not use as much CPU as only one merge will occur at a time.

In regards to overriding the MS.merge method either way you mentioned would


On Fri, May 15, 2009 at 1:41 PM, Dan OConnor <>wrote:

> All:
> I would like to be able to control when an index merge happens (by wall
> clock time) so that merges do not occur in the middle of the business day.
> I have a lucene system based on v2.3.2 and we add a couple hundred thousand
> documents per day - and we allow searching while documents are being added -
> we reopen an IndexReader periodically to expose newly arrived contents.
> There are times when merging causes significant performance impacts on
> search results - I've seen cases where merging will cause 200% load on a
> system (dual quad core x86_64 running Centos) with a raid-5 disk subsystem
> of 15k drives.
> I've seen some info on the MergeScheduler and but not necessarily enough to
> attempt a coding effort.
> Looking through the code for, is it as
> straightforward as over-riding the mergeScheduler.merge() method with a
> method that checks to see if a merge is allowed (by wall clock time)?  If a
> merge is not allowed at that time, can I just return();? Or do I have to
> sleep the thread until the merge is allowed?
> Thanks,
> Dan
> Dan O'Connor
> SVP, Engineering
> Acquire Media<>
> 77 South Bedford Street, Suite 350<
> >
> Burlington, MA 01803<
> >
> e:<>
> o: 781-250-0565
> f: 877-861-7724

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message