lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Libbrecht <p...@hoplahup.net>
Subject Re: gracefully interrupting an optimize
Date Wed, 26 Jan 2011 19:11:47 GMT
Please make sure all of that is in the javadoc.
This is precious info I feel.

paul


Le 26 janv. 2011 à 20:04, Michael McCandless a écrit :

> Yes, this is what's expected -- the exception notifies the thread
> calling optimize that the merge was aborted.
> 
> Mike
> 
> On Wed, Jan 26, 2011 at 9:33 AM,  <v.sevel@lombardodier.com> wrote:
>> Hi Michael,
>> 
>> I suppose that as you suggested, if I do a close(false) during an optimize
>> I am supposed to expect the following exception:
>> 
>> java.io.IOException: background merge hit exception: _3ud72:c33936445
>> _3uqhr:c126349 _3uuf8:c57041 _3v27p:c78599 _3vf2s:c111005 _3vfad:c6574
>> _3vrcj:c130263 _3vrci:c1224 into _3vrck [optimize] [mergeDocStores]
>>        at
>> org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:2859)
>>        at
>> org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:2794)
>>        at
>> org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:2764)
>>        at logserver.LuceneWriter.optimize(LuceneWriter.java:91)
>>        at logserver.SweeperMgr.optimize(SweeperMgr.java:196)
>>        at logserver.SweeperMgr.safelyExecute(SweeperMgr.java:104)
>>        at logserver.SweeperMgr.access$1(SweeperMgr.java:90)
>>        at logserver.SweeperMgr$1.run(SweeperMgr.java:74)
>> Caused by: org.apache.lucene.index.MergePolicy$MergeAbortedException:
>> merge is aborted: _3ud72:c33936445 _3uqhr:c126349 _3uuf8:c57041
>> _3v27p:c78599 _3vf2s:c111005 _3vfad:c6574 _3vrcj:c130263 _3vrci:c1224 into
>> _3vrck [optimize] [mergeDocStores]
>>        at
>> org.apache.lucene.index.MergePolicy$OneMerge.checkAborted(MergePolicy.java:122)
>>        at
>> org.apache.lucene.index.SegmentMerger$CheckAbort.work(SegmentMerger.java:788)
>>        at
>> org.apache.lucene.index.SegmentMerger.mergeTermInfos(SegmentMerger.java:650)
>>        at
>> org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:586)
>>        at
>> org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:154)
>>        at
>> org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:5045)
>>        at
>> org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4630)
>>        at
>> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:235)
>>        at
>> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:291)
>> 
>> right? I can interpret this as "everything is as planned".
>> 
>> thanks
>> 
>> 
>> Vincent Sevel
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> Michael McCandless <lucene@mikemccandless.com>
>> 
>> 
>> 21.01.2011 15:48
>> Please respond to
>> java-user@lucene.apache.org
>> 
>> 
>> 
>> To
>> java-user@lucene.apache.org
>> cc
>> 
>> Subject
>> Re: gracefully interrupting an optimize
>> 
>> 
>> 
>> 
>> 
>> 
>> No.
>> 
>> If you just do IW.close() <-- no boolean specified, then that defaults
>> to IW.close(true) which means "wait for all BG merges to finish".
>> 
>> So "normally" IW.close() reserves the right to take a long time.
>> 
>> But IW.close(false) should finish relatively quickly...
>> 
>> Mike
>> 
>> On Fri, Jan 21, 2011 at 9:20 AM, Paul Libbrecht <paul@hoplahup.net> wrote:
>>> Would that happen "automagically" at finalization?
>>> 
>>> paul
>>> 
>>> 
>>> Le 21 janv. 2011 à 15:13, Michael McCandless a écrit :
>>> 
>>>> If you call optimize(false), that'll return immediately but run the
>>>> optimize "in the background" (assuming you are using the default
>>>> ConcurrentMergeScheduler).
>>>> 
>>>> Later, when it's time to stop optimizing, call IW.close(false), which
>>>> will abort any running merges yet keep any merges that had finished
>>>> (so some work, but not necessarily all work, is lost); it should
>>>> return quickly.
>>>> 
>>>> Mike
>>>> 
>>>> On Fri, Jan 21, 2011 at 8:28 AM,  <v.sevel@lombardodier.com> wrote:
>>>>> Hi,
>>>>> 
>>>>> Each night I optimize an index that contains 35 millions docs. Its
>> takes
>>>>> about 1.5 hours. For maintenance reasons, it may happen that the
>> machine
>>>>> gets rebooted. In that case, server gets a chance to gracefully
>> shutdown,
>>>>> but eventually, the reboot script will kill the processes that did not
>>>>> stop in time. My preference would to stop gracefully the optimize
>> rather
>>>>> than getting the process killed. Is there a way to do that?
>>>>> 
>>>>> Thanks,
>>>>> 
>>>>> Vincent
>>>>> 
>>>>> ************************ DISCLAIMER ************************
>>>>> This message is intended only for use by the person to
>>>>> whom it is addressed. It may contain information that is
>>>>> privileged and confidential. Its content does not
>>>>> constitute a formal commitment by Lombard Odier
>>>>> Darier Hentsch & Cie or any of its branches or affiliates.
>>>>> If you are not the intended recipient of this message,
>>>>> kindly notify the sender immediately and destroy this
>>>>> message. Thank You.
>>>>> *****************************************************************
>>>>> 
>>>> 
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>> 
>>> 
>>> 
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>> 
>>> 
>> 
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>> 
>> 
>> 
>> 
>> ************************ DISCLAIMER ************************
>> This message is intended only for use by the person to
>> whom it is addressed. It may contain information that is
>> privileged and confidential. Its content does not
>> constitute a formal commitment by Lombard Odier
>> Darier Hentsch & Cie or any of its branches or affiliates.
>> If you are not the intended recipient of this message,
>> kindly notify the sender immediately and destroy this
>> message. Thank You.
>> *****************************************************************
>> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message