Return-Path: Delivered-To: apmail-lucene-java-dev-archive@www.apache.org Received: (qmail 70197 invoked from network); 28 Apr 2009 13:27:54 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 28 Apr 2009 13:27:54 -0000 Received: (qmail 53762 invoked by uid 500); 28 Apr 2009 13:27:53 -0000 Delivered-To: apmail-lucene-java-dev-archive@lucene.apache.org Received: (qmail 53702 invoked by uid 500); 28 Apr 2009 13:27:52 -0000 Mailing-List: contact java-dev-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: java-dev@lucene.apache.org Delivered-To: mailing list java-dev@lucene.apache.org Received: (qmail 53694 invoked by uid 99); 28 Apr 2009 13:27:52 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 28 Apr 2009 13:27:52 +0000 X-ASF-Spam-Status: No, hits=2.2 required=10.0 tests=HTML_MESSAGE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of serera@gmail.com designates 209.85.219.179 as permitted sender) Received: from [209.85.219.179] (HELO mail-ew0-f179.google.com) (209.85.219.179) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 28 Apr 2009 13:27:44 +0000 Received: by ewy27 with SMTP id 27so599101ewy.5 for ; Tue, 28 Apr 2009 06:27:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:content-type; bh=DsWxhRm/cfrKK8asKIn9dgXLR20pw1BvguTip1JH2UI=; b=E4V5eDGGVERfuiaP1MYLWt8MCLaUf285zk9dwEyvI4hIXdStqepDXM9KaraFOhWchr yVLpaYnlMlXSqyjiIShGHcuVUu/mhvcxNBsmNCa9d0ADXYGcn7Abs4/MnuEQUKf3N54o 29OsVVj8dTITZ8k6gXnSJpgVBdzykYdti6Jqo= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=V3R9J/X2aDVCLUSbWa6HhkubsZUgeSmRhmckfE/kXXevedKQSMKl/BbLx/u0yQ/X/p tJGXPBQV6I0gH6oqi/3p5U3G++ZCqVo+SacLsgEJy8RCri5JUj0z+KqG5Qq8OBdlNcMS 2pj/4tM6Hz3+PK2HMEiI8i2qbPUB7euWngAqE= MIME-Version: 1.0 Received: by 10.216.36.84 with SMTP id v62mr858213wea.128.1240925244227; Tue, 28 Apr 2009 06:27:24 -0700 (PDT) In-Reply-To: <9ac0c6aa0904280539x63ed2608t4ebca04316b28443@mail.gmail.com> References: <786fde50904280309r40e5c75ld4edd39da4e8e0a8@mail.gmail.com> <9ac0c6aa0904280523l5a1a67a3o32854f9ae7c8920@mail.gmail.com> <786fde50904280528q1a429db2n72b3f44483f46ea1@mail.gmail.com> <9ac0c6aa0904280539x63ed2608t4ebca04316b28443@mail.gmail.com> Date: Tue, 28 Apr 2009 15:27:24 +0200 Message-ID: <786fde50904280627p7797714cge5ffae441209ddb@mail.gmail.com> Subject: Re: ConcurrentMergeScheduler may spawn MergeThreads forever From: Shai Erera To: java-dev@lucene.apache.org Content-Type: multipart/alternative; boundary=001636499f5b5fff0e04689d6cdc X-Virus-Checked: Checked by ClamAV on apache.org --001636499f5b5fff0e04689d6cdc Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit > > It's there so "anyUnhandledExceptions" can be called; > I will check the code again, but I remember that after commenting it, the only compile errors I saw were from MergeThread adding the exception ... perhaps I'm missing something, so I'll re-check the code. I understand your point now - merging is an internal process to IW, therefore there's no real user to notify on errors (e.g., even if IW knew there is an error, what would it do exactly?), and I guess keep trying to execute the merges is reasonable (while throwing the exceptions further - in hope that the user code will catch those and do something with them). BTW, what will happen if I encounter such exceptions, that are thrown repeatedly, and shutdown the JVM? I guess the index will not be in a corrupt state, right? The next time I'll open it, it should be in a state prior to the merge, or at least prior to the merge that failed? I think that sleeping in case of exceptions makes sense .. in case of IO errors that are temporary, this will not spawn threads endlessly, and sleeping will give an opportunity for the IO problem to resolve. In case of bugs, which are supposed to be detected during test time, it should give the developer a chance to kill the process relatively quickly. On Tue, Apr 28, 2009 at 2:39 PM, Michael McCandless < lucene@mikemccandless.com> wrote: > On Tue, Apr 28, 2009 at 8:28 AM, Shai Erera wrote: > > Every merge hit the exception, yes. > > > > And actually, the exceptions list is not used anywhere besides MT adding > the > > exception to the list. That's why I was curious why it's there. > > It's there so "anyUnhandledExceptions" can be called; we could add a > getter so that an app could query the CMS to see if there were > exceptions? But, an app can also subclass CMS & override > handleMergeException to do something. > > > I still think we should protect this case somehow, because even if it > hits a > > disk-full exception, there's no point continuing to run infinitely? > > The disk full could clear up, eg if something external was trying to > copy a massive file onto the same disk, IW could hit disk full, then > the copy would fail and remove the partially copied massive file, and > lots of space becomes available again. > > > So maybe > > before spwaning the next thread, check the exceptions list and if it goes > > over a certain threshold (10?) fail? > > But what does "fail" mean? Stop doing any merges forever, for this IW > instance? That seems dangerous. EG maybe the massive merge will > fail, but little merges can proceed. Also, stopping merging like this > spooks me because you might see a few exceptions, but then they stop > and you think all is good but in fact way too many segments are piling > up. > > We already "fail" now, in that the thread that was doing the merge > will throw an exception up to the JRE's default thread exception > handler. It's just that we then let MergePolicy select that merge > again. > > I like the "sleep on exception" because 1) you will continue to see > that exceptions are being thrown, but 2) it won't saturate your CPU. > > Mike > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org > For additional commands, e-mail: java-dev-help@lucene.apache.org > > --001636499f5b5fff0e04689d6cdc Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
= It's there so "anyUnhandledExceptions" can be called;

I will check the code again, but I remember that after commenting it, t= he only compile errors I saw were from MergeThread adding the exception ...= perhaps I'm missing something, so I'll re-check the code.

I understand your point now - merging is an internal process to IW, therefo= re there's no real user to notify on errors (e.g., even if IW knew ther= e is an error, what would it do exactly?), and I guess keep trying to execu= te the merges is reasonable (while throwing the exceptions further - in hop= e that the user code will catch those and do something with them).

BTW, what will happen if I encounter such exceptions, that are thrown r= epeatedly, and shutdown the JVM? I guess the index will not be in a corrupt= state, right? The next time I'll open it, it should be in a state prio= r to the merge, or at least prior to the merge that failed?

I think that sleeping in case of exceptions makes sense .. in case of I= O errors that are temporary, this will not spawn threads endlessly, and sle= eping will give an opportunity for the IO problem to resolve. In case of bu= gs, which are supposed to be detected during test time, it should give the = developer a chance to kill the process relatively quickly.


On Tue, Apr 28, 2009 at 2:39 PM, Michael= McCandless <lucene@mikemccandless.com> wrote:
On Tue, Apr 28, 2009 at 8:28 AM, Shai Erera <serera@gmail.com> wrote:
> Every merge hit the exception, yes.
>
> And actually, the exceptions list is not used anywhere besides MT addi= ng the
> exception to the list. That's why I was curious why it's there= .

It's there so "anyUnhandledExceptions" can be called; w= e could add a
getter so that an app could query the CMS to see if there were
exceptions? =A0But, an app can also subclass CMS & override
handleMergeException to do something.

> I still think we should protect this case somehow, because even if it = hits a
> disk-full exception, there's no point continuing to run infinitely= ?

The disk full could clear up, eg if something external was trying to<= br> copy a massive file onto the same disk, IW could hit disk full, then
the copy would fail and remove the partially copied massive file, and
lots of space becomes available again.

> So maybe
> before spwaning the next thread, check the exceptions list and if it g= oes
> over a certain threshold (10?) fail?

But what does "fail" mean? =A0Stop doing any merges forever= , for this IW
instance? =A0That seems dangerous. =A0EG maybe the massive merge will
fail, but little merges can proceed. =A0Also, stopping merging like this spooks me because you might see a few exceptions, but then they stop
and you think all is good but in fact way too many segments are piling
up.

We already "fail" now, in that the thread that was doing the merg= e
will throw an exception up to the JRE's default thread exception
handler. =A0It's just that we then let MergePolicy select that merge again.

I like the "sleep on exception" because 1) you will continue to s= ee
that exceptions are being thrown, but 2) it won't saturate your CPU.

Mike

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


--001636499f5b5fff0e04689d6cdc--