lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael McCandless (JIRA)" <>
Subject [jira] [Commented] (LUCENE-5987) Make indexwriter a mere mortal when exceptions strike
Date Fri, 05 Dec 2014 19:38:12 GMT


Michael McCandless commented on LUCENE-5987:

bq. What is IgnoreAlreadyClosedExceptionConcurrentMergeScheduler? 

It's for tests that use CMS and also abort the IW, since CMS can hit ACE in such cases and
it's "ok".

We will likely need to use it in more tests...

> Make indexwriter a mere mortal when exceptions strike
> -----------------------------------------------------
>                 Key: LUCENE-5987
>                 URL:
>             Project: Lucene - Core
>          Issue Type: Task
>            Reporter: Robert Muir
>            Assignee: Michael McCandless
>         Attachments: LUCENE-5987.patch, LUCENE-5987.patch, LUCENE-5987.patch
> IndexWriter's exception handling is overly complicated. Every method in general reads
like this:
> {code}
> try {
>   try {
>     try { 
>      ...
>      // lock order: COMPLICATED
>      synchronized(this or that) {
>      }
>      ...
>    } finally {
>      if (!success5) {
>        deleter.deleteThisFileOrThat();
>      }
>     ...
>   }
> }
> {code}
> Part of the problem is it acts like its an invincible superhero, e.g. can take a disk
full on merge or flush to the face and just keep on trucking, and you can somehow fix the
root cause and then just go about making commits on the same instance.
> But we have a hard enough time ensuring exceptions dont do the wrong thing (e.g. cause
corruption), and I don't think we really test this crazy behavior anywhere: e.g. making commits
AFTER hitting disk full and so on.
> It would probably be simpler if when such things happen, IW just considered them "tragic"
just like OOM and rolled itself back, instead of doing all kinds of really scary stuff to
try to "keep itself healthy" (like the little dance it plays with IFD in mergeMiddle manually
deleting CFS files).
> Besides, without something like a WAL, Indexwriter isn't really fit to be a superhero
anyway: it can't prevent you from losing data in such situations. It just doesn't have the
right tools for the job.
> edit: just to be clear I am referring to abort (low level exception during flush) and
exceptions during merge. For simple non-aborting cases like analyzer errors, of course we
can deal with this. We already made great progress on turning a lot of BS exceptions that
would cause aborts into non-aborting ones recently.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message