lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Geoff Cooney <cooney.ge...@gmail.com>
Subject Re: CorruptIndexException when opening Index during first commit
Date Thu, 16 May 2013 18:59:17 GMT
Thanks for the response, Mike.

If I understand correctly, the problem was incorrectly identifying a large
corrupted index as a non-existant index?  It seems like you'd really want
 an index with first-commit in progress to behave like an index with zero
documents, as opposed to a non-existant index.  That is, if segments_1 is
corrupt, open an empty index.  This seems more analagous to how lucene
treats pre-committed segments_n files after the first.  I don't know how
difficult it would be to implement, in practice.

This isn't a major issue for us either way.  We can work around it by
having our searcher code check for the existence of a segments.gen and just
return zero hits by default.  Since this is a read path only, we don't need
to worry about incorrectly overwriting the index.

Cheers,
Geoff





On Thu, May 16, 2013 at 1:47 PM, Michael McCandless <
lucene@mikemccandless.com> wrote:

> Unfortunately this is expected behavior.
>
> We tried to fix it in LUCENE-2812, but this fix was too dangerous and
> could sometimes erase a good index (if transient IOExcs are happening,
> e.g. due to file descriptor exhaustion) so we reverted back in
> LUCENE-4738, so that indexExists will return true, and trying to open
> an IndexReader will throw CorruptIndexException, when the first-commit
> has started but not yet finished.
>
> Mike McCandless
>
> http://blog.mikemccandless.com
>
>
> On Thu, May 16, 2013 at 10:45 AM, Geoff Cooney <cooney.geoff@gmail.com>
> wrote:
> > Hi,
> >
> > We're occasionally seeing a CorruptIndexException when a searcher is
> opened
> > on a new index.
> >
> > When we see the exception, it looks like what is happening is that the
> > searcher is opening the index after prepareCommit for segments_1 but
> before
> > the commit is completed.  Because there is no prior commit to fall back
> to,
> > lucene throws the CorruptIndexException.  Does this sounds feasible and
> is
> > it a bug?  My expectation is that lucene would treat the index as
> > nonexistent until the first commit is completed.
> >
> > We're running lucene 3.5.
> >
> > Cheers,
> > Geoff
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message