lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael McCandless" <luc...@mikemccandless.com>
Subject Re: restoring a corrupt index?
Date Sat, 10 Nov 2007 22:40:34 GMT

"Yonik Seeley" <yonik@apache.org> wrote:
> On Nov 10, 2007 5:01 PM, Michael McCandless <lucene@mikemccandless.com>
> wrote:
> > "Yonik Seeley" <yonik@apache.org> wrote:
> >
> > > > How can this lead to index corruption?  The "no such file or directory"
on
> > > > loading _cf9.fnm sounds like index corruption?
> > >
> > > I don't think older versions of lucene handled these errors as well.
> > > Perhaps _cf9.fnm failed to be written, but the segments file succeeded.
> > > It could also be Solr's fault for allowing further operations on an
> > > index after one failed?  I'm not sure how that should be handled.
> >
> > Ahh, OK.
> >
> > Yeah it's not clear how Solr should handle this case, though Lucene
> > should be at (get to?) the point where on exception no harm to the
> > index can be done.  Ie, index on disk is left consistent, and, the
> > state of the writer is such that it can't corrupt the index even if
> > it's further used after the exception.
> 
> Right... keep in mind concurrent updates too... it's pretty much
> impossible for the user-level to avoid other calls to addDocument()
> from going ahead since they can be in different threads.

Ahh, true.

> > OK, though, writer doesn't use many descriptors at all, right?
> 
> Yeah, but other things might be usng up the descriptors... there is
> probably an open searcher for searching, possibly another one warming
> in the background,   plus whatever else is going on in the servlet
> container.

OK.

> Ryan, are you using stock solr, or embedded?  Anything that might
> prevent old searchers from being closed (you chould check the log
> files for #opens vs #closes)
> 
> >  It
> > opens 1 segment's worth to flush, and mergeFactor+1 segment's worth to
> > merge.  It's spooky if Lucene can create zillions of un-referenced
> > files.
> 
> Perhaps unreferenced files may not be deleted if an exception is
> encountered first, or perhaps even the deleter is failing due to lack
> of descriptors.

OK.  Still spooky that this failure mode is possible on 2.2.

> > Would be good to get to the root cause & make sure it's really
> > fixed on trunk.
> 
> Is there a tool we could have ryan point at the segments file and get
> a dump of the referenced segments?

LUCENE-1020 will print the details of all segments ... hmmm, I see
that I'm not closing each SegmentReader in that tool.  OK I just committed
a fix.  Otherwise I think Ryan would hit file descriptor limit (if these
files are referenced).

Ryan are you able to update to that commit I just did?  If so I think
you should run the tool without -fix and post back what it printed.  It
should report an error on that one segment due to the missing file.
Then, run -fix to remove that segment (please backup your index first!).
Then, if you have a zillion segments in the index, try optimizing it?

Oh, hang on: you can't run -fix because this tool will write a
segments_N file in the current trunk format, which is different
from 2.2.x.  It would not be hard to port the tool back to 2.2:
just comment out those System.out.println's that hit compilation
errors.

> > Or, maybe they are all referenced -- I think there were issues with
> > the old merge policy that could cause segments to not be merged when
> > they should have been?
> 
> That should not have been the case in Lucene 2.2 I think.

Ahh, right.

Mike

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message