lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yonik Seeley" <yo...@apache.org>
Subject Re: restoring a corrupt index?
Date Sat, 10 Nov 2007 21:45:53 GMT
On Nov 10, 2007 4:34 PM, Michael McCandless <lucene@mikemccandless.com> wrote:
>
> "Yonik Seeley" <yonik@apache.org> wrote:
> > On Nov 10, 2007 4:01 PM, Ryan McKinley <ryantxu@gmail.com> wrote:
> > > Using solr, we have been running an indexing process for a while and
> > > when I checked on it today, it spits out an error:
> > >
> > > java.lang.RuntimeException: java.io.FileNotFoundException:
> > > /path/to/index/_cf9.fnm (No such file or directory)
> > >         at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:584)
> > >         at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:475)
> > >
> > > Looking through the archives, it looks like we are up a creek.
> > >
> > > Any thoughts on what could have caused this?  The log files contains
> > > some 'too many open files' errors, I can't tell if that corresponds with
> > > when the index went bad though.
> >
> > Yup... that would most likely be it.
>
> How can this lead to index corruption?  The "no such file or directory" on
> loading _cf9.fnm sounds like index corruption?

I don't think older versions of lucene handled these errors as well.
Perhaps _cf9.fnm failed to be written, but the segments file succeeded.
It could also be Solr's fault for allowing further operations on an
index after one failed?  I'm not sure how that should be handled.

> Or maybe the index is not corrupt but then we are hitting the descriptor limit
> on opening a searcher and it's being reported as "no such file or directory"?

Hmmm, yes that's possible too...
Should be easy to tell by checking if the file _cf9.fnm already exists.

> If so, maybe opening an IndexWriter and optimizing the index would recover
> it?  (Make a backup copy of the index first!).
>
> > > 2. Does the ulimit number explain how the index got corrupted?  If so,
> > > it seems like a problem.
> >
> > I think newest lucene versions would prevent this with
> > lucene_autocommit=false.  A new segments file (the file which
> > references all other files in the current index) is not written until
> > a close of the writer.
>
> Yonik do you understand why so many unreferenced files are being produced
> here?  What's the root cause?

Just guesses... but perhaps new index files are written but things
fail before the stage where old files are deleted?

Ryan, what version of lucene is this version of Solr using?

-Yonik

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message