lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael McCandless" <luc...@mikemccandless.com>
Subject Re: out of order
Date Thu, 16 Aug 2007 18:42:44 GMT

Hmmm.  It is interesting, because that specific call (using
IndexWriter to "create" an index) was one of the causes in
LUCENE-140.  But I'm pretty sure we fixed that cause.  As part of
LUCENE-140 we also added further checks to catch re-using of an old
.del file at a lower level, and you're not hitting those.

After you close that IndexWriter, can you list the files in your
directory (that's a RAMDirectory right?)?  Something like this:

    import java.util.Arrays;

    String[] l = directory.list();
    Arrays.sort(l);
    for(int i=0;i<l.length;i++)
      System.out.println("file " + i + " = " + l[i] + " " + directory.fileLength(l[i]) + "
bytes");
    
Mike

"testn" <test1@doramail.com> wrote:
> 
> Does it help you to find out if I create an empty index before start the
> real
> operation?
> 
>             IndexWriter writer = new IndexWriter(directory, new
> SimpleAnalyzer(), true);
>             writer.close();
>             /* add new index afterward */
> 
> This is to clean up the index since springmodules doesn't support the way
> to
> clear up the existing index.
> 
> 
> 
> Michael McCandless-2 wrote:
> > 
> > 
> > OK.  Is it possible to capture this as small test case?
> > 
> > Maybe also call IndexWriter.setInfoStream(System.out) and capture details
> > on
> > what segments are being merged?
> > 
> > Can you shed some light on how the application is using Lucene?  Are you
> > doing
> > deletes as well as adds?  Opening readers against this RAMDirectory? 
> > Closing/
> > opening writers at different times?  Any changes to the default parameters
> > (mergeFactor, maxBufferedDocs, etc.)?
> > 
> > Mike
> > 
> > "testn" <test1@doramail.com> wrote:
> >> 
> >> Here you go....
> >> 
> >>  -> Error during the indexing : docs out of order (0 <= 0 )
> >> org.apache.lucene.index.CorruptIndexException: docs out of order (0 <= 0
> >> )
> >>         at
> >> org.apache.lucene.index.SegmentMerger.appendPostings(SegmentMerger.java:368)
> >>         at
> >> org.apache.lucene.index.SegmentMerger.mergeTermInfo(SegmentMerger.java:325)
> >>         at
> >> org.apache.lucene.index.SegmentMerger.mergeTermInfos(SegmentMerger.java:297)
> >>         at
> >> org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:261)
> >>         at
> >> org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:98)
> >>         at
> >> org.apache.lucene.index.IndexWriter.mergeSegments(IndexWriter.java:1883)
> >>         at
> >> org.apache.lucene.index.IndexWriter.flushRamSegments(IndexWriter.java:1741)
> >>         at
> >> org.apache.lucene.index.IndexWriter.flushRamSegments(IndexWriter.java:1733)
> >>         at
> >> org.apache.lucene.index.IndexWriter.maybeFlushRamSegments(IndexWriter.java:1727)
> >>         at
> >> org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1004)
> >>         at
> >> org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:983)
> >>         at
> >> org.springmodules.lucene.index.factory.SimpleLuceneIndexWriter.addDocument(SimpleLuceneIndexWriter.java:44)
> >> )
> >>         at
> >> org.springmodules.lucene.index.object.database.DefaultDatabaseIndexer.doHandleRequest(DefaultDatabaseIndexer.java:306)
> >>         at
> >> org.springmodules.lucene.index.object.database.DefaultDatabaseIndexer.index(DefaultDatabaseIndexer.java:354)
> >> 
> >> 
> >> Michael McCandless-2 wrote:
> >> > 
> >> > 
> >> > Well then that is particularly spooky!!
> >> > 
> >> > And, hopefully, possible/easy to reproduce.  Thanks.
> >> > 
> >> > Mike
> >> > 
> >> > "testn" <test1@doramail.com> wrote:
> >> >> 
> >> >> I use RAMDirectory and the error often shows the low number. Last time
> >> it
> >> >> happened with message "7<=7". Nest time it happens, I will try to
> >> capture
> >> >> the stacktrace.
> >> >> 
> >> >> 
> >> >> 
> >> >> Michael McCandless-2 wrote:
> >> >> > 
> >> >> > 
> >> >> > "testn" <test1@doramail.com> wrote:
> >> >> >> 
> >> >> >> Using Lucene 2.2.0, I still sporadically got doc out of order
> >> error. I
> >> >> >> indexed all of my stuff in one thread. Do you have any idea
why it
> >> >> >> happens?
> >> >> > 
> >> >> > Hmmmmm, that is not good.  I thought we had finally fixed this
with
> >> >> > LUCENE-140.  Though un-corrected disk errors could in theory lead
to
> >> >> > this too.
> >> >> > 
> >> >> > Are you able to easily reproduce it?  Can you post the full
> >> exception?
> >> >> > 
> >> >> > Mike
> >> >> > 
> >> >> >
> >> ---------------------------------------------------------------------
> >> >> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >> >> > For additional commands, e-mail: java-user-help@lucene.apache.org
> >> >> > 
> >> >> > 
> >> >> > 
> >> >> 
> >> >> -- 
> >> >> View this message in context:
> >> >> http://www.nabble.com/out-of-order-tf4276385.html#a12173705
> >> >> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
> >> >> 
> >> >> 
> >> >> ---------------------------------------------------------------------
> >> >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >> >> For additional commands, e-mail: java-user-help@lucene.apache.org
> >> >> 
> >> > 
> >> > ---------------------------------------------------------------------
> >> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >> > For additional commands, e-mail: java-user-help@lucene.apache.org
> >> > 
> >> > 
> >> > 
> >> 
> >> -- 
> >> View this message in context:
> >> http://www.nabble.com/out-of-order-tf4276385.html#a12184067
> >> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
> >> 
> >> 
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >> For additional commands, e-mail: java-user-help@lucene.apache.org
> >> 
> > 
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-user-help@lucene.apache.org
> > 
> > 
> > 
> 
> -- 
> View this message in context:
> http://www.nabble.com/out-of-order-tf4276385.html#a12186579
> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message