lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: segment exists in external directory yet the MergeScheduler executed the merge in a separate thread
Date Fri, 12 Sep 2008 10:40:31 GMT

OK I opened this issue and attached a patch:

     https://issues.apache.org/jira/browse/LUCENE-1384

If possible could you test this patch to see if it resolves your  
exceptions?  Thanks.

Mike

Anthony Urso wrote:

> I have implemented a MapReduce job to merge a bunch of Lucene 2.3.2
> indices together, but the reducers randomly fail with the following
> unchecked exception after thousands of successful merges:
>
> org.apache.lucene.index.MergePolicy$MergeException: segment "_0 exists
> in external directory yet the MergeScheduler executed the merge in a
> separate thread
> 	at  
> org 
> .apache 
> .lucene.index.IndexWriter.copyExternalSegments(IndexWriter.java:2362)
> 	at  
> org 
> .apache 
> .lucene.index.IndexWriter.addIndexesNoOptimize(IndexWriter.java:2307)
>
> Anyone know what would cause such a thing?
>
> Here is the relevant code:
>
>  IndexWriter writer = new IndexWriter(FSDirectory.getDirectory(name),
> new StandardAnalyzer());
>
>  Directory[] dir = new Directory[1];
>
>  for (String p: paths) {
>    dir[0] = FSDirectory.getDirectory(p);
>
>    writer.addIndexesNoOptimize(dir);
>  }
>
>  writer.optimize();
>
>  writer.close();
>
> Cheers,
> Anthony
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message