lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sumanta Bhowmik" <sumanta.bhow...@guavus.com>
Subject RE: Merge Exception in Lucene 2.4
Date Thu, 20 Aug 2009 11:41:33 GMT
We put all the IndexWriters in an array which is defined by

final Directory[] finalDir;	

We also declare an indexer
As 
private volatile static Indexer indexer;

final Indexer finalIndexer = indexer;

Next we call the merge in a new thread :

Thread thread = new Thread(){
			public void run()
			{
				try {
					logger.debug("starts merging w/o
optimization");
	
finalIndexer.getWriter().addIndexesNoOptimize(finalDir);
					logger.debug("ends merging w/o
optimization");
					
				} catch (CorruptIndexException e) {
					logger.notice("",e);
				} catch (IOException e) {
					logger.notice("",e);
				}
				
	}
}

Sumanta




-----Original Message-----
From: Michael McCandless [mailto:lucene@mikemccandless.com] 
Sent: Thursday, August 20, 2009 3:07 PM
To: java-user@lucene.apache.org
Subject: Re: Merge Exception in Lucene 2.4

You should definitely upgrade to the latest JDK 1.6 to get the fix for
the JRE bug in LUCENE-1282, but, I don't think you are hitting that
bug (read past EOF during merge is a different exception).

Can you describe more detail on how you merge 6 IndexWriters?

Mike

On Thu, Aug 20, 2009 at 5:21 AM, Sumanta
Bhowmik<sumanta.bhowmik@guavus.com> wrote:
> I checked at http://issues.apache.org/jira/browse/LUCENE-1282
> SegmentMerger.java has this code
>
> TermFreqVector[] vectors = reader.getTermFreqVectors(docNum);
> termVectorsWriter.addAllDocVectors(vectors);
>
> so this issue appears inspite of this fix.
>
> I am using java version "1.6.0_07". Is it fixed in jdk6u10 and above
> (http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6707044) ?
>
> Regards
> Sumanta
>
>
> -----Original Message-----
> From: Sumanta Bhowmik [mailto:sumanta.bhowmik@guavus.com]
> Sent: Thursday, August 20, 2009 1:15 PM
> To: java-user@lucene.apache.org
> Subject: Merge Exception in Lucene 2.4
>
> Hi
>
>
>
> I am getting this issue in Lucene2.4 when I try to merge multiple
> IndexWriters(generally 6)
>
>
>
> sh-3.2# Exception in thread "Lucene Merge Thread #5"
> org.apache.lucene.index.MergePolicy$MergeException: java.io.IOException:
> read past EOF
>
>        at
>
org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(Concur
> rentMergeScheduler.java:309)
>
>        at
>
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentM
> ergeScheduler.java:286)
>
> Caused by: java.io.IOException: read past EOF
>
>        at
>
org.apache.lucene.store.BufferedIndexInput.readBytes(BufferedIndexInput.java
> :135)
>
>        at
>
org.apache.lucene.index.CompoundFileWriter.copyFile(CompoundFileWriter.java:
> 228)
>
>        at
>
org.apache.lucene.index.CompoundFileWriter.close(CompoundFileWriter.java:184
> )
>
>        at
>
org.apache.lucene.index.SegmentMerger.createCompoundFile(SegmentMerger.java:
> 204)
>
>        at
> org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4260)
>
>        at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3877)
>
>        at
>
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeSche
> duler.java:205)
>
>        at
>
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentM
> ergeScheduler.java:260)
>
>
>
>
>
> Is this a known issue and has any fix been provided for it ? Would
> appreciate any help.
>
>
>
> Regards
>
> Sumanta
>
>
>
>
>
>
>
>
> __________ NOD32 4349 (20090819) Information __________
>
> This message was checked by NOD32 antivirus system.
> http://www.eset.com
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


__________ NOD32 4350 (20090820) Information __________

This message was checked by NOD32 antivirus system.
http://www.eset.com



---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message