lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jian chen <chenjian1...@gmail.com>
Subject Re: read past EOF
Date Sat, 27 Aug 2005 18:57:04 GMT
Hi,

It seems this problem only happens when the index files get really large. 
Could it be because java has trouble handling very large files on windows 
machine (guess there is max file size on windows)?

In Lucene, I think there is a maxDoc kind of parameter that you can use to 
specify, when the index gets really large containing more than that many of 
documents, it will not try to merge the index files into one. Could this be 
used to stop the index files from growing forever?

Cheers,

Jian

On 8/27/05, Ouyang, Hui <hui.ouyang@oracle.com> wrote:
> 
> Hi,
> I had lots of "docs out of order" issues when the index is optimized. I 
> did the changes based on the suggestion in this link
> http://nagoya.apache.org/bugzilla/show_bug.cgi?id=23650
> 
> It seems this issue is solved. But some index have "read past EOF" when I 
> do optimization. The index is over 2G and there are some documents deleted 
> from the index. It is based on Lucene 1.4.3 on Windows.
> Does anyone know how to avoid this issue? Thx.
> 
> Regards,
> hui
> 
> 
> 
> 
> merging segments _1ny5 (38708 docs) _1ot0 (1000 docs) _1t2m (4810 
> docs)java.io.I
> 
> OException: read past EOF
> 
> at
> 
> org.apache.lucene.index.CompoundFileReader$CSInputStream.readInternal
> 
> (CompoundFileReader.java:218)
> 
> at
> 
> org.apache.lucene.store.InputStream.readBytes(InputStream.java:61)
> 
> at
> 
> org.apache.lucene.index.SegmentReader.norms(SegmentReader.java:356)
> 
> at
> 
> org.apache.lucene.index.SegmentReader.norms(SegmentReader.java:323)
> 
> at
> 
> org.apache.lucene.index.SegmentMerger.mergeNorms(SegmentMerger.java:4
> 
> 29)
> 
> at
> 
> org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:94)
> 
> at
> 
> org.apache.lucene.index.IndexWriter.mergeSegments(IndexWriter.java:51
> 
> 0)
> 
> at
> 
> org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:370)
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
> 
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message