lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: "read past EOF" when merge
Date Thu, 01 Nov 2012 12:30:36 GMT
Well somehow you have a corrupt index.  I'd be very interested in how
that happened :)

You need to run CheckIndex with -fix (it's a command-line tool, too)
to remove that bad segment else some search will eventually hit an
exception on that segment ... and merging can never be done with that
segment.

But then we need to figure out how you got that corruption.  Is your
IndexWriter writing via NFS?  Were there any exceptions while
indexing?  Is the corruption repeatable?

Are you sure there was never more than one IndexWriter open on the same index?

Mike McCandless

http://blog.mikemccandless.com

On Wed, Oct 31, 2012 at 11:45 PM, pliu367 <pliu@ablesky.com> wrote:
> Hi Mike,
> I got the same problem as above. My search project is running on two
> servers(1 master for index writing and reading and 1 slaver for index
> reading), and the master server writes the index into a seperate disk(not
> belong to master) which is shared by the slaver. I have never changed the
> LockFactory on the Directory. I download the problematic index to local and
> check the problematic index by CheckIndex(with doFix=false), I got message
> as below:
> 3 of 22: name=_51w docCount=18
>     compound=true
>     hasProx=true
>     numFiles=2
>     size (MB)=0.007
>     diagnostics = {os.version=2.6.18-194.32.1.el5xen, os=Linux,
> lucene.version=3.4.0 1167142 - mike - 2011-09-09 09:02:09, source=flush,
> os.arch=amd64, java.version=1.6.0_20, java.vendor=Sun Microsystems Inc.}
>     has deletions [delFileName=_51w_5.del]
>     test: open reader.........OK [17 deleted docs]
>     test: fields..............OK [10 fields]
>     test: field norms.........OK [10 fields]
>     test: terms, freq, prox...ERROR [read past EOF]
> java.io.IOException: read past EOF
>         at
> org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:207)
>         at
> org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:39)
>         at org.apache.lucene.store.DataInput.readVInt(DataInput.java:105)
>         at
> org.apache.lucene.store.BufferedIndexInput.readVInt(BufferedIndexInput.java:181)
>         at
> org.apache.lucene.index.SegmentTermPositions.readDeltaPosition(SegmentTermPositions.java:76)
>         at
> org.apache.lucene.index.SegmentTermPositions.nextPosition(SegmentTermPositions.java:72)
>         at org.apache.lucene.index.CheckIndex.testTermIndex(CheckIndex.java:701)
>         at org.apache.lucene.index.CheckIndex.checkIndex(CheckIndex.java:563)
>         at org.apache.lucene.index.CheckIndex.main(CheckIndex.java:995)
>     test: stored fields.......OK [10 total field count; avg 10 fields per
> doc]
>     test: term vectors........OK [0 total vector count; avg 0 term/freq
> vector fields per doc]
> FAILED
>     WARNING: fixIndex() would remove reference to this segment; full
> exception:
> java.lang.RuntimeException: Term Index test failed
>         at org.apache.lucene.index.CheckIndex.checkIndex(CheckIndex.java:576)
>         at org.apache.lucene.index.CheckIndex.main(CheckIndex.java:995)
>
> Could u please tell me what will cause this error(test: terms, freq,
> prox...ERROR [read past EOF]) and how to avoid this problem?
> Thank u very much for ur reply!
>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/read-past-EOF-when-merge-tp4017179p4017444.html
> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message