lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: zero sized cfs files in index lead to IOException: read past EOF
Date Tue, 19 Jun 2012 15:18:57 GMT
This shouldn't normally happen, even on crash, kill -9, power loss, etc.

It can only mean either there is a bug in Lucene, or there's something
wrong with your hardware/IO system, or the fsync operation doesn't
actually work on the IO system.

You can run CheckIndex to see what's broken (then, add -fix to remove
the broken segments)...

What IO system are you using?  Is this reproducible?  Are you sure
there wasn't a problem in transferring the index from its source to
you....?

Mike McCandless

http://blog.mikemccandless.com

On Tue, Jun 19, 2012 at 10:50 AM, Chris Gioran
<chris.gioran@neotechnology.com> wrote:
> Hello everyone,
>
> I am having a problem with a lucene store. When starting an
> IndexWriter on it, it throws the following exception:
>
> Caused by: java.io.IOException: read past EOF:
> MMapIndexInput(path="/path/to/index/_drs.cfs")
>        at org.apache.lucene.store.MMapDirectory$MMapIndexInput.readByte(MMapDirectory.java:279)
>        at org.apache.lucene.store.DataInput.readVInt(DataInput.java:105)
>        at org.apache.lucene.index.CompoundFileReader.<init>(CompoundFileReader.java:69)
>        at org.apache.lucene.index.CompoundFileReader.<init>(CompoundFileReader.java:53)
>        at org.apache.lucene.index.IndexWriter.getFieldInfos(IndexWriter.java:1218)
>        at org.apache.lucene.index.IndexWriter.getCurrentFieldInfos(IndexWriter.java:1238)
>        at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:1171)
>
> The store is weird in that it has 3, 0-sized cfs files, which of
> course is the reason for the exception above. Here is the file
> listing:
>
> -rw-r--r-- 1 chris chris    36 Jun  6 16:42 _drr_1.del
> -rw-r--r-- 1 chris chris 47794 Jun  5 21:15 _drr.fdt
> -rw-r--r-- 1 chris chris  6476 Jun  5 21:15 _drr.fdx
> -rw-r--r-- 1 chris chris    23 Jun  5 21:15 _drr.fnm
> -rw-r--r-- 1 chris chris  2539 Jun  5 21:15 _drr.frq
> -rw-r--r-- 1 chris chris  1622 Jun  5 21:15 _drr.nrm
> -rw-r--r-- 1 chris chris  1618 Jun  5 21:15 _drr.prx
> -rw-r--r-- 1 chris chris   115 Jun  5 21:15 _drr.tii
> -rw-r--r-- 1 chris chris  6360 Jun  5 21:15 _drr.tis
> -rw-r--r-- 1 chris chris     0 Jun  6 16:42 _drs.cfs
> -rw-r--r-- 1 chris chris     0 Jun  6 16:42 _drt.cfs
> -rw-r--r-- 1 chris chris     0 Jun  6 16:42 _drx.cfs
> -rw-r--r-- 1 chris chris   974 Jun  6 16:42 segments_12
> -rw-r--r-- 1 chris chris    20 Jun  6 16:42 segments.gen
>
> The store was delivered to me like this with no steps for
> reproduction, but it has been observed in two separate indexes (not
> the same files of course, but the same symptoms as above).
> It was running on a Linux 2.6.18 amd64 with Lucene version 3.5 on
> HotSpot 1.6.0_22
>
> My question to you is how can a Lucene store reach this state? Is it
> possible to happen because of a crash while merging or something
> similar? How easy will it be to recover the data and how can i reduce
> the probability of this happening in the future?
>
> Thank you very much for you time,
> CG
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message