lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Gioran <chris.gio...@neotechnology.com>
Subject Re: zero sized cfs files in index lead to IOException: read past EOF
Date Tue, 19 Jun 2012 20:54:39 GMT
On Tue, Jun 19, 2012 at 6:18 PM, Michael McCandless
<lucene@mikemccandless.com> wrote:
> This shouldn't normally happen, even on crash, kill -9, power loss, etc.
>
> It can only mean either there is a bug in Lucene, or there's something
> wrong with your hardware/IO system, or the fsync operation doesn't
> actually work on the IO system.

I have seen two instances of this, both on the same machine. It isn't
far fetched to assume a hardware issue but no other problems have been
reported there.

> You can run CheckIndex to see what's broken (then, add -fix to remove
> the broken segments)...

Thank you for this information, will get back if something interesting comes up.

> What IO system are you using?  Is this reproducible?  Are you sure
> there wasn't a problem in transferring the index from its source to
> you....?

Commodity SATA disks on RHEL 5.
No, just happened twice, no clear pattern.
Yes, the the exception happened on site and afterwords the store was
given to me - everything in there works but that index.

Thank you for your response, i'll get back if i have more information.

CG

> Mike McCandless
>
> http://blog.mikemccandless.com
>
> On Tue, Jun 19, 2012 at 10:50 AM, Chris Gioran
> <chris.gioran@neotechnology.com> wrote:
>> Hello everyone,
>>
>> I am having a problem with a lucene store. When starting an
>> IndexWriter on it, it throws the following exception:
>>
>> Caused by: java.io.IOException: read past EOF:
>> MMapIndexInput(path="/path/to/index/_drs.cfs")
>>        at org.apache.lucene.store.MMapDirectory$MMapIndexInput.readByte(MMapDirectory.java:279)
>>        at org.apache.lucene.store.DataInput.readVInt(DataInput.java:105)
>>        at org.apache.lucene.index.CompoundFileReader.<init>(CompoundFileReader.java:69)
>>        at org.apache.lucene.index.CompoundFileReader.<init>(CompoundFileReader.java:53)
>>        at org.apache.lucene.index.IndexWriter.getFieldInfos(IndexWriter.java:1218)
>>        at org.apache.lucene.index.IndexWriter.getCurrentFieldInfos(IndexWriter.java:1238)
>>        at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:1171)
>>
>> The store is weird in that it has 3, 0-sized cfs files, which of
>> course is the reason for the exception above. Here is the file
>> listing:
>>
>> -rw-r--r-- 1 chris chris    36 Jun  6 16:42 _drr_1.del
>> -rw-r--r-- 1 chris chris 47794 Jun  5 21:15 _drr.fdt
>> -rw-r--r-- 1 chris chris  6476 Jun  5 21:15 _drr.fdx
>> -rw-r--r-- 1 chris chris    23 Jun  5 21:15 _drr.fnm
>> -rw-r--r-- 1 chris chris  2539 Jun  5 21:15 _drr.frq
>> -rw-r--r-- 1 chris chris  1622 Jun  5 21:15 _drr.nrm
>> -rw-r--r-- 1 chris chris  1618 Jun  5 21:15 _drr.prx
>> -rw-r--r-- 1 chris chris   115 Jun  5 21:15 _drr.tii
>> -rw-r--r-- 1 chris chris  6360 Jun  5 21:15 _drr.tis
>> -rw-r--r-- 1 chris chris     0 Jun  6 16:42 _drs.cfs
>> -rw-r--r-- 1 chris chris     0 Jun  6 16:42 _drt.cfs
>> -rw-r--r-- 1 chris chris     0 Jun  6 16:42 _drx.cfs
>> -rw-r--r-- 1 chris chris   974 Jun  6 16:42 segments_12
>> -rw-r--r-- 1 chris chris    20 Jun  6 16:42 segments.gen
>>
>> The store was delivered to me like this with no steps for
>> reproduction, but it has been observed in two separate indexes (not
>> the same files of course, but the same symptoms as above).
>> It was running on a Linux 2.6.18 amd64 with Lucene version 3.5 on
>> HotSpot 1.6.0_22
>>
>> My question to you is how can a Lucene store reach this state? Is it
>> possible to happen because of a crash while merging or something
>> similar? How easy will it be to recover the data and how can i reduce
>> the probability of this happening in the future?
>>
>> Thank you very much for you time,
>> CG
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message