lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ziming Dong <dzm1016397...@gmail.com>
Subject Re: org.apache.lucene.index.CorruptIndexException: checksum failed
Date Thu, 21 Jul 2016 04:49:38 GMT
I upgrade lucene to 6.1.0, hope it works, thank you very much!

the mount( contain all data directories and index directories ) is shared
by four different computers, but each index directory can be accessed only
by one computer, we have four index directories.

2016-07-21 5:13 GMT+08:00 Michael McCandless <lucene@mikemccandless.com>:

> This means that something is flipping bits from when Lucene wrote this
> file, to when Lucene wanted to merge it.
>
> NFS is dangerous to use with Lucene: it doesn't support
> delete-on-final-close semantics, its client side directory listing caches
> can be stale, locking is iffy, etc.  If you are using a single computer to
> both write and read to/from the index, then it should be OK, though not
> heavily tested, but if you share the mount across different computers
> taking turns doing indexing and searching, there will be problems.
>
> Finally, even if things are working OK over NFS, it will likely be very
> slow performance, compared to a local mount.
>
> All that being said, the known problems with NFS should not lead to this
> corruption, I think.  Something else is wrong.  Maybe run an fsck on the
> rocketsor disks?  Maybe try eliminating variables, e.g. network card/cables
> between your macmini and the rocketstor, swap in different hard drives in
> the rocketstor, etc.  Does the problem still happen with Lucene 6.1.0?
>
> Mike McCandless
>
> http://blog.mikemccandless.com
>
> On Wed, Jul 20, 2016 at 6:57 AM, 董自鸣 <dzm1016397507@gmail.com> wrote:
>
>> I use ubuntu to run lucene(java8, lucene6.0.0, not ES), and nfs to mount
>> disk(actually is rocketstor) of macmini. I get this error constantly. I
>> have already used org.apache.lucene.index.CheckIndex, but will still crash
>> after restart program.
>>
>>
>> Exception in thread "main" org.apache.lucene.store.AlreadyClosedException:
>> this IndexWriter is closed at
>> org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:724) at
>> org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:738) at
>> org.apache.lucene.index.IndexWriter.numDocs(IndexWriter.java:1198) at
>> xxxx.xxxxx.search.XxxxxxxxIndexer.close(XxxxxxxxIndexer.java:184) at
>>
>> xxxx.xxxxx.search.ThreadedXxxxxxxxIndexer.close(ThreadedXxxxxxxxIndexer.java:59)
>> at
>>
>> xxxx.xxxxx.search.ThreadedXxxxxxxxIndexer.main(ThreadedXxxxxxxxIndexer.java:136)
>> Caused by: org.apache.lucene.index.CorruptIndexException: checksum failed
>> (hardware problem?) : expected=51fbdb5c actual=6e964d17
>>
>> (resource=BufferedChecksumIndexInput(MMapIndexInput(path="/mnt/HPT8_56T/xxxxxxxxx-index/index1/_mq.cfs")
>> [slice=_mq_Lucene50_0.pos])) at
>> org.apache.lucene.codecs.CodecUtil.checkFooter(CodecUtil.java:365) at
>> org.apache.lucene.codecs.CodecUtil.checksumEntireFile(CodecUtil.java:469)
>> at
>>
>> org.apache.lucene.codecs.lucene50.Lucene50PostingsReader.checkIntegrity(Lucene50PostingsReader.java:1286)
>> at
>>
>> org.apache.lucene.codecs.blocktree.BlockTreeTermsReader.checkIntegrity(BlockTreeTermsReader.java:336)
>> at
>>
>> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.checkIntegrity(PerFieldPostingsFormat.java:317)
>> at org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java:96)
>> at
>> org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:211)
>> at
>> org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:96) at
>> org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4099) at
>> org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3679) at
>>
>> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
>> at
>>
>> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)
>>
>> --
>> *suiyuan*
>>
>
>


-- 
*http://suiyuan2009.github.io/ <http://suiyuan2009.github.io/>*
*suiyuan2009*

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message