lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: org.apache.lucene.index.CorruptIndexException: checksum failed
Date Wed, 20 Jul 2016 21:13:00 GMT
This means that something is flipping bits from when Lucene wrote this
file, to when Lucene wanted to merge it.

NFS is dangerous to use with Lucene: it doesn't support
delete-on-final-close semantics, its client side directory listing caches
can be stale, locking is iffy, etc.  If you are using a single computer to
both write and read to/from the index, then it should be OK, though not
heavily tested, but if you share the mount across different computers
taking turns doing indexing and searching, there will be problems.

Finally, even if things are working OK over NFS, it will likely be very
slow performance, compared to a local mount.

All that being said, the known problems with NFS should not lead to this
corruption, I think.  Something else is wrong.  Maybe run an fsck on the
rocketsor disks?  Maybe try eliminating variables, e.g. network card/cables
between your macmini and the rocketstor, swap in different hard drives in
the rocketstor, etc.  Does the problem still happen with Lucene 6.1.0?

Mike McCandless

http://blog.mikemccandless.com

On Wed, Jul 20, 2016 at 6:57 AM, 董自鸣 <dzm1016397507@gmail.com> wrote:

> I use ubuntu to run lucene(java8, lucene6.0.0, not ES), and nfs to mount
> disk(actually is rocketstor) of macmini. I get this error constantly. I
> have already used org.apache.lucene.index.CheckIndex, but will still crash
> after restart program.
>
>
> Exception in thread "main" org.apache.lucene.store.AlreadyClosedException:
> this IndexWriter is closed at
> org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:724) at
> org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:738) at
> org.apache.lucene.index.IndexWriter.numDocs(IndexWriter.java:1198) at
> xxxx.xxxxx.search.XxxxxxxxIndexer.close(XxxxxxxxIndexer.java:184) at
>
> xxxx.xxxxx.search.ThreadedXxxxxxxxIndexer.close(ThreadedXxxxxxxxIndexer.java:59)
> at
>
> xxxx.xxxxx.search.ThreadedXxxxxxxxIndexer.main(ThreadedXxxxxxxxIndexer.java:136)
> Caused by: org.apache.lucene.index.CorruptIndexException: checksum failed
> (hardware problem?) : expected=51fbdb5c actual=6e964d17
>
> (resource=BufferedChecksumIndexInput(MMapIndexInput(path="/mnt/HPT8_56T/xxxxxxxxx-index/index1/_mq.cfs")
> [slice=_mq_Lucene50_0.pos])) at
> org.apache.lucene.codecs.CodecUtil.checkFooter(CodecUtil.java:365) at
> org.apache.lucene.codecs.CodecUtil.checksumEntireFile(CodecUtil.java:469)
> at
>
> org.apache.lucene.codecs.lucene50.Lucene50PostingsReader.checkIntegrity(Lucene50PostingsReader.java:1286)
> at
>
> org.apache.lucene.codecs.blocktree.BlockTreeTermsReader.checkIntegrity(BlockTreeTermsReader.java:336)
> at
>
> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.checkIntegrity(PerFieldPostingsFormat.java:317)
> at org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java:96) at
> org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:211) at
> org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:96) at
> org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4099) at
> org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3679) at
>
> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
> at
>
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)
>
> --
> *suiyuan*
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message