lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 郑文兴 <zhen...@csdn.net>
Subject RE: Index partition corrupted during a regular flush due to FileNotFoundException on DEL file
Date Tue, 13 Sep 2016 01:07:11 GMT
Thanks to Erick. I will check the disk space first.

 

So you mean if there is no more than 10G free space, Lucene/Solr will delete some files to
save the disk space? Or it will cause the misbehave of Lucene/Solr?

 

Please note that we have several shards/partitions under the same root directory, so which
way the following is true to us. Let's assume we have 2 partitions, and A -> 10G, B->10G

l  do we have to make sure there are at least 20G disk space available?

l  Or we just need to make sure there are at least 10G disk space available?

 

Best,

Wenxing

 

-----Original Message-----
From: Erick Erickson [mailto:erickerickson@gmail.com] 
Sent: Monday, September 12, 2016 10:59 PM
To: dev@lucene.apache.org
Subject: Re: Index partition corrupted during a regular flush due to FileNotFoundException
on DEL file

 

The del file should be present for each segment assuming it has any documents that have been
updated or deleted.

 

Of course if some process external to Solr removed it, you'd get this error.

 

A less common reason is that your disk is full. Solr/Lucene require that you have at least
as much free space on your disk as the index occupies. Thus if you have 10G total disk space
used up by your index, you must have at least 10G free space, is it possible that you're running
without enough disk space?

 

If anything like that is the case you should see errors in your Solr logs, assuming they haven't
been rolled over. Is there anything suspicious there? Look for ERROR (all caps) and/or "Caused
by" as a start.

 

Best,

Erick

 

On Mon, Sep 12, 2016 at 3:31 AM, 郑文兴 < <mailto:zhengwx@csdn.net> zhengwx@csdn.net>
wrote:

> Dear all,

> 

> 

> 

> Today we found one of our index partitions was corrupted during the 

> regular flush, due to the FileNotFoundException on a del file. The 

> followings were the call stacks from the corresponding exception:

> 

> 

> 

> [2016-09-12 16:40:01,801][ERROR][qtp2107666786-40854][indexEngine ] 

> index [so_blog] commit ERROR:_oxep_7fa.del

> org.apache.lucene.store.FSDirectory.fileLength(FSDirectory.java:284)

> org.apache.lucene.index.SegmentInfo.sizeInBytes(SegmentInfo.java:303)

> org.apache.lucene.index.TieredMergePolicy.size(TieredMergePolicy.java:

> 635)

> org.apache.lucene.index.TieredMergePolicy.useCompoundFile(TieredMergeP

> olicy.java:611)

> org.apache.lucene.index.DocumentsWriter.flush(DocumentsWriter.java:593

> )

> org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java:3587)

> org.apache.lucene.index.IndexWriter.prepareCommit(IndexWriter.java:337

> 6)

> org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:34

> 85)

> org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:3467)

> org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:3451)

> org.apache.lucene.index.IndexEngine.flush(IndexEngine.java:409)

> 

> 

> 

> My questions are:

> 

> l  Does anyone know the situation here? From the file system, I can’t 

> find the _oxep_7fa.del.

> 

> l  How about the life cycle of the del file?

> 

> 

> 

> Note: The Lucene Core is on 3.6.2.

> 

> 

> 

> Appreciated for your kindly advice.

> 

> Best Regards, Wenxing

 

---------------------------------------------------------------------

To unsubscribe, e-mail:  <mailto:dev-unsubscribe@lucene.apache.org> dev-unsubscribe@lucene.apache.org
For additional commands, e-mail:  <mailto:dev-help@lucene.apache.org> dev-help@lucene.apache.org


Mime
View raw message