lucene-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Schindler" <...@thetaphi.de>
Subject RE: file handle leaks appearing on Index files
Date Thu, 13 Sep 2012 07:54:29 GMT
Hi,

this is not a leak, lsof will report "deleted" files with filehandles still open, when at
the time when the changes were committed by IndexWriter, in parallel another IndexReader is
open, that stays on an older snapshot of the files. In that case, this IndexReader still uses
files, not yet completely deleted by the file system (inode is still there, but directory
entry is already deleted).
You have to use IndexReader.openIfChanged() or open a completely new instance with IndexReader.open()
to get an updated view on the index after committing changes to IndexWriter. Don't forget
to close the old IndexReader! If you don't do this, the older snapshot view is still references,
preventing files from being completely deleted and disappearing from lsof.

Uwe

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de


> -----Original Message-----
> From: Jiajun Chen [mailto:cjjvictory@gmail.com]
> Sent: Thursday, September 13, 2012 6:36 AM
> To: general@lucene.apache.org
> Subject: file handle leaks appearing on Index files
> 
> heapdump show the instance of IndexReader hava 164 at Thu Sep 13 12:04:00
> CST 2012.
> 
> 
> $ lsof  |grep deleted
> 
> reports the following:
> ......
> java      13436 uu 5086r      REG                9,4       7573  79311898
> /var/index/full/20120910/_v68n.fdt (deleted)
> java      13436 uu 5087r      REG                9,4       2340  79311970
> /var/index/full/20120910/_v68n.fdx (deleted)
> java      13436 uu 5088r      REG                9,4       2058  79311887
> /var/index/full/20120910/_v68o.fdt (deleted)
> java      13436 uu 5089r      REG                9,4        636  79311854
> /var/index/full/20120910/_v68o.fdx (deleted)
> java      13436 uu 5090w     REG                9,4       8038  79312040
> /var/index/full/20120910/_v68p.fdt (deleted)
> java      13436 uu 5091r      REG                9,4       2476  79312050
> /var/index/full/20120910/_v68p.fdx (deleted)
> java      13436 uu 5092r      REG                9,4       7015  79312087
> /var/index/full/20120910/_v68q.fdt (deleted)
> java      13436 uu 5093r      REG                9,4       2332  79312091
> /var/index/full/20120910/_v68q.fdx (deleted)
> java      13436 uu 5094r      REG                9,4        648  79312128
> /var/index/full/20120910/_v68r.fdt (deleted) .....
> 
> $ lsof  |grep deleted |wc -l ;date
> 
> reports the following:
> 
> 494
> Wed Sep 12 23:11:40 CST 2012
> 
> 506
> Wed Sep 12 23:22:57 CST 2012
> 
> 560
> Wed Sep 12 23:34:56 CST 2012
> 
> 560
> Wed Sep 12 23:46:29 CST 2012
> 
> 560
> Wed Sep 12 23:49:56 CST 2012
> 
> 566
> Wed Sep 12 23:56:08 CST 2012
> 
> 4275
> Thu Sep 13 12:04:00 CST 2012


Mime
View raw message