lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ganesh" <emailg...@yahoo.co.in>
Subject Re: too many file descriptors opened by Lucene shows (deleted) in /proc
Date Fri, 04 Sep 2009 07:34:47 GMT
I am having only one process using Lucene DB. The same process writes and reads. I do re-open
indexreader. I am maintaing ref count for each reader/searcher and closing it if it is not
used. I am not able to understand, why the file descriptor is showing as (deleted)?

I guessing some issues? Could any one tell me what are the other part of code to look in to
(related to Lucene)? Any other way to identify the root cause?

Regards
Ganesh 


----- Original Message ----- 
From: "Uwe Schindler" <uwe@thetaphi.de>
To: <java-user@lucene.apache.org>
Sent: Friday, September 04, 2009 11:56 AM
Subject: RE: too many file descriptors opened by Lucene shows (deleted) in /proc 


> This is normal. When you open an IndexReader/IndexSearcher, it opens various
> file handles. If you additionally update/add/delete documents in parallel
> (even in other process), or optimize the index, the original IndexReader
> stays on using the "old" state of the index. IndexWriter deletes some files,
> but IndexReader still have them open (this is our segment based
> "transaction" handling of IndexWriter). If you reopen your IndexReader to
> get the last updates of the underlying index, these "deleted" files will
> disappear, as they can be closed. As long as they are open, they disappear
> from directory listing, but they still consume space on disk (because they
> are still used). This is why you see this deleted files.
> 
> -----
> Uwe Schindler
> H.-H.-Meier-Allee 63, D-28213 Bremen
> http://www.thetaphi.de
> eMail: uwe@thetaphi.de
> 
>> From: Ganesh [mailto:emailgane@yahoo.co.in]
>> Sent: Friday, September 04, 2009 8:11 AM
>> To: java-user@lucene.apache.org
>> Subject: too many file descriptors opened by Lucene shows (deleted) in
>> /proc
>> 
>> Hello all,
>> 
>> In my linux pc, there are too many fd counts for lucene database.
>> /proc/<processid>/fd shows very big list. I have provided sample below.
>> 
>> lr-x------    1 root     root           64 Sep  3 17:02 360 ->
>> /opt/ganesh/lucenedb/_2w5.tvf (deleted)
>> lr-x------    1 root     root           64 Sep  3 17:02 361 ->
>> /opt/ganesh/lucenedb/_hr.frq (deleted)
>> lr-x------    1 root     root           64 Sep  3 17:02 362 ->
>> /opt/ganesh/lucenedb/_hr.prx (deleted)
>> lr-x------    1 root     root           64 Sep  3 17:02 363 ->
>> /opt/ganesh/lucenedb/_hr.tvx (deleted)
>> lr-x------    1 root     root           64 Sep  3 21:01 364 ->
>> /opt/ganesh/lucenedb/_hr.tvd (deleted)
>> lr-x------    1 root     root           64 Sep  3 17:02 365 ->
>> /opt/ganesh/lucenedb/_hr.tvf (deleted)
>> lr-x------    1 root     root           64 Sep  3 17:02 366 ->
>> /opt/ganesh/lucenedb/_2w4.fdt (deleted)
>> lr-x------    1 root     root           64 Sep  3 17:02 367 ->
>> /opt/ganesh/lucenedb/_2w4.fdx (deleted)
>> 
>> These files not exist in the disk but why it is showing as (deleted) in
>> the /proc. This means the process is still holding the fd. Please provide
>> me more information on this. I am using Fedora Linux, ext-3 file system,
>> Lucene 2.4.1.
>> 
>> Regards
>> Ganesh
>> Send instant messages to your online friends http://in.messenger.yahoo.com
>> 
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
Send instant messages to your online friends http://in.messenger.yahoo.com 

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message