lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matt Magoffin" <apache....@msqr.us>
Subject RE: References to deleted file handles in long-running server application
Date Fri, 18 Nov 2005 07:57:58 GMT
I'm updating nearly continuously (probably average about every 10
seconds). I don't explicitly close the IndexSearcher objects I create, as
I share them across threads, but do leave them to be garbage collected. I
ran into index corruption issues when I explicitly closed them, since I
don't have any code keeping track of how many threads might be using a
given IndexSearcher.

I wonder if it's worth the effort of adding that code in, or can I be
ensured that when the IndexSearcher objects are garbage collected the file
handles will be released?

-- m@

> How often are you updating your index?  Are you closing your old
> IndexSearchers after switching over to the new index?  You'll need to
> close the searchers in order to release the file handle.  This was the
> same issue I was experiencing:
>
> http://mail-archives.apache.org/mod_mbox/lucene-java-user/200504.mbox/%3
> c200504281819640.SM01836@chicago%3e
>
> - Monsur
>
>
>
>> -----Original Message-----
>> From: Matt Magoffin [mailto:apache.org@msqr.us]
>> Sent: Thursday, November 17, 2005 11:05 PM
>> To: java-user@lucene.apache.org
>> Subject: Re: References to deleted file handles in
>> long-running server application
>>
>>
>> I've been watching our servers today, and now there are 2500
>> "deleted" file handles open like this. Seems to be quite
>> large. Still don't know why there are so many. I'm using the
>> compound index format already to reduce the number of open files.
>>
>> -- m@
>>
>> > Hello, I use Lucene in a long-running server application on a Linux
>> > server, and the other day I got the "Too many open files"
>> exception.
>> > I've increased the number of allowed file handles, but was checking
>> > out the open file handles using "lsof", and see about 300
>> files listed
>> > like the
>> > following:
>> >
>> > java       1458  jboss  436r   REG        8,2      3945   6258825
>> > /var/lucene-index/_1o6hl.cfs (deleted)
>> > java       1458  jboss  437r   REG        8,2      3467   6258829
>> > /var/lucene-index/_1o6hp.cfs (deleted)
>> > java       1458  jboss  438r   REG        8,2      2743   6258826
>> > /var/lucene-index/_1o6ht.cfs (deleted)
>> > java       1458  jboss  439r   REG        8,2      4069   6258817
>> > /var/lucene-index/_1o6hx.cfs (deleted)
>> > java       1458  jboss  440r   REG        8,2      4098   6258830
>> > /var/lucene-index/_1o6i1.cfs (deleted)
>> > java       1458  jboss  441r   REG        8,2      1817   6258821
>> > /var/lucene-index/_1o6i3.cfs (deleted)
>> >
>> > I haven't been monitoring this long enough to tell if the number
>> > simply continues to grow over time or levels off at some point.
>> >
>> > I was wondering if this behavior to be expected? When will the
>> > application release these deleted file handles (if ever)? Perhaps
>> > during garbage collection? Or might I be handling the index
>> > incorrectly in some way?
>> >
>> > Any thoughts appreciated.
>> > -- m@
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message