lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vanlerberghe, Luc" <Luc.Vanlerber...@bvdep.com>
Subject RE: References to deleted file handles in long-running server application
Date Fri, 18 Nov 2005 10:52:31 GMT
Good rule of thumb: don't ever count on the garbage collector cleaning
up for you (even if you call System.gc() to give it a hint).

You should close your IndexSearchers, but with a multithreaded
application  it's difficult to know when (you have to keep them open
until no thread uses it any more)

I created a class DelayCloseIndexSearcher class and posted it in jira
just for this purpose (See
http://issues.apache.org/jira/browse/LUCENE-445)
There's a use case in the javadoc.

I'm using it in my production system (but in a more complex way than in
the example code) and I don't (seem to :) have any problems...

Either way, I would very much appreciate some feedback about the code.

Thanks,

Luc

-----Original Message-----
From: Matt Magoffin [mailto:apache.org@msqr.us] 
Sent: vrijdag 18 november 2005 8:58
To: java-user@lucene.apache.org
Subject: RE: References to deleted file handles in long-running server
application

I'm updating nearly continuously (probably average about every 10
seconds). I don't explicitly close the IndexSearcher objects I create,
as
I share them across threads, but do leave them to be garbage collected.
I
ran into index corruption issues when I explicitly closed them, since I
don't have any code keeping track of how many threads might be using a
given IndexSearcher.

I wonder if it's worth the effort of adding that code in, or can I be
ensured that when the IndexSearcher objects are garbage collected the
file
handles will be released?

-- m@

> How often are you updating your index?  Are you closing your old
> IndexSearchers after switching over to the new index?  You'll need to
> close the searchers in order to release the file handle.  This was the
> same issue I was experiencing:
>
>
http://mail-archives.apache.org/mod_mbox/lucene-java-user/200504.mbox/%3
> c200504281819640.SM01836@chicago%3e
>
> - Monsur
>
>
>
>> -----Original Message-----
>> From: Matt Magoffin [mailto:apache.org@msqr.us]
>> Sent: Thursday, November 17, 2005 11:05 PM
>> To: java-user@lucene.apache.org
>> Subject: Re: References to deleted file handles in
>> long-running server application
>>
>>
>> I've been watching our servers today, and now there are 2500
>> "deleted" file handles open like this. Seems to be quite
>> large. Still don't know why there are so many. I'm using the
>> compound index format already to reduce the number of open files.
>>
>> -- m@
>>
>> > Hello, I use Lucene in a long-running server application on a Linux
>> > server, and the other day I got the "Too many open files"
>> exception.
>> > I've increased the number of allowed file handles, but was checking
>> > out the open file handles using "lsof", and see about 300
>> files listed
>> > like the
>> > following:
>> >
>> > java       1458  jboss  436r   REG        8,2      3945   6258825
>> > /var/lucene-index/_1o6hl.cfs (deleted)
>> > java       1458  jboss  437r   REG        8,2      3467   6258829
>> > /var/lucene-index/_1o6hp.cfs (deleted)
>> > java       1458  jboss  438r   REG        8,2      2743   6258826
>> > /var/lucene-index/_1o6ht.cfs (deleted)
>> > java       1458  jboss  439r   REG        8,2      4069   6258817
>> > /var/lucene-index/_1o6hx.cfs (deleted)
>> > java       1458  jboss  440r   REG        8,2      4098   6258830
>> > /var/lucene-index/_1o6i1.cfs (deleted)
>> > java       1458  jboss  441r   REG        8,2      1817   6258821
>> > /var/lucene-index/_1o6i3.cfs (deleted)
>> >
>> > I haven't been monitoring this long enough to tell if the number
>> > simply continues to grow over time or levels off at some point.
>> >
>> > I was wondering if this behavior to be expected? When will the
>> > application release these deleted file handles (if ever)? Perhaps
>> > during garbage collection? Or might I be handling the index
>> > incorrectly in some way?
>> >
>> > Any thoughts appreciated.
>> > -- m@
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message