lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From <karl.wri...@nokia.com>
Subject RE: Trunk file handle leak?
Date Fri, 10 Sep 2010 19:24:18 GMT
Hi Yonik,

Be that as it may, I'm seeing a steady increase in file handles used by that process over
an extended period of time (now 20+ minutes):

root@duck6:~# lsof -p 22379 | wc
    786    7714  108339
root@duck6:~# lsof -p 22379 | wc
    787    7723  108469
root@duck6:~# lsof -p 22379 | wc
    787    7723  108469
root@duck6:~# lsof -p 22379 | wc
    812    7948  111719
root@duck6:~# lsof -p 22379 | wc
    816    7984  112239
root@duck6:~# lsof -p 22379 | wc
    817    7993  112369
root@duck6:~# lsof -p 22379 | wc
    822    8038  113019
root@duck6:~# lsof -p 22379 | wc
    847    8308  116719
root@duck6:~# lsof -p 22379 | wc
    852    8353  117369
root@duck6:~# lsof -p 22379 | wc
    897    8803  123669
root@duck6:~# lsof -p 22379 | wc
   1022   10018  140819
root@duck6:~#

This doesn't smell like spiky resource usage to me.  It smells like a leak. ;-)

Karl

-----Original Message-----
From: yseeley@gmail.com [mailto:yseeley@gmail.com] On Behalf Of ext Yonik Seeley
Sent: Friday, September 10, 2010 2:08 PM
To: dev@lucene.apache.org
Subject: Re: Trunk file handle leak?

On Fri, Sep 10, 2010 at 1:51 PM,  <karl.wright@nokia.com> wrote:
> (1) There are periodic commits, every 10,000 records.
> (2) I have no searcher/reader open at the same time, that I am aware of.  This is a
straight indexing task.  (You ought to know, you wrote some of the code!)

A commit currently opens a new searcher in Solr.
It's not too hard to go past 1024 descriptors - either raise the limit
to 10240 or something, use the compound file format, or lower the
merge factor.


> (3) I *do* see auto warming being called, but it seems not to happen at the same time
as a commit, but rather afterwards.
>
> Once it starts happening, this happens repeatedly on every commit.

This would also be expected - it's at a point where there are too many
files for your descriptors.

-Yonik
http://lucenerevolution.org  Lucene/Solr Conference, Boston Oct 7-8

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message