lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Simon Rosenthal <simon.rosent...@yahoo.com>
Subject Re: Fwd: Trunk file handle leak?
Date Fri, 10 Sep 2010 20:04:24 GMT
Karl:

I reported something very similar a few months back and opened a Jira issue - 
see https://issues.apache.org/jira/browse/SOLR-1911. After I changed to a newer 
nightly build the leak went away. The issue is still open, so you mayw ant to 
update it.

-Simon




________________________________



---------- Forwarded message ----------
From:  <karl.wright@nokia.com>
Date: Fri, Sep 10, 2010 at 3:24 PM
Subject: RE: Trunk file handle leak?
To: dev@lucene.apache.org, yonik@lucidimagination.com


Hi Yonik,

Be that as it may, I'm seeing a steady increase in file handles used by that 
process over an extended period of time (now 20+ minutes):

root@duck6:~# lsof -p 22379 | wc
   786    7714  108339
root@duck6:~# lsof -p 22379 | wc
   787    7723  108469
root@duck6:~# lsof -p 22379 | wc
   787    7723  108469
root@duck6:~# lsof -p 22379 | wc
   812    7948  111719
root@duck6:~# lsof -p 22379 | wc
   816    7984  112239
root@duck6:~# lsof -p 22379 | wc
   817    7993  112369
root@duck6:~# lsof -p 22379 | wc
   822    8038  113019
root@duck6:~# lsof -p 22379 | wc
   847    8308  116719
root@duck6:~# lsof -p 22379 | wc
   852    8353  117369
root@duck6:~# lsof -p 22379 | wc
   897    8803  123669
root@duck6:~# lsof -p 22379 | wc
  1022   10018  140819
root@duck6:~#

This doesn't smell like spiky resource usage to me.  It smells like a leak. ;-)

Karl


-----Original Message-----
From: yseeley@gmail.com [mailto:yseeley@gmail.com] On Behalf Of ext Yonik Seeley
Sent: Friday, September 10, 2010 2:08 PM
To: dev@lucene.apache.org
Subject: Re: Trunk file handle leak?


On Fri, Sep 10, 2010 at 1:51 PM,  <karl.wright@nokia.com> wrote:
> (1) There are periodic commits, every 10,000 records.
> (2) I have no searcher/reader open at the same time, that I am aware of.  This 
>is a straight indexing task.  (You ought to know, you wrote some of the code!)

A commit currently opens a new searcher in Solr.
It's not too hard to go past 1024 descriptors - either raise the limit
to 10240 or something, use the compound file format, or lower the
merge factor.


> (3) I *do* see auto warming being called, but it seems not to happen at the 
>same time as a commit, but rather afterwards.
>
> Once it starts happening, this happens repeatedly on every commit.

This would also be expected - it's at a point where there are too many
files for your descriptors.

-Yonik
http://lucenerevolution.org  Lucene/Solr Conference, Boston Oct 7-8

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Mime
View raw message