lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Simon Willnauer <simon.willna...@googlemail.com>
Subject Re: Fwd: Trunk file handle leak?
Date Tue, 14 Sep 2010 09:01:26 GMT
An update on this, the error was on my side doubly incrementing the
searcher reference. no problem on trunk!

simon

On Fri, Sep 10, 2010 at 10:04 PM, Simon Rosenthal
<simon.rosenthal@yahoo.com> wrote:
> Karl:
>
> I reported something very similar a few months back and opened a Jira issue
> - see https://issues.apache.org/jira/browse/SOLR-1911. After I changed to a
> newer nightly build the leak went away. The issue is still open, so you mayw
> ant to update it.
>
> -Simon
>
> ________________________________
>
> ---------- Forwarded message ----------
> From: <karl.wright@nokia.com>
> Date: Fri, Sep 10, 2010 at 3:24 PM
> Subject: RE: Trunk file handle leak?
> To: dev@lucene.apache.org, yonik@lucidimagination.com
>
>
> Hi Yonik,
>
> Be that as it may, I'm seeing a steady increase in file handles used by that
> process over an extended period of time (now 20+ minutes):
>
> root@duck6:~# lsof -p 22379 | wc
>    786    7714  108339
> root@duck6:~# lsof -p 22379 | wc
>    787    7723  108469
> root@duck6:~# lsof -p 22379 | wc
>    787    7723  108469
> root@duck6:~# lsof -p 22379 | wc
>    812    7948  111719
> root@duck6:~# lsof -p 22379 | wc
>    816    7984  112239
> root@duck6:~# lsof -p 22379 | wc
>    817    7993  112369
> root@duck6:~# lsof -p 22379 | wc
>    822    8038  113019
> root@duck6:~# lsof -p 22379 | wc
>    847    8308  116719
> root@duck6:~# lsof -p 22379 | wc
>    852    8353  117369
> root@duck6:~# lsof -p 22379 | wc
>    897    8803  123669
> root@duck6:~# lsof -p 22379 | wc
>   1022   10018  140819
> root@duck6:~#
>
> This doesn't smell like spiky resource usage to me.  It smells like a leak.
> ;-)
>
> Karl
>
> -----Original Message-----
> From: yseeley@gmail.com [mailto:yseeley@gmail.com] On Behalf Of ext Yonik
> Seeley
> Sent: Friday, September 10, 2010 2:08 PM
> To: dev@lucene.apache.org
> Subject: Re: Trunk file handle leak?
>
> On Fri, Sep 10, 2010 at 1:51 PM,  <karl.wright@nokia.com> wrote:
>> (1) There are periodic commits, every 10,000 records.
>> (2) I have no searcher/reader open at the same time, that I am aware of.
>>  This is a straight indexing task.  (You ought to know, you wrote some of
>> the code!)
>
> A commit currently opens a new searcher in Solr.
> It's not too hard to go past 1024 descriptors - either raise the limit
> to 10240 or something, use the compound file format, or lower the
> merge factor.
>
>
>> (3) I *do* see auto warming being called, but it seems not to happen at
>> the same time as a commit, but rather afterwards.
>>
>> Once it starts happening, this happens repeatedly on every commit.
>
> This would also be expected - it's at a point where there are too many
> files for your descriptors.
>
> -Yonik
> http://lucenerevolution.org  Lucene/Solr Conference, Boston Oct 7-8
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message