lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Muir <rcm...@gmail.com>
Subject Re: Open files exceeded, freebsd.
Date Mon, 16 Apr 2012 21:11:06 GMT
On Mon, Apr 16, 2012 at 4:25 PM, Dawid Weiss
<dawid.weiss@cs.put.poznan.pl> wrote:
> We've established with @infra that this is caused by another jail
> eating up 8k descriptors. They're told to limit resource usage and
> after that jenkins should go back to normal.
>

Thanks for the help Dawid... sure was bad luck to happen just as you
committed the new test stuff.

I spent some time (after fixing a few problems last night) debugging
maximum file usage while running tests.
In general we have really no serious leaks, there are some
onesy-twoesy that we already know about,
like not being able to close the SpellChecker searcher in the solr
IndexBasedSpellChecker test, but leaks are really no issue.

Still, lucene-core tests on average (with 4 jvms) use about 300-400
open files at any point, and a few tests bring this up temporarily
to even close to 1000. So its still good to keep an eye on, I think
these indexing tests in lucene are really the only ones that might do
this,
especially if they get crazy configurations.

So long term i still think we should implement something along the
lines of https://issues.apache.org/jira/browse/LUCENE-3195, at least
so if a test does something truly absurd here as far as files we know
about it and can adjust the parameters appropriately: and can fix the
worst tests so they choose parameters that avoid this
[_TestUtil.reduceOpenFiles(IndexWriter)]. Otherwise, infra will be
banging on our door next time :)

-- 
lucidimagination.com

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message