directory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alex Karasulu <akaras...@apache.org>
Subject Re: Creating new connections in integration tests
Date Sat, 05 Feb 2011 03:20:29 GMT
On Fri, Feb 4, 2011 at 11:58 PM, Stefan Seelmann <seelmann@apache.org>wrote:

> Hi guys,
>
> one of our Hudson builds currently fails [1], the cause is an
> IOException (Too many open files). The reason this happens is that in
> our integration test (server-integ and ldap-api-test) we open real
> network connections. In many cases a new connection is opened in each
> test method. I also found a case where we open new connections within
> a loop.
>
>
Yes the tests all need to be revamped to make integration tests run smooth.
Some tests are performance tests like this one that was recently ignored:

     http://goo.gl/Cws6O

We also have network tests happening as you can see in the core-integ area
which should not be testing services. I'll take a pass at trying to organize
the tests so we at least localize all the network tests under server-integ.


> Most times the connections is closed after usage. However, as you
> certainly know, there is still a problem with allocating many new
> connections: sockets remain in TIME_WAIT state for some time after
> bein closed, see [2]. On the Hudson servers we can of course increase
> the ulimit -n and reduce the TIME_WAIT value. But I think that won't
> scale when connections are opened in a loop.
>

Agreed.


> I'd like to discuss possible options to improve the situation:
>
> - Fist I think in general we should avoid to create new connections
> within a loop in integration tests. We have other techniques to run
> such test [3]. In general such test patterns are stress or performance
> tests, but not integration tests.
>

You're right, we need to be more disciplined with making sure only
integration tests are present. We pay for these heavy long durations with
builds taking 15+ minutes on laptops.

It's a good thing we're refreshing our build config on hudson. It's showing
us we need to clean up a little.


> - Second we should avoid to create a new connection for each test
> method. One option to achieve this is to use a single connection per
> test class without opening/closing them @Before/@After each test.
> Maybe the better option is to use a connection pool. Of course when
> testing connect/disconnect/bind/unbind it may be required to use a
> fresh connection objects.
>
> Thoughts?
>

+1

If the test permits the reuse then we definitely should be reusing the
connection for each test class.

Thanks,
Alex

Mime
View raw message