directory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Emmanuel Lécharny <elecha...@gmail.com>
Subject Re: Release troubles and failing tests
Date Tue, 08 May 2012 20:13:33 GMT
Le 5/8/12 9:44 PM, Selcuk AYA a écrit :
> On Tue, May 8, 2012 at 12:34 PM, Emmanuel Lécharny<elecharny@gmail.com>  wrote:
>> Le 5/8/12 4:40 PM, Emmanuel Lécharny a écrit :
>>
>>> Le 5/8/12 8:05 AM, Selcuk AYA a écrit :
>>>> Hi,
>>>>
>>>> On Sun, May 6, 2012 at 7:26 AM, Emmanuel Lécharny<elecharny@gmail.com>
>>>>   wrote:
>>>>> Le 5/6/12 12:07 PM, Emmanuel Lécharny a écrit :
>>>>>
>>>>>> Ok, I think I have found the cursor that was not closed.
>>>>> Still having the same failure :/ All the cursors are correctly closed.
>>>>> If I
>>>>> disable the logs (that I use to check the pairing between open and
>>>>> close), I
>>>>> get a failure, and if I activate the log, then I get no failure.
>>>>>
>>>>> I will try to run all the server-integ tests with NOT_THEADSAFE to see
>>>>> if it
>>>>> works better.
>>>> I was trying to build the trunk and see if I can chase the issue we
>>>> see with the caches.
>>> FYI, I'm trying to narrow the pb to the tests that make the server to
>>> freeze, by removing tests from the server-integ suite. It will take a bt of
>>> time, but as soon as I will have reduced the set of test to the minimum,
>>> I'll let you know.
>>>
>> Ok, I was able to reproduce the pb with just two tests in the suite :
>>
>> package org.apache.directory.server.suites;
>>
>>
>> import org.apache.directory.server.annotations.CreateLdapServer;
>> import org.apache.directory.server.annotations.CreateTransport;
>> import org.apache.directory.server.core.annotations.ContextEntry;
>> import org.apache.directory.server.core.annotations.CreateDS;
>> import org.apache.directory.server.core.annotations.CreateIndex;
>> import org.apache.directory.server.core.annotations.CreatePartition;
>> import org.apache.directory.server.core.integ.FrameworkSuite;
>> import org.apache.directory.server.operations.search.PagedSearchIT;
>> import org.apache.directory.server.operations.search.SearchIT;
>> import org.junit.runner.RunWith;
>> import org.junit.runners.Suite;
>>
>>
>> /**
>>   * Stock (default configuration) server integration test suite.
>>   *
>>   * @author<a href="mailto:dev@directory.apache.org">Apache Directory
>> Project</a>
>>   */
>> @RunWith(FrameworkSuite.class)
>> @Suite.SuiteClasses(
>>     {
>>         // operations.search
>>         PagedSearchIT.class,
>>         SearchIT.class,
>> })
>> @CreateDS(
>> name = "SuiteDS",
>> partitions =
>>     {
>>         @CreatePartition(
>>             name = "example",
>>             suffix = "dc=example,dc=com",
>>             contextEntry = @ContextEntry(
>>                 entryLdif =
>>                 "dn: dc=example,dc=com\n" +
>>                     "dc: example\n" +
>>                     "objectClass: top\n" +
>>                     "objectClass: domain\n\n"),
>>             indexes =
>>                 {
>>                     @CreateIndex(attribute = "objectClass"),
>>                     @CreateIndex(attribute = "dc"),
>>                     @CreateIndex(attribute = "ou")
>>             })
>> })
>> @CreateLdapServer(
>> transports =
>>     {
>>         @CreateTransport(protocol = "LDAP"),
>>         @CreateTransport(protocol = "LDAPS")
>> })
>> public class StockServerISuite
>> {
>> }
>>
>> It seems like the PersistentSearch is keeping a cursor in the server until
>> the search is abandonned or the user unbinds. That might well be an issue.
>>
>> Now, what if we have some user starting a pagedSearch, which can last for a
>> very long period of time ? How would it impact the txn layer and the cache ?
> That would have memory implications for the txn layer. Non flushed
> part of the log would keep growing.
>
> Currently that could lead to deadlock at JDBM layer. However if we
> allow snapshot cache pages to be allocated on demand rather than
> having a fixed number of cache pages, that would translare into a
> memory problem as well.

Beside the global consequences on the server (we need to discuss further 
the item), I just checked the PagedSearch handling code, and found that 
we don't close the cursors when we run the PagedSearchIT tests, except 
at the end of the test when the context is closed.

I have added the needed code to close the cursors when abandoning the 
search request, in the AbandonHandler.

It's quite normal we don't see any unclosed cursor when we log all teh 
open/close pairs, because when the unbind occurs, the cursors are 
correctly closed.

I'll run the tests and let you know if it's any better.


-- 
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com


Mime
View raw message