directory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Emmanuel Lécharny <elecha...@gmail.com>
Subject Re: Release troubles and failing tests
Date Wed, 09 May 2012 04:47:47 GMT
Le 5/8/12 10:13 PM, Emmanuel Lécharny a écrit :
> Le 5/8/12 9:44 PM, Selcuk AYA a écrit :
>> On Tue, May 8, 2012 at 12:34 PM, Emmanuel 
>> Lécharny<elecharny@gmail.com>  wrote:
>>> Le 5/8/12 4:40 PM, Emmanuel Lécharny a écrit :
>>>
>>>> Le 5/8/12 8:05 AM, Selcuk AYA a écrit :
>>>>> Hi,
>>>>>
>>>>> On Sun, May 6, 2012 at 7:26 AM, Emmanuel 
>>>>> Lécharny<elecharny@gmail.com>
>>>>>   wrote:
>>>>>> Le 5/6/12 12:07 PM, Emmanuel Lécharny a écrit :
>>>>>>
>>>>>>> Ok, I think I have found the cursor that was not closed.
>>>>>> Still having the same failure :/ All the cursors are correctly 
>>>>>> closed.
>>>>>> If I
>>>>>> disable the logs (that I use to check the pairing between open and
>>>>>> close), I
>>>>>> get a failure, and if I activate the log, then I get no failure.
>>>>>>
>>>>>> I will try to run all the server-integ tests with NOT_THEADSAFE 
>>>>>> to see
>>>>>> if it
>>>>>> works better.
>>>>> I was trying to build the trunk and see if I can chase the issue we
>>>>> see with the caches.
>>>> FYI, I'm trying to narrow the pb to the tests that make the server to
>>>> freeze, by removing tests from the server-integ suite. It will take 
>>>> a bt of
>>>> time, but as soon as I will have reduced the set of test to the 
>>>> minimum,
>>>> I'll let you know.
>>>>
>>> Ok, I was able to reproduce the pb with just two tests in the suite :
>>>
>>> package org.apache.directory.server.suites;
>>>
>>>
>>> import org.apache.directory.server.annotations.CreateLdapServer;
>>> import org.apache.directory.server.annotations.CreateTransport;
>>> import org.apache.directory.server.core.annotations.ContextEntry;
>>> import org.apache.directory.server.core.annotations.CreateDS;
>>> import org.apache.directory.server.core.annotations.CreateIndex;
>>> import org.apache.directory.server.core.annotations.CreatePartition;
>>> import org.apache.directory.server.core.integ.FrameworkSuite;
>>> import org.apache.directory.server.operations.search.PagedSearchIT;
>>> import org.apache.directory.server.operations.search.SearchIT;
>>> import org.junit.runner.RunWith;
>>> import org.junit.runners.Suite;
>>>
>>>
>>> /**
>>>   * Stock (default configuration) server integration test suite.
>>>   *
>>>   * @author<a href="mailto:dev@directory.apache.org">Apache Directory
>>> Project</a>
>>>   */
>>> @RunWith(FrameworkSuite.class)
>>> @Suite.SuiteClasses(
>>>     {
>>>         // operations.search
>>>         PagedSearchIT.class,
>>>         SearchIT.class,
>>> })
>>> @CreateDS(
>>> name = "SuiteDS",
>>> partitions =
>>>     {
>>>         @CreatePartition(
>>>             name = "example",
>>>             suffix = "dc=example,dc=com",
>>>             contextEntry = @ContextEntry(
>>>                 entryLdif =
>>>                 "dn: dc=example,dc=com\n" +
>>>                     "dc: example\n" +
>>>                     "objectClass: top\n" +
>>>                     "objectClass: domain\n\n"),
>>>             indexes =
>>>                 {
>>>                     @CreateIndex(attribute = "objectClass"),
>>>                     @CreateIndex(attribute = "dc"),
>>>                     @CreateIndex(attribute = "ou")
>>>             })
>>> })
>>> @CreateLdapServer(
>>> transports =
>>>     {
>>>         @CreateTransport(protocol = "LDAP"),
>>>         @CreateTransport(protocol = "LDAPS")
>>> })
>>> public class StockServerISuite
>>> {
>>> }
>>>
>>> It seems like the PersistentSearch is keeping a cursor in the server 
>>> until
>>> the search is abandonned or the user unbinds. That might well be an 
>>> issue.
>>>
>>> Now, what if we have some user starting a pagedSearch, which can 
>>> last for a
>>> very long period of time ? How would it impact the txn layer and the 
>>> cache ?
>> That would have memory implications for the txn layer. Non flushed
>> part of the log would keep growing.
>>
>> Currently that could lead to deadlock at JDBM layer. However if we
>> allow snapshot cache pages to be allocated on demand rather than
>> having a fixed number of cache pages, that would translare into a
>> memory problem as well.
>
> Beside the global consequences on the server (we need to discuss 
> further the item), I just checked the PagedSearch handling code, and 
> found that we don't close the cursors when we run the PagedSearchIT 
> tests, except at the end of the test when the context is closed.
>
> I have added the needed code to close the cursors when abandoning the 
> search request, in the AbandonHandler.
>
> It's quite normal we don't see any unclosed cursor when we log all teh 
> open/close pairs, because when the unbind occurs, the cursors are 
> correctly closed.
>
> I'll run the tests and let you know if it's any better.
Still having issues, even after having closed the cursors in the 
PagedSearch :/
>
>


-- 
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com


Mime
View raw message