directory-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Emmanuel Lécharny <elecha...@gmail.com>
Subject Re: Threads issue
Date Fri, 28 Nov 2014 06:59:42 GMT
Le 28/11/14 02:16, Kiran Ayyagari a écrit :
> On Fri, Nov 28, 2014 at 7:06 AM, Sébastien Bahloul <
> sebastien.bahloul@gmail.com> wrote:
>
>> To Apache LDAP API guys,
>>
>> First, thank you because we are successfully using Apache LDAP API in Ldap
>> Synchronization Connector Open Source project (build 1.0.0-M22). But I
>> encountered an error that seems to be linked to its usage. That's probably
>> because we are using it in a wrong way, but I cannot figured why. Let me
>> introduce the use case: LSC is providing synchronization services based on
>> a source to a destination. Source can be a LDAP directory and there is a
>> mode to run it as a daemon. It's using LDAP Content Synchronization
>> Protocol to attach it to an OpenLDAP directory and to be able to catch
>> updated entries to sync them to a destination (database, another directory,
>> ...). In this configuration, after 1,5 hour the JVM is throwing an
>> OutOfMemory error with "unable to create new native threads". You will find
>> below a pastebin to the thread dump that have been generated each 15
>> minutes which mainly shows thread pool tasks. The fact is LSC is using its
>> own thread pool, but my understanding of the thread dump is that most
>> threads are linked to MINA and to other threads WAITING for MINA threads.
>>
>> check if you are not explicitly closing the connections using close()
> method.
> If the problem persists can you provide us a testcase?

I do think that there are literally hundreds (thousands ?) of opened
connections, which get never closed.

Your pool is probably not closing any connection, and creating new ones
when needed, without any bounds. I suspect that the way you open
connection in, for instance :

    public java.util.Map.Entry<String, LscDatasets> getNextId() throws
LscServiceException {
        Map<String, LscDatasets> temporaryMap = new HashMap<String,
LscDatasets>(1);
        if(sf == null || sf.isCancelled()) {
            try {
                SearchRequest searchRequest = new SearchRequestImpl();
               
searchRequest.addControl(getSearchContinuationControl(srsc.getServerType()));
                searchRequest.setBase(new Dn(getBaseDn()));
                searchRequest.setFilter(getFilterAll());
               
searchRequest.setDerefAliases(getAlias(ldapConn.getDerefAliases()));
                searchRequest.setScope(SearchScope.SUBTREE);
                searchRequest.addAttributes(getAttrsId().toArray(new
String[getAttrsId().size()]));
                sf =
getConnection(ldapConn).searchAsync(searchRequest); 
<<------------------------------- A connection is created here
            } catch (LdapInvalidDnException e) {
                throw new LscServiceException(e.toString(), e);
            } catch (LdapException e) {
                throw new LscServiceException(e.toString(), e);
            }
        }
        Response searchResponse = null;
        try {
            searchResponse = sf.get(1, TimeUnit.NANOSECONDS); 
<<------------------------------------------ A Search is done
        } catch (InterruptedException e) {
            LOGGER.warn("Interrupted search !");
        } catch (ExecutionException e) {
            LOGGER.warn("Execution exception while searching !");
        } catch (TimeoutException e) {
            LOGGER.warn("Timeout during search !");
        }
        if(checkSearchResponse(searchResponse)) {
            SearchResultEntryDecorator sre =
((SearchResultEntryDecorator) searchResponse);
            temporaryMap.put(sre.getObjectName().toString(),
convertEntry(sre.getEntry(), true));
            return temporaryMap.entrySet().iterator().next();
        } else if(searchResponse != null && searchResponse.getType() ==
MessageTypeEnum.SEARCH_RESULT_DONE){
            LdapResult result =
((SearchResultDone)searchResponse).getLdapResult();
            if(result.getResultCode() != ResultCodeEnum.SUCCESS) {
                throw new
LscServiceCommunicationException(result.getDiagnosticMessage(), null);
            }
            sf = null;
        }
        return null; 
<<------------------------------------------------------------------------------------
Where do we release the connection ???
    }



Mime
View raw message