airavata-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Marcus Christie (Jira)" <j...@apache.org>
Subject [jira] [Commented] (AIRAVATA-3186) ThriftClientPool$ThriftClientException: Could not get a resource from the pool
Date Fri, 08 Nov 2019 20:22:00 GMT

    [ https://issues.apache.org/jira/browse/AIRAVATA-3186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16970564#comment-16970564
] 

Marcus Christie commented on AIRAVATA-3186:
-------------------------------------------

Note: the abandoned object logging goes to STDOUT so it will end up in the output.log file,
not the airavata.log file.

> ThriftClientPool$ThriftClientException: Could not get a resource from the pool
> ------------------------------------------------------------------------------
>
>                 Key: AIRAVATA-3186
>                 URL: https://issues.apache.org/jira/browse/AIRAVATA-3186
>             Project: Airavata
>          Issue Type: Bug
>          Components: Airavata API
>            Reporter: Marcus Christie
>            Assignee: Marcus Christie
>            Priority: Major
>
> {noformat}
> 2019-07-26 16:02:39,871 [pool-24-thread-28] ERROR o.a.t.server.TThreadPoolServer  - Error
occurred during processing of message.
> org.apache.airavata.common.utils.ThriftClientPool$ThriftClientException: Could not get
a resource from the pool
>         at org.apache.airavata.common.utils.ThriftClientPool.getResource(ThriftClientPool.java:144)
>         at org.apache.airavata.api.server.handler.AiravataServerHandler.userHasAccess(AiravataServerHandler.java:5370)
>         at org.apache.airavata.service.security.interceptor.SecurityInterceptor.invoke(SecurityInterceptor.java:56)
>         at org.apache.airavata.api.Airavata$Processor$userHasAccess.getResult(Airavata.java:22848)
>         at org.apache.airavata.api.Airavata$Processor$userHasAccess.getResult(Airavata.java:22832)
>         at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
>         at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
>         at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>         at java.lang.Thread.run(Thread.java:748)
> Caused by: java.util.NoSuchElementException: Timeout waiting for idle object
>         at org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:1174)
>         at org.apache.airavata.common.utils.ThriftClientPool.getResource(ThriftClientPool.java:132)
>         ... 10 common frames omitted
> {noformat}
> h5. TODO
> - [x] add properties for configuring abandoned object removal. There's some overhead
so probably don't want this turned on in production.
> - [ ] Resource leak warning here: https://github.com/apache/airavata/blob/staging/modules/commons/src/main/java/org/apache/airavata/common/utils/ThriftClientPool.java#L111
> - [ ] merge back to develop
> - [ ] add property for configuring whether to [logAbandoned|https://commons.apache.org/proper/commons-pool/api-2.7.0/index.html]?
This is what we would want to turn off in production because of the overhead involved, but
we might wanted to keep abandoned object removal turned on in production.
> - [ ] question: can we use JMX to monitor size/status of connection pools?



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message