hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "jack levin (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-4733) Rest client does not close zookeeper sessions (leaking sessions for each GET or PUT)
Date Fri, 04 Nov 2011 19:27:51 GMT

    [ https://issues.apache.org/jira/browse/HBASE-4733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13144264#comment-13144264
] 

jack levin commented on HBASE-4733:
-----------------------------------

FYI this patch applies well to 0.90.4, those folks that use it in production should be able
to fix their REST issue right away.  Works well in our cluster.
                
> Rest client does not close zookeeper sessions (leaking sessions for each GET or PUT)
> ------------------------------------------------------------------------------------
>
>                 Key: HBASE-4733
>                 URL: https://issues.apache.org/jira/browse/HBASE-4733
>             Project: HBase
>          Issue Type: Bug
>          Components: rest
>    Affects Versions: 0.90.4
>         Environment: Fedora 13.
>            Reporter: jack levin
>              Labels: newbie
>             Fix For: 0.90.4
>
>         Attachments: my.patch
>
>
> This will appear in the log once the zookeeper connection/session leaking will grow to
2000, zookeeper won't be able to accept any more connections causing REST RPC calls to fail,
here is the log when the problem is in progress:
> 2011-10-26 01:35:49,270 WARN org.apache.zookeeper.server.NIOServerCnxn: Too many connections
from /10.102.255.3 - max is 2000
> 2011-10-26 01:35:49,300 WARN org.apache.zookeeper.server.NIOServerCnxn: Too many connections
from /10.102.255.4 - max is 2000
> 2011-10-26 01:35:49,317 WARN org.apache.zookeeper.server.NIOServerCnxn: Too many connections
from /10.102.255.4 - max is 2000
> 2011-10-26 01:35:49,321 WARN org.apache.zookeeper.server.NIOServerCnxn: Too many connections
from /10.102.255.3 - max is 2000
> 2011-10-26 01:35:49,323 WARN org.apache.zookeeper.server.NIOServerCnxn: Too many connections
from /10.101.255.3 - max is 2000
> 2011-10-26 01:35:49,367 WARN org.apache.zookeeper.server.NIOServerCnxn: Too many connections
from /10.101.255.2 - max is 2000
> 2011-10-26 01:35:49,375 WARN org.apache.zookeeper.server.NIOServerCnxn: Too many connections
from /10.102.255.4 - max is 2000
> 2011-10-26 01:35:49,382 WARN org.apache.zookeeper.server.NIOServerCnxn: Too many connections
from /10.102.255.2 - max is 2000
> 2011-10-26 01:35:49,404 WARN org.apache.zookeeper.server.NIOServerCnxn: Too many connections
from /10.101.255.2 - max is 2000
> 2011-10-26 01:35:49,429 WARN org.apache.zookeeper.server.NIOServerCnxn: Too many connections
from /10.102.255.2 - max is 2000
> 2011-10-26 01:35:49,439 WARN org.apache.zookeeper.server.NIOServerCnxn: Too many connections
from /10.102.255.3 - max is 2000
> 2011-10-26 01:35:49,469 WARN org.apache.zookeeper.server.NIOServerCnxn: Too many connections
from /10.102.255.3 - max is 2000
> 2011-10-26 01:35:49,489 WARN org.apache.zookeeper.server.NIOServerCnxn: Too many connections
from /10.101.255.3 - max is 2000
> 2011-10-26 01:35:49,501 WARN org.apache.zookeeper.server.NIOServerCnxn: Too many connections
from /10.101.255.2 - max is 2000
> 2011-10-26 01:35:49,584 WARN org.apache.zookeeper.server.NIOServerCnxn: Too many connections
from /10.102.255.2 - max is 2000
> After the fix, the log looks much better, and we can observe zookeeper connections closing
after every RPC call:
> 2011-11-02 15:50:14,339 INFO org.apache.zookeeper.server.NIOServerCnxn: Accepted socket
connection from /10.101.255.2:37225
> 2011-11-02 15:50:14,339 INFO org.apache.zookeeper.server.NIOServerCnxn: Client attempting
to establish new session at /10.101.255.2:37225
> 2011-11-02 15:50:14,340 INFO org.apache.zookeeper.server.NIOServerCnxn: Established session
0x3352857cb1f19f with negotiated timeout 180000 for client /10.101.255.2:37225
> 2011-11-02 15:50:14,363 INFO org.apache.zookeeper.server.NIOServerCnxn: Closed socket
connection for client /10.101.255.2:37225 which had sessionid 0x3352857cb1f19f
> 2011-11-02 15:50:14,723 INFO org.apache.zookeeper.server.NIOServerCnxn: Accepted socket
connection from /10.101.255.2:38089
> 2011-11-02 15:50:14,723 INFO org.apache.zookeeper.server.NIOServerCnxn: Client attempting
to establish new session at /10.101.255.2:38089
> 2011-11-02 15:50:14,725 INFO org.apache.zookeeper.server.NIOServerCnxn: Established session
0x3352857cb1f1a0 with negotiated timeout 180000 for client /10.101.255.2:38089
> 2011-11-02 15:50:14,771 INFO org.apache.zookeeper.server.NIOServerCnxn: Closed socket
connection for client /10.101.255.2:38089 which had sessionid 0x3352857cb1f1a0
> 2011-11-02 15:50:16,326 INFO org.apache.zookeeper.server.NIOServerCnxn: Accepted socket
connection from /10.101.255.3:34085
> 2011-11-02 15:50:16,326 INFO org.apache.zookeeper.server.NIOServerCnxn: Client attempting
to establish new session at /10.101.255.3:34085
> 2011-11-02 15:50:16,328 INFO org.apache.zookeeper.server.NIOServerCnxn: Established session
0x3352857cb1f1a1 with negotiated timeout 180000 for client /10.101.255.3:34085

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message