Return-Path: X-Original-To: apmail-hive-dev-archive@www.apache.org Delivered-To: apmail-hive-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 453FFEEF6 for ; Wed, 6 Mar 2013 19:58:13 +0000 (UTC) Received: (qmail 53016 invoked by uid 500); 6 Mar 2013 19:58:12 -0000 Delivered-To: apmail-hive-dev-archive@hive.apache.org Received: (qmail 52924 invoked by uid 500); 6 Mar 2013 19:58:12 -0000 Mailing-List: contact dev-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hive.apache.org Delivered-To: mailing list dev@hive.apache.org Received: (qmail 52916 invoked by uid 500); 6 Mar 2013 19:58:12 -0000 Delivered-To: apmail-hadoop-hive-dev@hadoop.apache.org Received: (qmail 52882 invoked by uid 99); 6 Mar 2013 19:58:12 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 06 Mar 2013 19:58:12 +0000 Date: Wed, 6 Mar 2013 19:58:12 +0000 (UTC) From: "Brock Noland (JIRA)" To: hive-dev@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HIVE-4132) Connection to Zookeeper is created for each client of HS2 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HIVE-4132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13595042#comment-13595042 ] Brock Noland commented on HIVE-4132: ------------------------------------ Setting maxClientCnxns to a high number in zoo.cfg works around this issue. However, since the Zookeeper object itself is thread safe (except for a few methods noted here http://zookeeper.apache.org/doc/r3.4.5/api/org/apache/zookeeper/ZooKeeper.html) it might make sense to use a single object or a pool of objects. > Connection to Zookeeper is created for each client of HS2 > --------------------------------------------------------- > > Key: HIVE-4132 > URL: https://issues.apache.org/jira/browse/HIVE-4132 > Project: Hive > Issue Type: Sub-task > Components: Server Infrastructure > Reporter: Brock Noland > Labels: HiveServer2 > > Currently when concurrency is enabled and HS2 is used each client will have a connection to Zookeeper. Zookeeper has a property, maxClientCnxns, in the zoo.cfg which limits the number of connections *per* host to a low number like 50. The stack trace of the threads accessing zookeeper is below for reference. > {noformat} > - org.apache.hadoop.hive.ql.lockmgr.zookeeper.ZooKeeperHiveLockManager.lock(java.util.List, boolean) @bci=94, line=220 (Interpreted frame) > - org.apache.hadoop.hive.ql.Driver.acquireReadWriteLocks() @bci=428, line=834 (Interpreted frame) > - org.apache.hadoop.hive.ql.Driver.run(java.lang.String) @bci=453, line=944 (Interpreted frame) > - org.apache.hive.service.cli.operation.SQLOperation.run() @bci=126, line=95 (Interpreted frame) > - org.apache.hive.service.cli.session.HiveSessionImpl.executeStatement(java.lang.String, java.util.Map) @bci=19, line=168 (Interpreted frame) > - org.apache.hive.service.cli.CLIService.executeStatement(org.apache.hive.service.cli.SessionHandle, java.lang.String, java.util.Map) @bci=10, line=153 (Interpreted frame) > - org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(org.apache.hive.service.cli.thrift.TExecuteStatementReq) @bci=41, line=190 (Interpreted frame) > - org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(org.apache.hive.service.cli.thrift.TCLIService$Iface, org.apache.hive.service.cli.thrift.TCLIService$ExecuteStatement_args) @bci=14, line=1133 (Interpreted frame) > {noformat} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira