hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Haroon (JIRA)" <>
Subject [jira] [Commented] (HIVE-5853) Hive Lock Manager leaks zookeeper connections
Date Thu, 28 Nov 2013 16:26:37 GMT


Haroon commented on HIVE-5853:

Yep, the issue persists with HiveServer2 as well. I am connecting through c++ thrift interface.
Connections continue to grow until the max connection limit is hit. As Harel Ben Attia mentioned,
it works fine with concurrency support turned off.

Looks like Hive-4132 and Hive-4719 directly relate to this issue and should help the cause
once they get fixed.

> Hive Lock Manager leaks zookeeper connections
> ---------------------------------------------
>                 Key: HIVE-5853
>                 URL:
>             Project: Hive
>          Issue Type: Bug
>    Affects Versions: 0.10.0
>            Reporter: Harel Ben Attia
> Hive 0.10 leaks zookeeper connections from ZooKeeperHiveLockManager. HIVE-3723 describes
a similar issue for cases of semantic errors and failures, but we're experiencing a consistent
connection leak per query (even simple successful queries like "select * from dual").
> Workaround: When turning off, everything works fine - no leak
(obviously, since the lock manager is not used).
> Details:
> OS: CentOS 5.9
> Hive version: hive-server-0.10.0+67-1.cdh4.2.0.p0.10.el5 and hive-0.10.0+198-1.cdh4.4.0.p0.15.el5
> Hadoop version: CDH4.2
> Namenode uses HA. Hive's zookeeper configuration uses the NN zookeeper.
> The problem occurs both when using the python thrift API, and the java thrift API. 
> The leak happens even when we're running repeated "select * from dual" queries. We've
checked the zookeeper connections using "netstat -n | grep 2181 | grep ESTAB | wc -l".
> Eventually, the connection from the client reach the max connections per client limit
in ZK, causing new queries to get stuck and never return.
> We'll gladly provide more information if needed.

This message was sent by Atlassian JIRA

View raw message