hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Edward Capriolo <edlinuxg...@gmail.com>
Subject Re: hiveserver leak zookeeper connections?
Date Mon, 07 Jan 2013 16:35:41 GMT
I had a similar problem with hive 0.7.0 and zk-locking. I turned it off.
Just too much trouble.

On Mon, Jan 7, 2013 at 8:48 AM, wzc <wzc1989@gmail.com> wrote:

> @ Ashutosh:
> I patched HIVE-3723 with hive 0.9 but it didn't work, I would try using
> hive 0.10.0.
> Thanks~
>
>
> 2013/1/6 Ashutosh Chauhan <hashutosh@apache.org>
>
>> We have found one source of leak which we think we have fixed.
>> https://issues.apache.org/jira/browse/HIVE-3723 This is a part of
>> hive-0.10 RC0 http://people.apache.org/~hashutosh/hive-0.10.0-rc0/ You
>> can try hive from there and see if this fixes this connection leaking
>> problem.
>>
>> Thanks,
>> Ashutosh
>>
>>
>> On Fri, Jan 4, 2013 at 10:51 PM, wzc <wzc1989@gmail.com> wrote:
>>
>>> Hi all:
>>> I am using hive 0.9 and zookeeper 3.3.5, and I set
>>> hive.support.concurrency=true for locking. when I run query through
>>> hiveserver I keep receiving warnings like :
>>>
>>> 2013-01-05 12:03:56,109 WARN  zookeeper.ClientCnxn (ClientCnxn.java:run(1057))
- Session 0x0 for server data-dev-01/192.168.2.241:2181, unexpected error, closing socket
connection and a
>>> ttempting reconnect
>>> java.io.IOException: Connection reset by peer
>>>         at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
>>>         at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
>>>         at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:218)
>>>         at sun.nio.ch.IOUtil.read(IOUtil.java:186)
>>>         at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:359)
>>>         at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:66
>>> 2013-01-05 12:03:56,350 WARN  zookeeper.ClientCnxn (ClientCnxn.java:run(1057))
- Session 0x0 for server data-dev-02/192.168.2.237:2181, unexpected error, closing socket
connection and a
>>> ttempting reconnect
>>> java.io.IOException: Connection reset by peer
>>>         at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
>>>         at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
>>>         at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:218)
>>>         at sun.nio.ch.IOUtil.read(IOUtil.java:186)
>>>         at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:359)
>>>         at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:66)
>>>         at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:291)
>>>         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1035)
>>> 2013-01-05 12:03:56,476 WARN  zookeeper.ClientCnxn (ClientCnxn.java:run(1057))
- Session 0x0 for server data-dev-02/192.168.2.237:2181, unexpected error, closing socket
connection and a
>>> ttempting reconnect
>>> java.io.IOException: Connection reset by peer
>>>         at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
>>>         at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
>>>         at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:218)
>>>         at sun.nio.ch.IOUtil.read(IOUtil.java:186)
>>>         at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:359)
>>>         at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:66)
>>>         at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:291)
>>>         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1035)
>>>
>>>
>>> and the query fails. I run 'netstat -anlp | grep 2181' on the server which hiveserver
is running and find many ESTABLISHED connections:
>>>
>>> tcp6       0      0 192.168.2.241:50033     192.168.2.241:2181      ESTABLISHED
11765/java
>>> tcp6       0      0 192.168.2.241:2181      192.168.2.241:50024     ESTABLISHED
22103/java
>>> tcp6       0      0 192.168.2.241:59436     192.168.2.237:2181      ESTABLISHED
11765/java
>>> tcp6       0      0 192.168.2.241:50015     192.168.2.241:2181      ESTABLISHED
11765/java
>>> tcp6       0      0 192.168.2.241:59432     192.168.2.237:2181      ESTABLISHED
11765/java
>>> tcp6       0      0 192.168.2.241:2181      192.168.2.241:50014     ESTABLISHED
22103/java
>>> tcp6       0      0 192.168.2.241:59444     192.168.2.237:2181      ESTABLISHED
11765/java
>>> tcp6       0      0 192.168.2.241:2181      192.168.2.241:50029     ESTABLISHED
22103/java
>>> tcp6       0      0 192.168.2.241:2181      192.168.2.241:50015     ESTABLISHED
22103/java
>>> tcp6       0      0 192.168.2.241:50031     192.168.2.241:2181      ESTABLISHED
11765/java
>>> tcp6       0      0 192.168.2.241:50024     192.168.2.241:2181      ESTABLISHED
11765/java
>>> tcp6       0      0 192.168.2.241:2181      192.168.2.241:49998     ESTABLISHED
22103/java
>>> tcp6       0      0 192.168.2.241:59423     192.168.2.237:2181      ESTABLISHED
11765/java
>>> tcp6       0      0 192.168.2.241:50021     192.168.2.241:2181      ESTABLISHED
11765/java
>>> tcp6       0      0 192.168.2.241:59442     192.168.2.237:2181      ESTABLISHED
11765/java
>>> tcp6       0      0 192.168.2.241:2181      192.168.2.241:50033     ESTABLISHED
22103/java
>>> tcp6       0      0 192.168.2.241:59434     192.168.2.237:2181      ESTABLISHED
11765/java
>>> tcp6       0      0 192.168.2.241:59426     192.168.2.237:2181      ESTABLISHED
11765/java
>>> tcp6       0      0 192.168.2.241:59437     192.168.2.237:2181      ESTABLISHED
11765/java
>>> tcp6       0      0 192.168.2.241:2181      192.168.2.241:50005     ESTABLISHED
22103/java
>>> tcp6       0      0 192.168.2.241:49998     192.168.2.241:2181      ESTABLISHED
11765/java
>>> tcp6       0      0 192.168.2.241:50029     192.168.2.241:2181      ESTABLISHED
11765/java
>>> tcp6       0      0 192.168.2.241:59439     192.168.2.237:2181      ESTABLISHED
11765/java
>>>
>>>
>>> when I turn off hive.support.concurrency option it's ok.
>>>
>>> I think maybe hiveserver is leaking zookeeper connections. Can anyone give me
some hint?
>>>
>>>
>>> Thanks.
>>>
>>>
>>
>

Mime
View raw message