Hi yatir,
Can you confirm that zook1 , zook2 can be nslookedup from the client
machine?
We havent seen a bug like this. It would be great to nail this down.
Thanks
mahadev
On 9/14/10 8:44 AM, "Yatir Ben Shlomo" <yatirb@outbrain.com> wrote:
> zook1:2181,zook2:2181,zook3:2181
>
>
> -----Original Message-----
> From: Ted Dunning [mailto:ted.dunning@gmail.com]
> Sent: Tuesday, September 14, 2010 4:11 PM
> To: zookeeper-user@hadoop.apache.org
> Subject: Re: possible bug in zookeeper ?
>
> What was the list of servers that was given originally to open the
> connection to ZK?
>
> On Tue, Sep 14, 2010 at 6:15 AM, Yatir Ben Shlomo <yatirb@outbrain.com>wrote:
>
>> Hi I am using solrCloud which uses an ensemble of 3 zookeeper instances.
>>
>> I am performing survivability tests:
>> Taking one of the zookeeper instances down I would expect the client to use
>> a different zookeeper server instance.
>>
>> But as you can see in the below logs attached
>> Depending on which instance I choose to take down (in my case, the last
>> one in the list of zookeeper servers)
>> the client is constantly insisting on the same zookeeper server (Attempting
>> connection to server zook3/192.168.252.78:2181)
>> and not switching to a different one
>> the problem seems to arrive from ClientCnxn.java
>> Any one has an idea on this ?
>>
>> Solr cloud currently is using zookeeper-3.2.2.jar
>> Is this a know bug that was fixed in later versions ?( 3.3.1)
>>
>> Thanks in advance,
>> Yatir
>>
>>
>> Logs:
>>
>> Sep 14, 2010 9:02:20 AM org.apache.log4j.Category warn
>> WARNING: Ignoring exception during shutdown input
>> java.nio.channels.ClosedChannelException
>> at
>> sun.nio.ch.SocketChannelImpl.shutdownInput(SocketChannelImpl.java:638)
>> at sun.nio.ch.SocketAdaptor.shutdownInput(SocketAdaptor.java:360)
>> at
>> org.apache.zookeeper.ClientCnxn$SendThread.cleanup(zookeeper:ClientCnxn.java)
>> :999)
>> at
>>
org.apache.zookeeper.ClientCnxn$SendThread.run(zookeeper:ClientCnxn.java):970>>
)
>> Sep 14, 2010 9:02:20 AM org.apache.log4j.Category warn
>> WARNING: Ignoring exception during shutdown output
>> java.nio.channels.ClosedChannelException
>> at
>> sun.nio.ch.SocketChannelImpl.shutdownOutput(SocketChannelImpl.java:649)
>> at sun.nio.ch.SocketAdaptor.shutdownOutput(SocketAdaptor.java:368)
>> at
>> org.apache.zookeeper.ClientCnxn$SendThread.cleanup(zookeeper:ClientCnxn.java)
>> :1004)
>> at
>>
org.apache.zookeeper.ClientCnxn$SendThread.run(zookeeper:ClientCnxn.java):970>>
)
>> Sep 14, 2010 9:02:22 AM org.apache.log4j.Category info
>> INFO: Attempting connection to server zook3/192.168.252.78:2181
>> Sep 14, 2010 9:02:22 AM org.apache.log4j.Category warn
>> WARNING: Exception closing session 0x32b105244a20001 to
>> sun.nio.ch.SelectionKeyImpl@3ca58cbf
>> java.net.ConnectException: Connection refused
>> at sun.nio.ch.SocketChannelImpl.$$YJP$$checkConnect(Native Method)
>> at sun.nio.ch.SocketChannelImpl.checkConnect(SocketChannelImpl.java)
>> at
>> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574)
>> at
>>
org.apache.zookeeper.ClientCnxn$SendThread.run(zookeeper:ClientCnxn.java):933>>
)
>> Sep 14, 2010 9:02:22 AM org.apache.log4j.Category warn
>> WARNING: Ignoring exception during shutdown input
>> java.nio.channels.ClosedChannelException
>> at
>> sun.nio.ch.SocketChannelImpl.shutdownInput(SocketChannelImpl.java:638)
>> at sun.nio.ch.SocketAdaptor.shutdownInput(SocketAdaptor.java:360)
>> at
>> org.apache.zookeeper.ClientCnxn$SendThread.cleanup(zookeeper:ClientCnxn.java)
>> :999)
>> at
>>
org.apache.zookeeper.ClientCnxn$SendThread.run(zookeeper:ClientCnxn.java):970>>
)
>> Sep 14, 2010 9:02:22 AM org.apache.log4j.Category warn
>> WARNING: Ignoring exception during shutdown output
>> java.nio.channels.ClosedChannelException
>> at
>> sun.nio.ch.SocketChannelImpl.shutdownOutput(SocketChannelImpl.java:649)
>> at sun.nio.ch.SocketAdaptor.shutdownOutput(SocketAdaptor.java:368)
>> at
>> org.apache.zookeeper.ClientCnxn$SendThread.cleanup(zookeeper:ClientCnxn.java)
>> :1004)
>> at
>>
org.apache.zookeeper.ClientCnxn$SendThread.run(zookeeper:ClientCnxn.java):970>>
)
>> Sep 14, 2010 9:02:22 AM org.apache.log4j.Category info
>> INFO: Attempting connection to server zook3/192.168.252.78:2181
>> Sep 14, 2010 9:02:22 AM org.apache.log4j.Category warn
>> WARNING: Exception closing session 0x32b105244a20000 to
>> sun.nio.ch.SelectionKeyImpl@3960f81b
>> java.net.ConnectException: Connection refused
>> at sun.nio.ch.SocketChannelImpl.$$YJP$$checkConnect(Native Method)
>> at sun.nio.ch.SocketChannelImpl.checkConnect(SocketChannelImpl.java)
>> at
>> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574)
>> at
>>
org.apache.zookeeper.ClientCnxn$SendThread.run(zookeeper:ClientCnxn.java):933>>
)
>> Sep 14, 2010 9:02:22 AM org.apache.log4j.Category warn
>> WARNING: Ignoring exception during shutdown input
>> java.nio.channels.ClosedChannelException
>> at
>> sun.nio.ch.SocketChannelImpl.shutdownInput(SocketChannelImpl.java:638)
>> at sun.nio.ch.SocketAdaptor.shutdownInput(SocketAdaptor.java:360)
>> at
>> org.apache.zookeeper.ClientCnxn$SendThread.cleanup(zookeeper:ClientCnxn.java)
>> :999)
>> at
>>
org.apache.zookeeper.ClientCnxn$SendThread.run(zookeeper:ClientCnxn.java):970>>
)
>> Sep 14, 2010 9:02:22 AM org.apache.log4j.Category warn
>> WARNING: Ignoring exception during shutdown output
>> java.nio.channels.ClosedChannelException
>> at
>> sun.nio.ch.SocketChannelImpl.shutdownOutput(SocketChannelImpl.java:649)
>> at sun.nio.ch.SocketAdaptor.shutdownOutput(SocketAdaptor.java:368)
>> at
>> org.apache.zookeeper.ClientCnxn$SendThread.cleanup(zookeeper:ClientCnxn.java)
>> :1004)
>> at
>>
org.apache.zookeeper.ClientCnxn$SendThread.run(zookeeper:ClientCnxn.java):970>>
)
>>
>>
>
|