accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Wyatt Frelot <Wyatt.Fre...@altamiracorp.com>
Subject Re: Failed to connect to zookeeper within 2x ZK timeout period 30000
Date Mon, 02 Feb 2015 23:10:04 GMT
Good evening all,

Thanks for the replies. I found the problem. Somehow my /etc/hosts file was changed for the
zookeeper node which was the issue.

Isolated the problem by swapping node name for IP address and started receiving Unknown Host
errors which led me to the client’s etc/hosts file.

Thanks,

Wyatt
> On Feb 2, 2015, at 17:32, Josh Elser <elserj@apache.org> wrote:
> 
> See swappiness: http://en.wikipedia.org/wiki/Swappiness
> 
> $ cat /proc/sys/vm/swappiness
> 
> # echo 20 > /proc/sys/vm/swappiness
> 
> tl;dr Having a "large" value for swappiness might cause the operating system to move
your process to "swap" instead of main memory which would likely cause it to lose its ZooKeeper
locks and die. The larger the value, the more aggressive the OS is in putting things to swap,
the lower the value, the less so. It's not recommended to ever disable swap completely though.
> 
> Wyatt Frelot wrote:
>> I have been looking in the logs and I found something…just not sure about it’s
impact:
>> 2015-02-02 22:21:04,390 [server.Accumulo] WARN : System swappiness setting is greater
than ten (60) which can cause time-sensitive operations to be delayed. Accumulo is time sensitive
because it needs to maintain distributed lock agreement.
>> 
>> Would this cause it?
>> 
>> Wyatt
>>> On Feb 2, 2015, at 17:03, John Vines <vines@apache.org <mailto:vines@apache.org>>
wrote:
>>> 
>>> I'm sorry, I meant nmode:2181 in your actual code.
>>> 
>>> On Mon, Feb 2, 2015 at 4:53 PM, Wyatt Frelot <Wyatt.Frelot@altamiracorp.com
<mailto:Wyatt.Frelot@altamiracorp.com>> wrote:
>>> 
>>>    John,
>>> 
>>>    I think I did it right. Here is what I get when I run it the way
>>>    that you suggested.
>>> 
>>>    vagrant@mnode:/cloud/accumulo/lib$ nc mnode:2181
>>>    This is nc from the netcat-openbsd package. An alternative nc is
>>>    available
>>>    in the netcat-traditional package.
>>>    usage: nc [-46bCDdhjklnrStUuvZz] [-I length] [-i interval] [-O
>>>    length]
>>>    [-P proxy_username] [-p source_port] [-q seconds] [-s source]
>>>    [-T toskeyword] [-V rtable] [-w timeout] [-X proxy_protocol]
>>>    [-x proxy_address[:port]] [destination] [port]
>>> 
>>> 
>>>>    On Feb 2, 2015, at 16:38, John Vines <jvines@gmail.com
>>>>    <mailto:jvines@gmail.com>> wrote:
>>>> 
>>>>    It looks like you're declaring your ZK connect string as
>>>>    "mnode". What happens if you do "mnode:2181"?
>>>> 
>>>>    On Mon, Feb 2, 2015 at 4:17 PM, Wyatt Frelot
>>>>    <Wyatt.Frelot@altamiracorp.com
>>>>    <mailto:Wyatt.Frelot@altamiracorp.com>> wrote:
>>>> 
>>>>        Mike
>>>> 
>>>>        If I am reading this correctly, it appears as if everything
>>>>        is in order:
>>>> 
>>>>        vagrant@mnode:/vagrant$ nc mnode 2181
>>>>        stat
>>>>        Zookeeper version: 3.4.6-1569965, built on 02/20/2014 09:09 GMT
>>>>        Clients:
>>>>        /192.168.15.4:44488[1](queued=0,recved=84,sent=85)
>>>>        /192.168.15.2:48504[0](queued=0,recved=1,sent=0)
>>>>        /192.168.15.2:48262[1](queued=0,recved=1577,sent=1598)
>>>>        /192.168.15.5:55540[1](queued=0,recved=434,sent=449)
>>>>        /192.168.15.3:37401[1](queued=0,recved=236,sent=239)
>>>>        /192.168.15.4:44480[1](queued=0,recved=244,sent=251)
>>>>        /192.168.15.5:55559[1](queued=0,recved=100,sent=101)
>>>>        /192.168.15.5:55564[1](queued=0,recved=88,sent=89)
>>>> 
>>>>        Latency min/avg/max: 0/0/62
>>>>        Received: 3114
>>>>        Sent: 3162
>>>>        Connections: 8
>>>>        Outstanding: 0
>>>>        Zxid: 0x71a
>>>>        Mode: standalone
>>>>        Node count: 270
>>>> 
>>>>        Wyatt
>>>>>        On Feb 2, 2015, at 16:06, Mike Drob <mdrob@mdrob.com
>>>>>        <mailto:mdrob@mdrob.com>> wrote:
>>>>> 
>>>>>        nc [zk-host] [zk-port]
>>>>> 
>>>>>        --         Cheers
>>>>>        ~John
>>>> 
>>> 
>>> 
>> 

Mime
View raw message