Mailing-List: contact user-help@zookeeper.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@zookeeper.apache.org
MIME-Version: 1.0
In-Reply-To: 
 <CAL85UBavr-==ncfwP8ewDNUaStbbJoqZhRsCW6ug69vTqA9swQ@mail.gmail.com>
References: 
 <CAL85UBYqa7NoV_r-Ni4ViDUt1zn9L=X0Hf8KwuhW1x7ff=BYUw@mail.gmail.com>
	<CAL85UBa6cBiNPmk6PkoUZDfXYuznG_WVeqtdqW4VSrNALfvgOg@mail.gmail.com>
	<AD345FD6-545A-4E3B-B223-9E19C7B8A0EB@apache.org>
	<CAL85UBYogrWw_oUaEj8mutEvekJWQ1106_b5cakTV+-UxeX+yQ@mail.gmail.com>
	<CAL85UBa-EKBu+kLzg5K0WPO=8+vU8kGPr7wv+=1iiuyWZ539Dw@mail.gmail.com>
	<CAB5oV28ua69iaZZbz1qNp96=sS5r8PbKM9-ngcSrTTZH_VT-SQ@mail.gmail.com>
	<CAL85UBavr-==ncfwP8ewDNUaStbbJoqZhRsCW6ug69vTqA9swQ@mail.gmail.com>
Date: Thu, 7 Apr 2016 18:50:42 -0700
Message-ID: 
 <CAL85UBYOMCveSDhqLhCacNXZngsScvbtpADVYt2mBb7pXAZR+g@mail.gmail.com>
Subject: Re: node 2 not rejoining cluster
From: s influxdb <elastic.l.k@gmail.com>
To: user@zookeeper.apache.org
Content-Type: multipart/alternative; boundary=089e01493e622ec398052fef6cfb

--089e01493e622ec398052fef6cfb
Content-Type: text/plain; charset=UTF-8

I ran tcpdump on all the three nodes.
It looks like that for every  [PSH, ACK] there is a missing [ACK] from the
other nodes to this 2nd node on port 3888.


On Thu, Apr 7, 2016 at 1:29 PM, s influxdb <elastic.l.k@gmail.com> wrote:

> Thanks Flavio for your quick replies.
> The zookeeper version is 3.4.6
>
>
>
> On Thu, Apr 7, 2016 at 1:23 PM, Flavio P JUNQUEIRA <fpj@apache.org> wrote:
>
>> You need to determine why it is not receiving notification messages. From
>> the information you've given, it doesn't look like a zookeeper code issue.
>>
>> BTW, which version are you using?
>>
>> -Flavio
>> On 7 Apr 2016 21:20, "s influxdb" <elastic.l.k@gmail.com> wrote:
>>
>> > nothin on the iptables firewall .
>> >
>> > What options do i have to reconnect this node to the cluster ?
>> >
>> >
>> > On Thu, Apr 7, 2016 at 10:14 AM, s influxdb <elastic.l.k@gmail.com>
>> wrote:
>> >
>> > > telnet works on 2888 and 3888 to the other nodes. Now i see
>> > > java.net.SocketTimeoutException: connect timed out messages in the
>> logs
>> > for
>> > > node 2
>> > >
>> > > On Thu, Apr 7, 2016 at 3:05 AM, Flavio Junqueira <fpj@apache.org>
>> wrote:
>> > >
>> > >> I only see notifications from the node to itself. It says that it is
>> > >> connected to 1, but it doesn't seem to be receiving the notification
>> > from
>> > >> 1. It also doesn't seem to be receiving the connection request from
>> 3.
>> > >>
>> > >> Last time I've seen something like this was due to iptables rules,
>> but
>> > if
>> > >> it was working before and no configuration has changed, then I don't
>> > know
>> > >> what it could be.
>> > >>
>> > >> -Flavio
>> > >>
>> > >> > On 07 Apr 2016, at 05:43, s influxdb <elastic.l.k@gmail.com>
>> wrote:
>> > >> >
>> > >> > this is the pastie
>> > >> > http://pastie.org/10788301
>> > >> >
>> > >> > On Wed, Apr 6, 2016 at 9:41 PM, s influxdb <elastic.l.k@gmail.com>
>> > >> wrote:
>> > >> >
>> > >> >> We had one of the node giving OOM java.lang.OutOfMemoryError:
>> unable
>> > to
>> > >> >> create new native thread and then being unresponsive.
>> > >> >>
>> > >> >> We tried to add the node back to the cluster but with no luck.
>> > >> >>
>> > >> >> It doesn't seem to "Receive any notification "  messages from the
>> > other
>> > >> >> nodes.
>> > >> >> Keeps "Sending notifications " in loop
>> > >> >>
>> > >> >> Please see attached the logs of the node that is out of rotation.
>> > >> >>
>> > >> >> Any inputs appreciated.
>> > >> >>
>> > >> >> Thanks
>> > >> >>
>> > >>
>> > >>
>> > >
>> >
>>
>
>

--089e01493e622ec398052fef6cfb--