Mailing-List: contact user-help@zookeeper.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@zookeeper.apache.org
MIME-Version: 1.0
In-Reply-To: <B38835B3-E673-4EE6-9717-AB2B1AE12577@yahoo-inc.com>
References: <1331111068048-7351244.post@n2.nabble.com>
 <B38835B3-E673-4EE6-9717-AB2B1AE12577@yahoo-inc.com>
From: Patrick Hunt <phunt@apache.org>
Date: Wed, 7 Mar 2012 09:16:22 -0800
Message-ID: 
 <CANLc_9JCQbUpTRg+qVQVRvDqstAmLjAPYvdFT5-UcgpHukeXKQ@mail.gmail.com>
Subject: Re: Setting Zookeeper server cluster on single host
To: user@zookeeper.apache.org
Cc: "hrishikesh7@yahoo.com" <hrishikesh7@yahoo.com>,
	"zookeeper-user@hadoop.apache.org" <zookeeper-user@hadoop.apache.org>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

I created the following tool specifically to help with this, you might
find it useful: https://github.com/phunt/zkconf

Patrick

On Wed, Mar 7, 2012 at 7:12 AM, Alexander Shraer <shralex@yahoo-inc.com> wr=
ote:
> you need to have a different set of ports for each server in the config f=
iles. Otherwise ports 5181 and 5888 just get taken by the first server...
>
> server.1=3D127.0.0.1:5181:5887
> server.2=3D127.0.0.1:5182:5888
> server.3=3D127.0.0.1:5183:5889
>
>
> Alex
>
>
> On Mar 7, 2012, at 6:57 AM, "hrm" <hrishikesh7@yahoo.com<mailto:hrishikes=
h7@yahoo.com>> wrote:
>
> I am using Zookeeper 3.4.3 for setting server cluster on single host.
> I have three separate installations of zookeeper and corresponding dataDi=
r
> on a host.
>
> Following configuration is present on each server:
> # The number of milliseconds of each tick
> tickTime=3D2000
> # The number of ticks that the initial
> # synchronization phase can take
> initLimit=3D10
> # The number of ticks that can pass between
> # sending a request and getting an acknowledgement
> syncLimit=3D5
> # the directory where the snapshot is stored.
> # do not use /tmp for storage, /tmp here is just
> # example sakes.
> dataDir=3D/var/lib/zookeeper
> # the port at which the clients will connect
> clientPort=3D2181
> # cluster servers
> server.1=3D127.0.0.1:5181:5888
> server.2=3D127.0.0.1:5181:5888
> server.3=3D127.0.0.1:5181:5888
>
> I cannot start the server 2 and 3 because the client port is already take=
n.
> I get following error.
>
> 012-03-07 10:56:34,481 [myid:] - INFO =A0[main:QuorumPeerConfig@101][] -
> Reading configuration from: /usr/local/zookeeper_3/bin/../conf/zoo.cfg
> 2012-03-07 10:56:34,540 [myid:] - INFO =A0[main:QuorumPeerConfig@334][] -
> Defaulting to majority quorums
> 2012-03-07 10:56:34,589 [myid:3] - INFO =A0[main:DatadirCleanupManager@78=
][] -
> autopurge.snapRetainCount set to 3
> 2012-03-07 10:56:34,590 [myid:3] - INFO =A0[main:DatadirCleanupManager@79=
][] -
> autopurge.purgeInterval set to 0
> 2012-03-07 10:56:34,591 [myid:3] - INFO =A0[main:DatadirCleanupManager@10=
1][]
> - Purge task is not scheduled.
> 2012-03-07 10:56:34,616 [myid:3] - INFO =A0[main:QuorumPeerMain@127][] -
> Starting quorum peer
> 2012-03-07 10:56:34,901 [myid:3] - INFO =A0[main:NIOServerCnxnFactory@110=
][] -
> binding to port 0.0.0.0/0.0.0.0:2181
> 2012-03-07 10:56:34,902 [myid:3] - ERROR [main:QuorumPeerMain@89][] -
> Unexpected exception, exiting abnormally
> java.net.BindException: Address already in use
> =A0 =A0 =A0 at sun.nio.ch.Net.bind(Native Method)
> =A0 =A0 =A0 at
> sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:137)
> =A0 =A0 =A0 at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.ja=
va:77)
> =A0 =A0 =A0 at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.ja=
va:70)
> =A0 =A0 =A0 at
> org.apache.zookeeper.server.NIOServerCnxnFactory.configure(NIOServerCnxnF=
actory.java:111)
> =A0 =A0 =A0 at
> org.apache.zookeeper.server.quorum.QuorumPeerMain.runFromConfig(QuorumPee=
rMain.java:130)
> =A0 =A0 =A0 at
> org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(Quorum=
PeerMain.java:111)
> =A0 =A0 =A0 at
> org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.jav=
a:78)
>
> After giving each server a different client port, I could start all serve=
rs.
> However, I could see following error in one of the server logs.
>
> 2012-03-07 13:21:33,399 [myid:2] - INFO
> [QuorumPeer[myid=3D2]/0:0:0:0:0:0:0:0:2182:Leader@296][] - LEADING - LEAD=
ER
> ELECTION TOOK - 431
> 2012-03-07 13:21:33,403 [myid:2] - INFO
> [QuorumPeer[myid=3D2]/0:0:0:0:0:0:0:0:2182:FileSnap@83][] - Reading snaps=
hot
> /var/lib/zookeeper_2/version-2/snapshot.400000000
> 2012-03-07 13:21:33,408 [myid:2] - INFO
> [QuorumPeer[myid=3D2]/0:0:0:0:0:0:0:0:2182:FileTxnSnapLog@238][] -
> Snapshotting: 0x400000000 to
> /var/lib/zookeeper_2/version-2/snapshot.400000000
> 2012-03-07 13:21:33,960 [myid:2] - INFO
> [/127.0.0.1:5888:QuorumCnxManager$Listener@486][] - My election bind port=
:
> 0.0.0.0/0.0.0.0:5888
> 2012-03-07 13:21:33,961 [myid:2] - ERROR
> [/127.0.0.1:5888:QuorumCnxManager$Listener@499][] - Exception while
> listening
> java.net.BindException: Address already in use
> =A0 =A0 =A0 at java.net.PlainSocketImpl.socketBind(Native Method)
> =A0 =A0 =A0 at
> java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:353)
> =A0 =A0 =A0 at java.net.ServerSocket.bind(ServerSocket.java:336)
> =A0 =A0 =A0 at java.net.ServerSocket.bind(ServerSocket.java:294)
> =A0 =A0 =A0 at
> org.apache.zookeeper.server.quorum.QuorumCnxManager$Listener.run(QuorumCn=
xManager.java:489)
> 2012-03-07 13:21:34,208 [myid:2] - INFO
> [LearnerHandler-/127.0.0.1:33958:LearnerHandler@264][] - Follower sid: 1 =
:
> info : org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@1ad98ef
> 2012-03-07 13:21:34,353 [myid:2] - INFO
> [LearnerHandler-/127.0.0.1:33958:LearnerHandler@319][] - Synchronizing wi=
th
> Follower sid: 1 maxCommittedLog=3D0x0 minCommittedLog=3D0x0
> peerLastZxid=3D0x300000000
> 2012-03-07 13:21:34,354 [myid:2] - INFO
> [LearnerHandler-/127.0.0.1:33958:LearnerHandler@418][] - Sending snapshot
> last zxid of peer is 0x300000000 =A0zxid of leader is 0x500000000sent zxi=
d of
> db as 0x400000000
> 2012-03-07 13:21:34,420 [myid:2] - WARN
> [LearnerHandler-/127.0.0.1:33958:Leader@505][] - Commiting zxid 0x5000000=
00
> from /127.0.0.1:5181 not first!
> 2012-03-07 13:21:34,421 [myid:2] - WARN
> [LearnerHandler-/127.0.0.1:33958:Leader@507][] - First is 0x0
> 2012-03-07 13:21:34,421 [myid:2] - INFO
> [LearnerHandler-/127.0.0.1:33958:Leader@529][] - Have quorum of supporter=
s;
> starting up and setting last processed zxid: 0x500000000
> 2012-03-07 13:21:34,961 [myid:2] - INFO
> [/127.0.0.1:5888:QuorumCnxManager$Listener@486][] - My election bind port=
:
> 0.0.0.0/0.0.0.0:5888
> 2012-03-07 13:21:34,961 [myid:2] - ERROR
> [/127.0.0.1:5888:QuorumCnxManager$Listener@499][] - Exception while
> listening
> java.net.BindException: Address already in use
> =A0 =A0 =A0 at java.net.PlainSocketImpl.socketBind(Native Method)
> =A0 =A0 =A0 at
> java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:353)
> =A0 =A0 =A0 at java.net.ServerSocket.bind(ServerSocket.java:336)
> =A0 =A0 =A0 at java.net.ServerSocket.bind(ServerSocket.java:294)
> =A0 =A0 =A0 at
> org.apache.zookeeper.server.quorum.QuorumCnxManager$Listener.run(QuorumCn=
xManager.java:489)
> 2012-03-07 13:21:35,962 [myid:2] - INFO
> [/127.0.0.1:5888:QuorumCnxManager$Listener@512][] - Leaving listener
> 2012-03-07 13:21:35,963 [myid:2] - ERROR
> [/127.0.0.1:5888:QuorumCnxManager$Listener@514][] - As I'm leaving the
> listener thread, I won't be able to participate in leader election any
> longer: /127.0.0.1:5888
>
>
> So my queries are:
> 1. What does the above server error means? After becoming leader (myid:2)=
 is
> it somehow cannot participate in leader election.
> 1. =A0Is the configuration proper? I looked at online resources I couldn'=
t get
> any pointers indicating something wrong. I am trying to create cluster of
> three servers (znodes) on single host.
> 2. =A0Sometime I could start at most 2 client which talk to servers and I=
 get
> data replicated across, however third client does experience problems whi=
le
> connecting to server. The error stack is given below:
>
> 2012-03-07 12:46:36,496 [myid:] - INFO
> [main-SendThread(localhost:2183):ClientCnxn$SendThread@933] - Opening soc=
ket
> connection to server localhost/127.0.0.1:2183
> 2012-03-07 12:46:36,497 [myid:] - INFO
> [main-SendThread(localhost:2183):ZooKeeperSaslClient@125] - Client will n=
ot
> SASL-authenticate because the default JAAS configuration section 'Client'
> could not be found. If you are not using SASL, you may ignore this. On th=
e
> other hand, if you expected SASL to work, please fix your JAAS
> configuration.
> 2012-03-07 12:46:36,497 [myid:] - INFO
> [main-SendThread(localhost:2183):ClientCnxn$SendThread@846] - Socket
> connection established to localhost/127.0.0.1:2183, initiating session
> 2012-03-07 12:46:36,498 [myid:] - INFO
> [main-SendThread(localhost:2183):ClientCnxn$SendThread@1053] - Unable to
> read additional data from server sessionid 0x0, likely server has closed
> socket, closing socket connection and attempting reconnect
>
>
> Any help (or pointers) regarding above queries is highly appreciated.
>
>
> thanks,
> hrm
>
>
>
>
>
>
>
>
>
> --
> View this message in context: http://zookeeper-user.578899.n2.nabble.com/=
Setting-Zookeeper-server-cluster-on-single-host-tp7351244p7351244.html
> Sent from the zookeeper-user mailing list archive at Nabble.com<http://Na=
bble.com>.