zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michi Mutsuzaki <mi...@cs.stanford.edu>
Subject Re: New Zookeeper Installation - connection refused
Date Thu, 12 Mar 2015 05:21:07 GMT
The clientPort parameter needs to be different for each instance if
you are running them on a same box. Something like this should work:

https://paste.apache.org/n59c?action=download

On Wed, Mar 11, 2015 at 8:02 AM, penguin <barry.barnett@wellsfargo.com> wrote:
> Ok, I have setup my ZooKeeper ensemble on the same server.  I have 3 ZK instances setup
on the same physical machine, so no need to worry about whether multiple physical machines
can communicate from one to the other, right?
> My setup on the Linux sever is as follows:
>
> Directory for instance 1: /home/zk1
> Directory for instance 2: /home/zk2
> Directory for instance 3: /home/zk3
>
> In my conf directory, I have my zoo.cfg file for each (replace zk1 with zk2, zk3 for
the other instances):
> # The number of milliseconds of each tick
> tickTime=2000
> # The number of ticks that the initial
> # synchronization phase can take
> initLimit=10
> # The number of ticks that can pass between
> # sending a request and getting an acknowledgement
> syncLimit=5
> # the directory where the snapshot is stored.
> # do not use /tmp for storage, /tmp here is just
> # example sakes.
> dataDir=/home/zk1/zookeeper-3.4.6/data
> dataLogDir=/home/zk1/zookeeper-3.4.6/data
> # the port at which the clients will connect
> clientPort=2181
> # the maximum number of client connections.
> maxClientCnxns=60
> #
> # Be sure to read the maintenance section of the
> # administrator guide before turning on autopurge.
> #
> # http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
> #
> # The number of snapshots to retain in dataDir
> #autopurge.snapRetainCount=3
> # Purge task interval in hours
> # Set to "0" to disable auto purge feature
> #autopurge.purgeInterval=1
> server.6=localhost:2888:3888
> server.7=localhost:2889:3889
> server.8=localhost:2890:3890
>
> the myid files are 6, 7 and 8 respectively for each instance.  Is there something I'm
missing because I'm still getting the connection refused warning, and the status of the zk
servers doesn't show anything but an error.
>
> Zookeeper.out shows:
> 2015-03-11 08:59:22,238 [myid:6] - WARN  [WorkerSender[myid=6]:QuorumCnxManager@38
> 2] - Cannot open channel to 8 at election address localhost/127.0.0.1:3890
> java.net.ConnectException: Connection refused
>         at java.net.PlainSocketImpl.socketConnect(Native Method)
>         at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java
> :339)
>         at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketIm
> pl.java:200)
>         at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:1
> 82)
>         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
>         at java.net.Socket.connect(Socket.java:579)
>
> zkServer.sh status shows:
>
> $ ./zkServer.sh status
> JMX enabled by default
> Using config: /opt/apps/zk1/zookeeper-3.4.6/bin/../conf/zoo.cfg
> Error contacting service. It is probably not running.
> $ ./zkServer.sh start
> JMX enabled by default
> Using config: /opt/apps/zk1/zookeeper-3.4.6/bin/../conf/zoo.cfg
> Starting zookeeper ... already running as process 27102.
>
> Regards,
>
> Barry
>
> From: Flavio Junqueira-2 [via zookeeper-user] [mailto:ml-node+s578899n7580878h66@n2.nabble.com]
> Sent: Tuesday, February 24, 2015 6:53 PM
> To: Barnett, Barry F
> Subject: Re: New Zookeeper Installation - connection refused
>
> It's hard to say what the problem is without more information. For leader election, the
servers will try to create a TCP connection with each other, so if you have tested separately
and your sample code worked for the same ports, the only thing I can think of is that there
is a problem with the server name resolution. Is it possible that the ZK servers aren't resolving
the names of your servers appropriately?
>
> Perhaps you could also use tcpdump or some similar tool to inspect the network traffic.
In fact, I think someone here has written a tool to inspect the leader election traffic and
display it. Was it Raul?
>
> -Flavio
>
>> On 24 Feb 2015, at 16:31, penguin <[hidden email]</user/SendEmail.jtp?type=node&node=7580878&i=0>>
wrote:
>>
>> Ok, I have tried various setups (5 and 3 ZK Server instances), and neither work,
both receiving the 'connection refused' error.
>>
>> I'm now at the 3 ZK ensemble, 1 ZK server instance per physical machine.
>>
>> Myid file is set to 1,2 and 3 respectively on the machines.
>> Zoo.cfg has the following, and uses 2181 as its client port:
>>
>> server.1=servername:2888:3888
>> server.2=servername:2888:3888
>> server.3=servername:2888:3888
>>
>> Interestingly enough, when I start up the server1 instance, the zookeeper.out file
receives ongoing information on the connection refused to each server instance.  When I start
up the servers on machines 2 and 3, those zookeeper.out files do not receive any information.
 Is that normal?
>>
>> In any case, I've checked the ports by using nc -v hostname port, and connections
are successful.  So this is not a firewall issue or port block issue.  I've had my OSE's check
the servers as well to ensure no communications issue.
>>
>> *Note, port 2181 is an eForward port.  Is this supposed to be used by Zookeeper?
>>
>> Same errors over and over:
>> 2015-02-24 09:38:00,601 [myid:1] - INFO  [QuorumPeer[myid=1]/0.0.0.0:2181:QuorumPeer@714]
- LOOKING
>> 2015-02-24 09:38:00,603 [myid:1] - INFO  [QuorumPeer[myid=1]/0.0.0.0:2181:FastLeaderElection@815]
- New e
>> lection. My id =  1, proposed zxid=0x0
>> 2015-02-24 09:38:00,605 [myid:1] - INFO  [WorkerReceiver[myid=1]:FastLeaderElection@597]
- Notification:
>> 1 (message format version), 1 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state),
1 (n.sid), 0x0
>> (n.peerEpoch) LOOKING (my state)
>> 2015-02-24 09:38:00,608 [myid:1] - WARN  [WorkerSender[myid=1]:QuorumCnxManager@382]
- Cannot open channe
>> l to 2 at election address servername:3888
>> java.net.ConnectException: Connection refused
>>        at java.net.PlainSocketImpl.socketConnect(Native Method)
>>        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
>>        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
>>        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
>>
>> Regards,
>>
>> Barry
>>
>> From: Flavio Junqueira-2 [via zookeeper-user] [mailto:[hidden email]</user/SendEmail.jtp?type=node&node=7580878&i=1>
<mailto:[hidden email]</user/SendEmail.jtp?type=node&node=7580878&i=2>>]
>> Sent: Wednesday, February 04, 2015 4:17 PM
>> To: Barnett, Barry F
>> Subject: Re: New Zookeeper Installation - connection refused
>>
>> That's because some client is trying to connect to the server, but the server is
looking (not following or leading) and the result of the nc run confirms it. I think you still
have a connectivity problem. There is something preventing the servers from connecting to
each other and it seems to be something related to your environment.
>>
>> -Flavio
>>
>>> On 04 Feb 2015, at 17:51, penguin <[hidden email]</user/SendEmail.jtp?type=node&node=7580844&i=0>>
wrote:
>>>
>>> I now made this a 3 server ensemble as opposed to a 5 server ensemble.  Now getting
different messages in my out logs:P
>>>
>>> 2015-02-04 11:48:29,851 [myid:1] - WARN  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@362]
- Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running
>>> 2015-02-04 11:48:29,851 [myid:1] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@1007]
- Closed socket connection for client /xxxxxxxx:64089 (no session established for client)
>>>
>>> $ ./zkServer.sh status
>>> JMX enabled by default
>>> Using config: /opt/apps/current/bin/../conf/zoo.cfg
>>> Error contacting service. It is probably not running.
>>>
>>>
>>> $ echo mntr | nc localhost 2181
>>> This ZooKeeper instance is not currently serving requests
>>>
>>>
>>>
>>> Regards,
>>>
>>> Barry
>>>
>>>
>>> From: "Jürgen Wagner (DVT)" [via zookeeper-user] [mailto:[hidden email]</user/SendEmail.jtp?type=node&node=7580844&i=1>]
>>> Sent: Wednesday, February 04, 2015 10:56 AM
>>> To: Barnett, Barry F
>>> Subject: Re: New Zookeeper Installation - connection refused
>>>
>>> Barry,
>>> looks like you want to go to the machine where Zk is running and check
>>> with "netstat -an" or some other tool on which port Zk is really
>>> running... unless this is a NAT issue ;-)
>>>
>>> Cheers,
>>> --Jürgen
>>>
>>> On 04.02.2015 16:51, penguin wrote:
>>>> $ nc -v remotehostname 2181
>>>> nc: connect to remotehostname port 2181 (tcp) failed: Connection refused
>>>>
>>>> Regards,
>>>>
>>>> Barry
>>>>
>>>
>>>
>>> ________________________________
>>> If you reply to this email, your message will be added to the discussion below:
>>> http://zookeeper-user.578899.n2.nabble.com/New-Zookeeper-Installation-connection-refused-tp7580825p7580838.html
<http://zookeeper-user.578899.n2.nabble.com/New-Zookeeper-Installation-connection-refused-tp7580825p7580838.html>
>>> To unsubscribe from New Zookeeper Installation - connection refused, click here<
>>> NAML<http://zookeeper-user.578899.n2.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>>
>>>
>>>
>>>
>>> --
>>> View this message in context: http://zookeeper-user.578899.n2.nabble.com/New-Zookeeper-Installation-connection-refused-tp7580825p7580840.html
>>> Sent from the zookeeper-user mailing list archive at Nabble.com.
>>
>>
>> ________________________________
>> If you reply to this email, your message will be added to the discussion below:
>> http://zookeeper-user.578899.n2.nabble.com/New-Zookeeper-Installation-connection-refused-tp7580825p7580844.html
<http://zookeeper-user.578899.n2.nabble.com/New-Zookeeper-Installation-connection-refused-tp7580825p7580844.html>
>> To unsubscribe from New Zookeeper Installation - connection refused, click here<
href="" target="_top" rel="nofollow" link="external">
>> NAML<http://zookeeper-user.578899.n2.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml
<http://zookeeper-user.578899.n2.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>>
>>
>>
>>
>>
>> --
>> View this message in context: http://zookeeper-user.578899.n2.nabble.com/New-Zookeeper-Installation-connection-refused-tp7580825p7580875.html
<http://zookeeper-user.578899.n2.nabble.com/New-Zookeeper-Installation-connection-refused-tp7580825p7580875.html>
>> Sent from the zookeeper-user mailing list archive at Nabble.com <http://nabble.com/>.
>
>
> ________________________________
> If you reply to this email, your message will be added to the discussion below:
> http://zookeeper-user.578899.n2.nabble.com/New-Zookeeper-Installation-connection-refused-tp7580825p7580878.html
> To unsubscribe from New Zookeeper Installation - connection refused, click here<http://zookeeper-user.578899.n2.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=7580825&code=YmFycnkuYmFybmV0dEB3ZWxsc2ZhcmdvLmNvbXw3NTgwODI1fC0xNDYzMjU3NTMx>.
> NAML<http://zookeeper-user.578899.n2.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
>
>
>
> --
> View this message in context: http://zookeeper-user.578899.n2.nabble.com/New-Zookeeper-Installation-connection-refused-tp7580825p7580904.html
> Sent from the zookeeper-user mailing list archive at Nabble.com.

Mime
View raw message