zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From penguin <barry.barn...@wellsfargo.com>
Subject RE: New Zookeeper Installation - connection refused
Date Wed, 11 Mar 2015 15:02:23 GMT
Ok, I have setup my ZooKeeper ensemble on the same server.  I have 3 ZK instances setup on
the same physical machine, so no need to worry about whether multiple physical machines can
communicate from one to the other, right?
My setup on the Linux sever is as follows:

Directory for instance 1: /home/zk1
Directory for instance 2: /home/zk2
Directory for instance 3: /home/zk3

In my conf directory, I have my zoo.cfg file for each (replace zk1 with zk2, zk3 for the other
instances):
# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just
# example sakes.
dataDir=/home/zk1/zookeeper-3.4.6/data
dataLogDir=/home/zk1/zookeeper-3.4.6/data
# the port at which the clients will connect
clientPort=2181
# the maximum number of client connections.
maxClientCnxns=60
#
# Be sure to read the maintenance section of the
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1
server.6=localhost:2888:3888
server.7=localhost:2889:3889
server.8=localhost:2890:3890

the myid files are 6, 7 and 8 respectively for each instance.  Is there something I’m missing
because I’m still getting the connection refused warning, and the status of the zk servers
doesn’t show anything but an error.

Zookeeper.out shows:
2015-03-11 08:59:22,238 [myid:6] - WARN  [WorkerSender[myid=6]:QuorumCnxManager@38
2] - Cannot open channel to 8 at election address localhost/127.0.0.1:3890
java.net.ConnectException: Connection refused
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java
:339)
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketIm
pl.java:200)
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:1
82)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
        at java.net.Socket.connect(Socket.java:579)

zkServer.sh status shows:

$ ./zkServer.sh status
JMX enabled by default
Using config: /opt/apps/zk1/zookeeper-3.4.6/bin/../conf/zoo.cfg
Error contacting service. It is probably not running.
$ ./zkServer.sh start
JMX enabled by default
Using config: /opt/apps/zk1/zookeeper-3.4.6/bin/../conf/zoo.cfg
Starting zookeeper ... already running as process 27102.

Regards,

Barry

From: Flavio Junqueira-2 [via zookeeper-user] [mailto:ml-node+s578899n7580878h66@n2.nabble.com]
Sent: Tuesday, February 24, 2015 6:53 PM
To: Barnett, Barry F
Subject: Re: New Zookeeper Installation - connection refused

It's hard to say what the problem is without more information. For leader election, the servers
will try to create a TCP connection with each other, so if you have tested separately and
your sample code worked for the same ports, the only thing I can think of is that there is
a problem with the server name resolution. Is it possible that the ZK servers aren't resolving
the names of your servers appropriately?

Perhaps you could also use tcpdump or some similar tool to inspect the network traffic. In
fact, I think someone here has written a tool to inspect the leader election traffic and display
it. Was it Raul?

-Flavio

> On 24 Feb 2015, at 16:31, penguin <[hidden email]</user/SendEmail.jtp?type=node&node=7580878&i=0>>
wrote:
>
> Ok, I have tried various setups (5 and 3 ZK Server instances), and neither work, both
receiving the ‘connection refused’ error.
>
> I’m now at the 3 ZK ensemble, 1 ZK server instance per physical machine.
>
> Myid file is set to 1,2 and 3 respectively on the machines.
> Zoo.cfg has the following, and uses 2181 as its client port:
>
> server.1=servername:2888:3888
> server.2=servername:2888:3888
> server.3=servername:2888:3888
>
> Interestingly enough, when I start up the server1 instance, the zookeeper.out file receives
ongoing information on the connection refused to each server instance.  When I start up the
servers on machines 2 and 3, those zookeeper.out files do not receive any information.  Is
that normal?
>
> In any case, I’ve checked the ports by using nc –v hostname port, and connections
are successful.  So this is not a firewall issue or port block issue.  I’ve had my OSE’s
check the servers as well to ensure no communications issue.
>
> *Note, port 2181 is an eForward port.  Is this supposed to be used by Zookeeper?
>
> Same errors over and over:
> 2015-02-24 09:38:00,601 [myid:1] - INFO  [QuorumPeer[myid=1]/0.0.0.0:2181:QuorumPeer@714]
- LOOKING
> 2015-02-24 09:38:00,603 [myid:1] - INFO  [QuorumPeer[myid=1]/0.0.0.0:2181:FastLeaderElection@815]
- New e
> lection. My id =  1, proposed zxid=0x0
> 2015-02-24 09:38:00,605 [myid:1] - INFO  [WorkerReceiver[myid=1]:FastLeaderElection@597]
- Notification:
> 1 (message format version), 1 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state),
1 (n.sid), 0x0
> (n.peerEpoch) LOOKING (my state)
> 2015-02-24 09:38:00,608 [myid:1] - WARN  [WorkerSender[myid=1]:QuorumCnxManager@382]
- Cannot open channe
> l to 2 at election address servername:3888
> java.net.ConnectException: Connection refused
>        at java.net.PlainSocketImpl.socketConnect(Native Method)
>        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
>        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
>        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
>
> Regards,
>
> Barry
>
> From: Flavio Junqueira-2 [via zookeeper-user] [mailto:[hidden email]</user/SendEmail.jtp?type=node&node=7580878&i=1>
<mailto:[hidden email]</user/SendEmail.jtp?type=node&node=7580878&i=2>>]
> Sent: Wednesday, February 04, 2015 4:17 PM
> To: Barnett, Barry F
> Subject: Re: New Zookeeper Installation - connection refused
>
> That's because some client is trying to connect to the server, but the server is looking
(not following or leading) and the result of the nc run confirms it. I think you still have
a connectivity problem. There is something preventing the servers from connecting to each
other and it seems to be something related to your environment.
>
> -Flavio
>
>> On 04 Feb 2015, at 17:51, penguin <[hidden email]</user/SendEmail.jtp?type=node&node=7580844&i=0>>
wrote:
>>
>> I now made this a 3 server ensemble as opposed to a 5 server ensemble.  Now getting
different messages in my out logs:P
>>
>> 2015-02-04 11:48:29,851 [myid:1] - WARN  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@362]
- Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running
>> 2015-02-04 11:48:29,851 [myid:1] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@1007]
- Closed socket connection for client /xxxxxxxx:64089 (no session established for client)
>>
>> $ ./zkServer.sh status
>> JMX enabled by default
>> Using config: /opt/apps/current/bin/../conf/zoo.cfg
>> Error contacting service. It is probably not running.
>>
>>
>> $ echo mntr | nc localhost 2181
>> This ZooKeeper instance is not currently serving requests
>>
>>
>>
>> Regards,
>>
>> Barry
>>
>>
>> From: "Jürgen Wagner (DVT)" [via zookeeper-user] [mailto:[hidden email]</user/SendEmail.jtp?type=node&node=7580844&i=1>]
>> Sent: Wednesday, February 04, 2015 10:56 AM
>> To: Barnett, Barry F
>> Subject: Re: New Zookeeper Installation - connection refused
>>
>> Barry,
>> looks like you want to go to the machine where Zk is running and check
>> with "netstat -an" or some other tool on which port Zk is really
>> running... unless this is a NAT issue ;-)
>>
>> Cheers,
>> --Jürgen
>>
>> On 04.02.2015 16:51, penguin wrote:
>>> $ nc -v remotehostname 2181
>>> nc: connect to remotehostname port 2181 (tcp) failed: Connection refused
>>>
>>> Regards,
>>>
>>> Barry
>>>
>>
>>
>> ________________________________
>> If you reply to this email, your message will be added to the discussion below:
>> http://zookeeper-user.578899.n2.nabble.com/New-Zookeeper-Installation-connection-refused-tp7580825p7580838.html
<http://zookeeper-user.578899.n2.nabble.com/New-Zookeeper-Installation-connection-refused-tp7580825p7580838.html>
>> To unsubscribe from New Zookeeper Installation - connection refused, click here<
>> NAML<http://zookeeper-user.578899.n2.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>
>>
>>
>>
>> --
>> View this message in context: http://zookeeper-user.578899.n2.nabble.com/New-Zookeeper-Installation-connection-refused-tp7580825p7580840.html
>> Sent from the zookeeper-user mailing list archive at Nabble.com.
>
>
> ________________________________
> If you reply to this email, your message will be added to the discussion below:
> http://zookeeper-user.578899.n2.nabble.com/New-Zookeeper-Installation-connection-refused-tp7580825p7580844.html
<http://zookeeper-user.578899.n2.nabble.com/New-Zookeeper-Installation-connection-refused-tp7580825p7580844.html>
> To unsubscribe from New Zookeeper Installation - connection refused, click here< href=""
target="_top" rel="nofollow" link="external">
> NAML<http://zookeeper-user.578899.n2.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml
<http://zookeeper-user.578899.n2.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>>
>
>
>
>
> --
> View this message in context: http://zookeeper-user.578899.n2.nabble.com/New-Zookeeper-Installation-connection-refused-tp7580825p7580875.html
<http://zookeeper-user.578899.n2.nabble.com/New-Zookeeper-Installation-connection-refused-tp7580825p7580875.html>
> Sent from the zookeeper-user mailing list archive at Nabble.com <http://nabble.com/>.


________________________________
If you reply to this email, your message will be added to the discussion below:
http://zookeeper-user.578899.n2.nabble.com/New-Zookeeper-Installation-connection-refused-tp7580825p7580878.html
To unsubscribe from New Zookeeper Installation - connection refused, click here<http://zookeeper-user.578899.n2.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=7580825&code=YmFycnkuYmFybmV0dEB3ZWxsc2ZhcmdvLmNvbXw3NTgwODI1fC0xNDYzMjU3NTMx>.
NAML<http://zookeeper-user.578899.n2.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>




--
View this message in context: http://zookeeper-user.578899.n2.nabble.com/New-Zookeeper-Installation-connection-refused-tp7580825p7580904.html
Sent from the zookeeper-user mailing list archive at Nabble.com.
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message