zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Patrick Hunt <ph...@apache.org>
Subject Re: New Zookeeper Installation - connection refused
Date Thu, 12 Mar 2015 05:22:41 GMT
Consider zkconf? https://github.com/phunt/zkconf

Patrick

On Wed, Mar 11, 2015 at 10:21 PM, Michi Mutsuzaki <michi@cs.stanford.edu> wrote:
> The clientPort parameter needs to be different for each instance if
> you are running them on a same box. Something like this should work:
>
> https://paste.apache.org/n59c?action=download
>
> On Wed, Mar 11, 2015 at 8:02 AM, penguin <barry.barnett@wellsfargo.com> wrote:
>> Ok, I have setup my ZooKeeper ensemble on the same server.  I have 3 ZK instances
setup on the same physical machine, so no need to worry about whether multiple physical machines
can communicate from one to the other, right?
>> My setup on the Linux sever is as follows:
>>
>> Directory for instance 1: /home/zk1
>> Directory for instance 2: /home/zk2
>> Directory for instance 3: /home/zk3
>>
>> In my conf directory, I have my zoo.cfg file for each (replace zk1 with zk2, zk3
for the other instances):
>> # The number of milliseconds of each tick
>> tickTime=2000
>> # The number of ticks that the initial
>> # synchronization phase can take
>> initLimit=10
>> # The number of ticks that can pass between
>> # sending a request and getting an acknowledgement
>> syncLimit=5
>> # the directory where the snapshot is stored.
>> # do not use /tmp for storage, /tmp here is just
>> # example sakes.
>> dataDir=/home/zk1/zookeeper-3.4.6/data
>> dataLogDir=/home/zk1/zookeeper-3.4.6/data
>> # the port at which the clients will connect
>> clientPort=2181
>> # the maximum number of client connections.
>> maxClientCnxns=60
>> #
>> # Be sure to read the maintenance section of the
>> # administrator guide before turning on autopurge.
>> #
>> # http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
>> #
>> # The number of snapshots to retain in dataDir
>> #autopurge.snapRetainCount=3
>> # Purge task interval in hours
>> # Set to "0" to disable auto purge feature
>> #autopurge.purgeInterval=1
>> server.6=localhost:2888:3888
>> server.7=localhost:2889:3889
>> server.8=localhost:2890:3890
>>
>> the myid files are 6, 7 and 8 respectively for each instance.  Is there something
I'm missing because I'm still getting the connection refused warning, and the status of the
zk servers doesn't show anything but an error.
>>
>> Zookeeper.out shows:
>> 2015-03-11 08:59:22,238 [myid:6] - WARN  [WorkerSender[myid=6]:QuorumCnxManager@38
>> 2] - Cannot open channel to 8 at election address localhost/127.0.0.1:3890
>> java.net.ConnectException: Connection refused
>>         at java.net.PlainSocketImpl.socketConnect(Native Method)
>>         at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java
>> :339)
>>         at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketIm
>> pl.java:200)
>>         at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:1
>> 82)
>>         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
>>         at java.net.Socket.connect(Socket.java:579)
>>
>> zkServer.sh status shows:
>>
>> $ ./zkServer.sh status
>> JMX enabled by default
>> Using config: /opt/apps/zk1/zookeeper-3.4.6/bin/../conf/zoo.cfg
>> Error contacting service. It is probably not running.
>> $ ./zkServer.sh start
>> JMX enabled by default
>> Using config: /opt/apps/zk1/zookeeper-3.4.6/bin/../conf/zoo.cfg
>> Starting zookeeper ... already running as process 27102.
>>
>> Regards,
>>
>> Barry
>>
>> From: Flavio Junqueira-2 [via zookeeper-user] [mailto:ml-node+s578899n7580878h66@n2.nabble.com]
>> Sent: Tuesday, February 24, 2015 6:53 PM
>> To: Barnett, Barry F
>> Subject: Re: New Zookeeper Installation - connection refused
>>
>> It's hard to say what the problem is without more information. For leader election,
the servers will try to create a TCP connection with each other, so if you have tested separately
and your sample code worked for the same ports, the only thing I can think of is that there
is a problem with the server name resolution. Is it possible that the ZK servers aren't resolving
the names of your servers appropriately?
>>
>> Perhaps you could also use tcpdump or some similar tool to inspect the network traffic.
In fact, I think someone here has written a tool to inspect the leader election traffic and
display it. Was it Raul?
>>
>> -Flavio
>>
>>> On 24 Feb 2015, at 16:31, penguin <[hidden email]</user/SendEmail.jtp?type=node&node=7580878&i=0>>
wrote:
>>>
>>> Ok, I have tried various setups (5 and 3 ZK Server instances), and neither work,
both receiving the 'connection refused' error.
>>>
>>> I'm now at the 3 ZK ensemble, 1 ZK server instance per physical machine.
>>>
>>> Myid file is set to 1,2 and 3 respectively on the machines.
>>> Zoo.cfg has the following, and uses 2181 as its client port:
>>>
>>> server.1=servername:2888:3888
>>> server.2=servername:2888:3888
>>> server.3=servername:2888:3888
>>>
>>> Interestingly enough, when I start up the server1 instance, the zookeeper.out
file receives ongoing information on the connection refused to each server instance.  When
I start up the servers on machines 2 and 3, those zookeeper.out files do not receive any information.
 Is that normal?
>>>
>>> In any case, I've checked the ports by using nc -v hostname port, and connections
are successful.  So this is not a firewall issue or port block issue.  I've had my OSE's check
the servers as well to ensure no communications issue.
>>>
>>> *Note, port 2181 is an eForward port.  Is this supposed to be used by Zookeeper?
>>>
>>> Same errors over and over:
>>> 2015-02-24 09:38:00,601 [myid:1] - INFO  [QuorumPeer[myid=1]/0.0.0.0:2181:QuorumPeer@714]
- LOOKING
>>> 2015-02-24 09:38:00,603 [myid:1] - INFO  [QuorumPeer[myid=1]/0.0.0.0:2181:FastLeaderElection@815]
- New e
>>> lection. My id =  1, proposed zxid=0x0
>>> 2015-02-24 09:38:00,605 [myid:1] - INFO  [WorkerReceiver[myid=1]:FastLeaderElection@597]
- Notification:
>>> 1 (message format version), 1 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING
(n.state), 1 (n.sid), 0x0
>>> (n.peerEpoch) LOOKING (my state)
>>> 2015-02-24 09:38:00,608 [myid:1] - WARN  [WorkerSender[myid=1]:QuorumCnxManager@382]
- Cannot open channe
>>> l to 2 at election address servername:3888
>>> java.net.ConnectException: Connection refused
>>>        at java.net.PlainSocketImpl.socketConnect(Native Method)
>>>        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
>>>        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
>>>        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
>>>
>>> Regards,
>>>
>>> Barry
>>>
>>> From: Flavio Junqueira-2 [via zookeeper-user] [mailto:[hidden email]</user/SendEmail.jtp?type=node&node=7580878&i=1>
<mailto:[hidden email]</user/SendEmail.jtp?type=node&node=7580878&i=2>>]
>>> Sent: Wednesday, February 04, 2015 4:17 PM
>>> To: Barnett, Barry F
>>> Subject: Re: New Zookeeper Installation - connection refused
>>>
>>> That's because some client is trying to connect to the server, but the server
is looking (not following or leading) and the result of the nc run confirms it. I think you
still have a connectivity problem. There is something preventing the servers from connecting
to each other and it seems to be something related to your environment.
>>>
>>> -Flavio
>>>
>>>> On 04 Feb 2015, at 17:51, penguin <[hidden email]</user/SendEmail.jtp?type=node&node=7580844&i=0>>
wrote:
>>>>
>>>> I now made this a 3 server ensemble as opposed to a 5 server ensemble.  Now
getting different messages in my out logs:P
>>>>
>>>> 2015-02-04 11:48:29,851 [myid:1] - WARN  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@362]
- Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running
>>>> 2015-02-04 11:48:29,851 [myid:1] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@1007]
- Closed socket connection for client /xxxxxxxx:64089 (no session established for client)
>>>>
>>>> $ ./zkServer.sh status
>>>> JMX enabled by default
>>>> Using config: /opt/apps/current/bin/../conf/zoo.cfg
>>>> Error contacting service. It is probably not running.
>>>>
>>>>
>>>> $ echo mntr | nc localhost 2181
>>>> This ZooKeeper instance is not currently serving requests
>>>>
>>>>
>>>>
>>>> Regards,
>>>>
>>>> Barry
>>>>
>>>>
>>>> From: "Jürgen Wagner (DVT)" [via zookeeper-user] [mailto:[hidden email]</user/SendEmail.jtp?type=node&node=7580844&i=1>]
>>>> Sent: Wednesday, February 04, 2015 10:56 AM
>>>> To: Barnett, Barry F
>>>> Subject: Re: New Zookeeper Installation - connection refused
>>>>
>>>> Barry,
>>>> looks like you want to go to the machine where Zk is running and check
>>>> with "netstat -an" or some other tool on which port Zk is really
>>>> running... unless this is a NAT issue ;-)
>>>>
>>>> Cheers,
>>>> --Jürgen
>>>>
>>>> On 04.02.2015 16:51, penguin wrote:
>>>>> $ nc -v remotehostname 2181
>>>>> nc: connect to remotehostname port 2181 (tcp) failed: Connection refused
>>>>>
>>>>> Regards,
>>>>>
>>>>> Barry
>>>>>
>>>>
>>>>
>>>> ________________________________
>>>> If you reply to this email, your message will be added to the discussion
below:
>>>> http://zookeeper-user.578899.n2.nabble.com/New-Zookeeper-Installation-connection-refused-tp7580825p7580838.html
<http://zookeeper-user.578899.n2.nabble.com/New-Zookeeper-Installation-connection-refused-tp7580825p7580838.html>
>>>> To unsubscribe from New Zookeeper Installation - connection refused, click
here<
>>>> NAML<http://zookeeper-user.578899.n2.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> View this message in context: http://zookeeper-user.578899.n2.nabble.com/New-Zookeeper-Installation-connection-refused-tp7580825p7580840.html
>>>> Sent from the zookeeper-user mailing list archive at Nabble.com.
>>>
>>>
>>> ________________________________
>>> If you reply to this email, your message will be added to the discussion below:
>>> http://zookeeper-user.578899.n2.nabble.com/New-Zookeeper-Installation-connection-refused-tp7580825p7580844.html
<http://zookeeper-user.578899.n2.nabble.com/New-Zookeeper-Installation-connection-refused-tp7580825p7580844.html>
>>> To unsubscribe from New Zookeeper Installation - connection refused, click here<
href="" target="_top" rel="nofollow" link="external">
>>> NAML<http://zookeeper-user.578899.n2.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml
<http://zookeeper-user.578899.n2.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>>
>>>
>>>
>>>
>>>
>>> --
>>> View this message in context: http://zookeeper-user.578899.n2.nabble.com/New-Zookeeper-Installation-connection-refused-tp7580825p7580875.html
<http://zookeeper-user.578899.n2.nabble.com/New-Zookeeper-Installation-connection-refused-tp7580825p7580875.html>
>>> Sent from the zookeeper-user mailing list archive at Nabble.com <http://nabble.com/>.
>>
>>
>> ________________________________
>> If you reply to this email, your message will be added to the discussion below:
>> http://zookeeper-user.578899.n2.nabble.com/New-Zookeeper-Installation-connection-refused-tp7580825p7580878.html
>> To unsubscribe from New Zookeeper Installation - connection refused, click here<http://zookeeper-user.578899.n2.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=7580825&code=YmFycnkuYmFybmV0dEB3ZWxsc2ZhcmdvLmNvbXw3NTgwODI1fC0xNDYzMjU3NTMx>.
>> NAML<http://zookeeper-user.578899.n2.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>
>>
>>
>>
>> --
>> View this message in context: http://zookeeper-user.578899.n2.nabble.com/New-Zookeeper-Installation-connection-refused-tp7580825p7580904.html
>> Sent from the zookeeper-user mailing list archive at Nabble.com.

Mime
View raw message