hadoop-zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Patrick Hunt <ph...@apache.org>
Subject Re: Starting Zookeeper on Amazon EC2
Date Thu, 10 Dec 2009 19:16:19 GMT
Would you be willing to create a "zk on ec2" page on the wiki?
http://wiki.apache.org/hadoop/ZooKeeper/ZooKeeperOnEC2

Not sure on the exactly content -- basically outline what you did to get 
ZK running. Something you think would be helpful to other users trying 
to deploy ZK on ec2 for the first time. :-)

Patrick


Something Something wrote:
> Sorry, ignore this.  My bad.
> 
> These messages disappeared when I started ZooKeeper on other two instances.
> I tested this by killing 1 or 2 processes at a time.  These messages start
> popping up when a process gets killed - implying that these instances are
> indeed talking to each other.
> 
> Thanks.
> 
> 
> On Wed, Dec 9, 2009 at 10:22 PM, Something Something <
> mailinglists19@gmail.com> wrote:
> 
>> I keep getting "Connection refused" on EC2.  Seems like it's a security
>> issue on Amazon's EC2 environment.  I have tried specifying machine names in
>> following 4 ways, but nothing seems to help:
>>
>> server.1=domU-12-31-38-01-B8-47.compute-1.internal:2888:3888
>> server.1=domU-12-31-38-01-B8-47:2888:3888
>> server.1=ec2-72-44-42-160.compute-1.amazonaws.com:2888:3888
>> server.1=10.253.191.181:2888:3888
>>
>> Please help.  Thanks.
>>
>> Here's the message:
>>
>> ---------------------------------------------------------------------------------------------------------------------
>> root@domU-12-31-38-01-B8-47:/ebs1/zookeeper-3.2.1/bin# 2009-12-10
>> 06:03:42,577 - WARN  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:QuorumCnxManager@336]
>> - Cannot open channel to 2 at election address
>> domU-12-31-38-01-B8-A3.compute-1.internal/10.253.191.81:3888
>> java.net.ConnectException: Connection refused
>>     at sun.nio.ch.Net.connect(Native Method)
>>     at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:507)
>>     at java.nio.channels.SocketChannel.open(SocketChannel.java:146)
>>     at
>> org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:323)
>>     at
>> org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:356)
>>     at
>> org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:603)
>>     at
>> org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:488)
>> 2009-12-10 06:03:42,578 - WARN
>> [QuorumPeer:/0:0:0:0:0:0:0:0:2181:QuorumCnxManager@336] - Cannot open
>> channel to 3 at election address domU-12-31-38-00-94-A6.compute-1.internal/
>> 10.252.155.84:3888
>> java.net.ConnectException: Connection refused
>>     at sun.nio.ch.Net.connect(Native Method)
>>     at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:507)
>>     at java.nio.channels.SocketChannel.open(SocketChannel.java:146)
>>     at
>> org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:323)
>>     at
>> org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:356)
>>     at
>> org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:603)
>>     at
>> org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:488)
>> 2009-12-10 06:03:42,578 - INFO
>> [QuorumPeer:/0:0:0:0:0:0:0:0:2181:FastLeaderElection@612] - Notification
>> time out: 60000
>>
>>
>>
>> On Wed, Dec 9, 2009 at 11:08 AM, Henry Robinson <henry@cloudera.com>wrote:
>>
>>> Nearly! 1+2 are correct, but you also need to start ZooKeeper on all three
>>> instances with bin/zkServer.sh start.
>>>
>>> Henry
>>>
>>> On Wed, Dec 9, 2009 at 11:00 AM, Something Something <
>>> mailinglists19@gmail.com> wrote:
>>>
>>>> Now that I have your attention..next question... :)
>>>>
>>>> Now I would like to start a Zookeeper Quorum on 3 EC Instances.  Read
>>> the
>>>> doc regarding... "Running Replicated ZooKeeper".  It says "all servers
>>> in
>>>> the quorum should have the same configuration file"..  Does this mean...
>>> I
>>>> should..
>>>>
>>>> 1)  Download & Install ZooKeeper on all 3 instances (at the same
>>> location.)
>>>> 2)  Save the same zoo.cfg in /conf for all 3 instances.
>>>> 3)  On one instance (Master?), run...
>>>>
>>>> bin/zkServer.sh start
>>>>
>>>> Would that start ZooKeeper on all 3 instances?  Thanks for the help.
>>>>
>>>>
>>>> On Wed, Dec 9, 2009 at 10:24 AM, Something Something <
>>>> mailinglists19@gmail.com> wrote:
>>>>
>>>>> Switched to 3.2.1.  Much better.  Got a command prompt.  Thank you
>>> both.
>>>>>
>>>>> On Wed, Dec 9, 2009 at 10:09 AM, Henry Robinson <henry@cloudera.com
>>>>> wrote:
>>>>>
>>>>>> The 3.2.1 command line is a lot nicer (has an actual prompt, tab
>>>>>> auto-completion, shows your connection status etc) - if you can
>>> upgrade
>>>> to
>>>>>> 3.2.1 which is a good deal more modern, I would recommend it. If
I
>>>> recall
>>>>>> correctly, there was no prompt in 3.1.1...
>>>>>>
>>>>>> Henry
>>>>>>
>>>>>> On Wed, Dec 9, 2009 at 9:36 AM, Something Something <
>>>>>> mailinglists19@gmail.com> wrote:
>>>>>>
>>>>>>> Without -server made some progress, but don't see a command prompt.
>>>>>>> Shouldn't I see one?
>>>>>>>
>>>>>>> This is what I see:
>>>>>>> 2009-12-09 17:27:56,709 - INFO  [main:ZooKeeper@341] - Initiating
>>>>>> client
>>>>>>> connection, host=127.0.0.1:2181 sessionTimeout=5000
>>>>>>> watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@32fb4f
>>>>>>> 2009-12-09 17:27:56,710 - INFO  [main:ClientCnxn@91] -
>>>>>>> zookeeper.disableAutoWatchReset is false
>>>>>>> 2009-12-09 17:27:56,792 - INFO
>>>>>>  [main-SendThread:ClientCnxn$SendThread@800
>>>>>>> ]
>>>>>>> - Attempting connection to server /127.0.0.1:2181
>>>>>>> 2009-12-09 17:27:56,802 - INFO
>>>>>>  [main-SendThread:ClientCnxn$SendThread@716
>>>>>>> ]
>>>>>>> - Priming connection to java.nio.channels.SocketChannel[connected
>>>>>> local=/
>>>>>>> 127.0.0.1:49619 remote=/127.0.0.1:2181]
>>>>>>> 2009-12-09 17:27:56,806 - INFO
>>>>>>  [main-SendThread:ClientCnxn$SendThread@868
>>>>>>> ]
>>>>>>> - Server connection successful
>>>>>>> WatchedEvent: Server state change. New state: SyncConnected
>>>>>>>
>>>>>>>
>>>>>>> Should I just use 3.2.1 version?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Dec 9, 2009 at 9:20 AM, Mahadev Konar <
>>> mahadev@yahoo-inc.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>>  Can you try this?
>>>>>>>>
>>>>>>>> bin/zkCli.sh 127.0.0.1:2181
>>>>>>>>
>>>>>>>> The -server command was added later as far as I remember.
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>> mahadev
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On 12/9/09 9:05 AM, "Something Something" <
>>> mailinglists19@gmail.com
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> I am trying to start ZooKeeper on an EC2 instance.  Here's
what
>>> I
>>>>>> did:
>>>>>>>>> 1)  Downloaded & Unpacked ZooKeeper 3.1.1 on EC2
instance.
>>>>>>>>> 2)  cp /conf/zoo_sample.cfg /conf/zoo.cfg
>>>>>>>>> 3)  Changed the dataDir path to point to my EBS volume.
>>>>>>>>> 4)  In one command window, ran /bin/zkServer.sh start
>>>>>>>>> (The last message I see is... "Snapshotting: 0)
>>>>>>>>>
>>>>>>>>> 5)  Opened another command window, and ran jps
>>>>>>>>> (This shows a new process called, QuorumPeerMain.  That's
the
>>> only
>>>>>> one
>>>>>>> I
>>>>>>>>> see.)
>>>>>>>>>
>>>>>>>>> 6)  As per documentation, tried
>>>>>>>>>
>>>>>>>>> bin/zkCli.sh -server 127.0.0.1:2181
>>>>>>>>>
>>>>>>>>> (This gives me IOException: USAGE)
>>>>>>>>>
>>>>>>>>> 7) So I ran:
>>>>>>>>>
>>>>>>>>> bin/zkCli.sh -server 127.0.0.1:2181 ls
>>>>>>>>>
>>>>>>>>> Got UnknownHostException: -server
>>>>>>>>>
>>>>>>>>> 8)  So I tried various ways of specifying IP address
in EC2,
>>> such
>>>>>> as:
>>>>>>>>> 10.xx.xx.xx
>>>>>>>>> ec2-xx-xx-xx-xxx.compute-1.amazonaws.com
>>>>>>>>> domU-12-31-xx-xx-xx-xx.compute-1.internal
>>>>>>>>> domU-12-31-xx-xx-xx-xx
>>>>>>>>>
>>>>>>>>> None of them worked.  Keep getting UnknownHostException.
>>>>>>>>>
>>>>>>>>> What am I doing wrong.  Please help.  Thanks.
>>>>>>>>
>>>>>
>>
> 

Mime
View raw message