hadoop-zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Patrick Hunt <ph...@apache.org>
Subject Re: Starting Zookeeper on Amazon EC2
Date Thu, 10 Dec 2009 21:20:50 GMT
Awesome, thanks!

Patrick

Something Something wrote:
> Sure.  I will add my notes to this WIKI page and we can fix them later as
> necessary.
> 
> On Thu, Dec 10, 2009 at 11:16 AM, Patrick Hunt <phunt@apache.org> wrote:
> 
>> Would you be willing to create a "zk on ec2" page on the wiki?
>> http://wiki.apache.org/hadoop/ZooKeeper/ZooKeeperOnEC2
>>
>> Not sure on the exactly content -- basically outline what you did to get ZK
>> running. Something you think would be helpful to other users trying to
>> deploy ZK on ec2 for the first time. :-)
>>
>> Patrick
>>
>>
>>
>> Something Something wrote:
>>
>>> Sorry, ignore this.  My bad.
>>>
>>> These messages disappeared when I started ZooKeeper on other two
>>> instances.
>>> I tested this by killing 1 or 2 processes at a time.  These messages start
>>> popping up when a process gets killed - implying that these instances are
>>> indeed talking to each other.
>>>
>>> Thanks.
>>>
>>>
>>> On Wed, Dec 9, 2009 at 10:22 PM, Something Something <
>>> mailinglists19@gmail.com> wrote:
>>>
>>>  I keep getting "Connection refused" on EC2.  Seems like it's a security
>>>> issue on Amazon's EC2 environment.  I have tried specifying machine names
>>>> in
>>>> following 4 ways, but nothing seems to help:
>>>>
>>>> server.1=domU-12-31-38-01-B8-47.compute-1.internal:2888:3888
>>>> server.1=domU-12-31-38-01-B8-47:2888:3888
>>>> server.1=ec2-72-44-42-160.compute-1.amazonaws.com:2888:3888
>>>> server.1=10.253.191.181:2888:3888
>>>>
>>>> Please help.  Thanks.
>>>>
>>>> Here's the message:
>>>>
>>>>
>>>> ---------------------------------------------------------------------------------------------------------------------
>>>> root@domU-12-31-38-01-B8-47:/ebs1/zookeeper-3.2.1/bin# 2009-12-10
>>>> 06:03:42,577 - WARN
>>>>  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:QuorumCnxManager@336]
>>>> - Cannot open channel to 2 at election address
>>>> domU-12-31-38-01-B8-A3.compute-1.internal/10.253.191.81:3888
>>>> java.net.ConnectException: Connection refused
>>>>    at sun.nio.ch.Net.connect(Native Method)
>>>>    at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:507)
>>>>    at java.nio.channels.SocketChannel.open(SocketChannel.java:146)
>>>>    at
>>>>
>>>> org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:323)
>>>>    at
>>>>
>>>> org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:356)
>>>>    at
>>>>
>>>> org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:603)
>>>>    at
>>>> org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:488)
>>>> 2009-12-10 06:03:42,578 - WARN
>>>> [QuorumPeer:/0:0:0:0:0:0:0:0:2181:QuorumCnxManager@336] - Cannot open
>>>> channel to 3 at election address
>>>> domU-12-31-38-00-94-A6.compute-1.internal/
>>>> 10.252.155.84:3888
>>>> java.net.ConnectException: Connection refused
>>>>    at sun.nio.ch.Net.connect(Native Method)
>>>>    at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:507)
>>>>    at java.nio.channels.SocketChannel.open(SocketChannel.java:146)
>>>>    at
>>>>
>>>> org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:323)
>>>>    at
>>>>
>>>> org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:356)
>>>>    at
>>>>
>>>> org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:603)
>>>>    at
>>>> org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:488)
>>>> 2009-12-10 06:03:42,578 - INFO
>>>> [QuorumPeer:/0:0:0:0:0:0:0:0:2181:FastLeaderElection@612] - Notification
>>>> time out: 60000
>>>>
>>>>
>>>>
>>>> On Wed, Dec 9, 2009 at 11:08 AM, Henry Robinson <henry@cloudera.com
>>>>> wrote:
>>>>  Nearly! 1+2 are correct, but you also need to start ZooKeeper on all
>>>>> three
>>>>> instances with bin/zkServer.sh start.
>>>>>
>>>>> Henry
>>>>>
>>>>> On Wed, Dec 9, 2009 at 11:00 AM, Something Something <
>>>>> mailinglists19@gmail.com> wrote:
>>>>>
>>>>>  Now that I have your attention..next question... :)
>>>>>> Now I would like to start a Zookeeper Quorum on 3 EC Instances. 
Read
>>>>>>
>>>>> the
>>>>>
>>>>>> doc regarding... "Running Replicated ZooKeeper".  It says "all servers
>>>>>>
>>>>> in
>>>>>
>>>>>> the quorum should have the same configuration file"..  Does this
>>>>>> mean...
>>>>>>
>>>>> I
>>>>>
>>>>>> should..
>>>>>>
>>>>>> 1)  Download & Install ZooKeeper on all 3 instances (at the same
>>>>>>
>>>>> location.)
>>>>>
>>>>>> 2)  Save the same zoo.cfg in /conf for all 3 instances.
>>>>>> 3)  On one instance (Master?), run...
>>>>>>
>>>>>> bin/zkServer.sh start
>>>>>>
>>>>>> Would that start ZooKeeper on all 3 instances?  Thanks for the help.
>>>>>>
>>>>>>
>>>>>> On Wed, Dec 9, 2009 at 10:24 AM, Something Something <
>>>>>> mailinglists19@gmail.com> wrote:
>>>>>>
>>>>>>  Switched to 3.2.1.  Much better.  Got a command prompt.  Thank you
>>>>>> both.
>>>>>>> On Wed, Dec 9, 2009 at 10:09 AM, Henry Robinson <henry@cloudera.com
>>>>>>> wrote:
>>>>>>>
>>>>>>>  The 3.2.1 command line is a lot nicer (has an actual prompt,
tab
>>>>>>>> auto-completion, shows your connection status etc) - if you
can
>>>>>>>>
>>>>>>> upgrade
>>>>>> to
>>>>>>
>>>>>>> 3.2.1 which is a good deal more modern, I would recommend it.
If I
>>>>>>> recall
>>>>>>> correctly, there was no prompt in 3.1.1...
>>>>>>>> Henry
>>>>>>>>
>>>>>>>> On Wed, Dec 9, 2009 at 9:36 AM, Something Something <
>>>>>>>> mailinglists19@gmail.com> wrote:
>>>>>>>>
>>>>>>>>  Without -server made some progress, but don't see a command
prompt.
>>>>>>>>> Shouldn't I see one?
>>>>>>>>>
>>>>>>>>> This is what I see:
>>>>>>>>> 2009-12-09 17:27:56,709 - INFO  [main:ZooKeeper@341]
- Initiating
>>>>>>>>>
>>>>>>>> client
>>>>>>>>
>>>>>>>>> connection, host=127.0.0.1:2181 sessionTimeout=5000
>>>>>>>>> watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@32fb4f
>>>>>>>>> 2009-12-09 17:27:56,710 - INFO  [main:ClientCnxn@91]
-
>>>>>>>>> zookeeper.disableAutoWatchReset is false
>>>>>>>>> 2009-12-09 17:27:56,792 - INFO
>>>>>>>>>
>>>>>>>>  [main-SendThread:ClientCnxn$SendThread@800
>>>>>>>>
>>>>>>>>> ]
>>>>>>>>> - Attempting connection to server /127.0.0.1:2181
>>>>>>>>> 2009-12-09 17:27:56,802 - INFO
>>>>>>>>>
>>>>>>>>  [main-SendThread:ClientCnxn$SendThread@716
>>>>>>>>
>>>>>>>>> ]
>>>>>>>>> - Priming connection to java.nio.channels.SocketChannel[connected
>>>>>>>>>
>>>>>>>> local=/
>>>>>>>>
>>>>>>>>> 127.0.0.1:49619 remote=/127.0.0.1:2181]
>>>>>>>>> 2009-12-09 17:27:56,806 - INFO
>>>>>>>>>
>>>>>>>>  [main-SendThread:ClientCnxn$SendThread@868
>>>>>>>>
>>>>>>>>> ]
>>>>>>>>> - Server connection successful
>>>>>>>>> WatchedEvent: Server state change. New state: SyncConnected
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Should I just use 3.2.1 version?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wed, Dec 9, 2009 at 9:20 AM, Mahadev Konar <
>>>>>>>>>
>>>>>>>> mahadev@yahoo-inc.com>
>>>>>> wrote:
>>>>>>>>>  Hi,
>>>>>>>>>>  Can you try this?
>>>>>>>>>>
>>>>>>>>>> bin/zkCli.sh 127.0.0.1:2181
>>>>>>>>>>
>>>>>>>>>> The -server command was added later as far as I remember.
>>>>>>>>>>
>>>>>>>>>> Thanks
>>>>>>>>>> mahadev
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 12/9/09 9:05 AM, "Something Something" <
>>>>>>>>>>
>>>>>>>>> mailinglists19@gmail.com
>>>>>> wrote:
>>>>>>>>>>  I am trying to start ZooKeeper on an EC2 instance.
 Here's what
>>>>>>>>>> I
>>>>>> did:
>>>>>>>>> 1)  Downloaded & Unpacked ZooKeeper 3.1.1 on EC2
instance.
>>>>>>>>>>> 2)  cp /conf/zoo_sample.cfg /conf/zoo.cfg
>>>>>>>>>>> 3)  Changed the dataDir path to point to my EBS
volume.
>>>>>>>>>>> 4)  In one command window, ran /bin/zkServer.sh
start
>>>>>>>>>>> (The last message I see is... "Snapshotting:
0)
>>>>>>>>>>>
>>>>>>>>>>> 5)  Opened another command window, and ran jps
>>>>>>>>>>> (This shows a new process called, QuorumPeerMain.
 That's the
>>>>>>>>>>>
>>>>>>>>>> only
>>>>>> one
>>>>>>>>> I
>>>>>>>>>
>>>>>>>>>> see.)
>>>>>>>>>>> 6)  As per documentation, tried
>>>>>>>>>>>
>>>>>>>>>>> bin/zkCli.sh -server 127.0.0.1:2181
>>>>>>>>>>>
>>>>>>>>>>> (This gives me IOException: USAGE)
>>>>>>>>>>>
>>>>>>>>>>> 7) So I ran:
>>>>>>>>>>>
>>>>>>>>>>> bin/zkCli.sh -server 127.0.0.1:2181 ls
>>>>>>>>>>>
>>>>>>>>>>> Got UnknownHostException: -server
>>>>>>>>>>>
>>>>>>>>>>> 8)  So I tried various ways of specifying IP
address in EC2,
>>>>>>>>>>>
>>>>>>>>>> such
>>>>>> as:
>>>>>>>>> 10.xx.xx.xx
>>>>>>>>>>> ec2-xx-xx-xx-xxx.compute-1.amazonaws.com
>>>>>>>>>>> domU-12-31-xx-xx-xx-xx.compute-1.internal
>>>>>>>>>>> domU-12-31-xx-xx-xx-xx
>>>>>>>>>>>
>>>>>>>>>>> None of them worked.  Keep getting UnknownHostException.
>>>>>>>>>>>
>>>>>>>>>>> What am I doing wrong.  Please help.  Thanks.
>>>>>>>>>>>
>>>>>>>>>>
> 

Mime
View raw message