Return-Path: X-Original-To: apmail-zookeeper-user-archive@www.apache.org Delivered-To: apmail-zookeeper-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 932A9D635 for ; Thu, 28 Jun 2012 19:54:30 +0000 (UTC) Received: (qmail 76337 invoked by uid 500); 28 Jun 2012 19:54:30 -0000 Delivered-To: apmail-zookeeper-user-archive@zookeeper.apache.org Received: (qmail 76293 invoked by uid 500); 28 Jun 2012 19:54:30 -0000 Mailing-List: contact user-help@zookeeper.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@zookeeper.apache.org Delivered-To: mailing list user@zookeeper.apache.org Received: (qmail 76285 invoked by uid 99); 28 Jun 2012 19:54:30 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 28 Jun 2012 19:54:29 +0000 X-ASF-Spam-Status: No, hits=-0.7 required=5.0 tests=FSL_RCVD_USER,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of david.bosschaert@gmail.com designates 209.85.210.42 as permitted sender) Received: from [209.85.210.42] (HELO mail-pz0-f42.google.com) (209.85.210.42) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 28 Jun 2012 19:54:24 +0000 Received: by dalf4 with SMTP id f4so2885060dal.15 for ; Thu, 28 Jun 2012 12:54:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type:content-transfer-encoding; bh=uNqF0xAWQ8Of6yH8f+4jMM2P18hP6DE7dPWRYHfPJac=; b=kDF5xrU0vNZmZqFsRPiRCpo8lbbTR+4dXZfsTx8CV2BB5YfX5WOip4BQECrJ45dhJ6 GrgocSkDtEXIfwTwe+X4fl9hBHPquBvE4pQv3ARktqsLzkyRlqgEmiMawu/p3JGuH4uZ h5eg/xtPibsq+wO7X3qGwRcfylTQl0FOD+Zp+ckmcUm+0I9GvOxr+ih8yyw9a/03BBkB KIT+DCy1Rj9fC9Fz6KYTb2+bGZMX1D7kdlrXPlYbGk/QpMp8jGrJlodu9qmspMq8wu2A gUl9gDXgrJrVOvQML3RApyT3JMIBmafyh8Mmuf3e4D+lgcE7/+l4XiSokxGx4sGbKyxf 8Bdw== Received: by 10.68.138.166 with SMTP id qr6mr11326176pbb.43.1340913241995; Thu, 28 Jun 2012 12:54:01 -0700 (PDT) MIME-Version: 1.0 Received: by 10.143.36.12 with HTTP; Thu, 28 Jun 2012 12:53:41 -0700 (PDT) In-Reply-To: References: From: David Bosschaert Date: Thu, 28 Jun 2012 20:53:41 +0100 Message-ID: Subject: Re: Using zookeeper in a hightly restricted (cloud) environment To: user@zookeeper.apache.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Just to follow up, the blog that uses ZooKeeper as a discovery component for an OSGi Cloud domain is now here: http://coderthoughts.blogspot.com/2012/06/cloud-ecosystems-with-osgi.html It doesn't go into much detail around setting up ZooKeeper, that's all encapsulated in here: https://github.com/bosschaert/osgi-cloud-discovery Not sure whether this is suitable for a ZooKeeper FAQ, but I thought I'd share it anyway. Best regards, David On 11 June 2012 20:47, David Bosschaert wrote: > Hi Patrick, > > This was in relation to getting it to work on RedHat OpenShift. I'll > write a blog article about it in the not too distant future. I'll ping > back with the link when that's done. We could then possibly put the > link into one of the ZK pages... > > Best regards, > > David > > On 8 June 2012 23:15, Patrick Hunt wrote: >> Sweet. Perhaps you could update the FAQ or troubleshooting guide with >> your findings (ie the problem and how you debugged/fixed it), if you >> think it's appropriate? >> >> https://cwiki.apache.org/confluence/display/ZOOKEEPER/Troubleshooting >> https://cwiki.apache.org/confluence/display/ZOOKEEPER/FAQ >> >> Patrick >> >> On Thu, Jun 7, 2012 at 2:17 PM, David Bosschaert >> wrote: >>> Thanks for confirming this Patrick, this helped me isolate the problem >>> I was having. >>> It turned out that the problem was outside of Zookeeper and I have now >>> successfully been able to use Zookeeper in my environment. >>> >>> Thanks again, >>> >>> David >>> >>> On 30 May 2012 17:45, Patrick Hunt wrote: >>>> This should work fine. Notice that while the connection is established >>>> the session is never established, eventually it times out. >>>> >>>> Have you tried running a 4 letter word and see if that at least works? >>>> http://zookeeper.apache.org/doc/r3.4.3/zookeeperAdmin.html#sc_zkComman= ds >>>> (try 'stat' for example) >>>> >>>> Was there anything further in the server log after the client >>>> attempted to connect? There should have been a message that the client >>>> was attempting to connect, if this is not the case then perhaps the >>>> firewall is blocking. Try the 4lw or using telnet to verify that your >>>> client can get past the firewall to the socket bound by the server. >>>> >>>> Patrick >>>> >>>> On Fri, May 25, 2012 at 2:23 PM, David Bosschaert >>>> wrote: >>>>> Hi, >>>>> >>>>> I'm trying to use zookeeper in a cloud environment where I have an >>>>> extremely limited set of server ports available. >>>>> The setup is as follows. I want to run the ZooKeeper server inside my >>>>> cloud node so that it can be accessed from outside. In the node I can >>>>> open a Server Socket on an IP that looks like this 127.3.241.1:8080. >>>>> Port 8080 is then only Server Port I can open, no other ServerSockets >>>>> are allowed. >>>>> From the outside this gets translated into some host.somedomain.com:8= 0 >>>>> (port 80!). >>>>> >>>>> The zkServer starts up fairly happy (I set >>>>> clientPortAddress=3D127.3.241.1 and clientPort=3D8080) - see the log >>>>> below. >>>>> >>>>> However, when I start the client console >>>>> $ bin/zkCli.sh -server host.somedomain.com:80 >>>>> it does not connect properly, and 'ls /' just hangs. I've also pasted >>>>> the log below >>>>> >>>>> My question is: is it possible to use ZooKeeper in such a restricted >>>>> environment at all? If so how should I configure it? >>>>> >>>>> Thanks, >>>>> >>>>> David >>>>> >>>>> Here's the server log: >>>>> remote: JMX disabled by user request >>>>> remote: Using config: >>>>> /var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookee= per-3.4.3/bin/../conf/zoo.cfg >>>>> remote: 2012-05-25 11:11:54,642 [myid:] - INFO >>>>> [main:QuorumPeerConfig@101] - Reading configuration from: >>>>> /var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookee= per-3.4.3/bin/../conf/zoo.cfg >>>>> remote: 2012-05-25 11:11:54,735 [myid:] - INFO >>>>> [main:DatadirCleanupManager@78] - autopurge.snapRetainCount set to 3 >>>>> remote: 2012-05-25 11:11:54,736 [myid:] - INFO >>>>> [main:DatadirCleanupManager@79] - autopurge.purgeInterval set to 0 >>>>> remote: 2012-05-25 11:11:54,737 [myid:] - INFO >>>>> [main:DatadirCleanupManager@101] - Purge task is not scheduled. >>>>> remote: 2012-05-25 11:11:54,738 [myid:] - WARN >>>>> [main:QuorumPeerMain@113] - Either no config or no quorum defined in >>>>> config, running =A0in standalone mode >>>>> remote: 2012-05-25 11:11:55,053 [myid:] - INFO >>>>> [main:QuorumPeerConfig@101] - Reading configuration from: >>>>> /var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookee= per-3.4.3/bin/../conf/zoo.cfg >>>>> remote: 2012-05-25 11:11:55,054 [myid:] - INFO >>>>> [main:ZooKeeperServerMain@95] - Starting server >>>>> remote: 2012-05-25 11:11:55,142 [myid:] - INFO =A0[main:Environment@9= 8] >>>>> - Server environment:zookeeper.version=3D3.4.3-1240972, built on >>>>> 02/06/2012 10:48 GMT >>>>> remote: 2012-05-25 11:11:55,143 [myid:] - INFO =A0[main:Environment@9= 8] >>>>> - Server environment:host.name=3Dex-std-node68.prod.rhcloud.com >>>>> remote: 2012-05-25 11:11:55,143 [myid:] - INFO =A0[main:Environment@9= 8] >>>>> - Server environment:java.version=3D1.6.0_22 >>>>> remote: 2012-05-25 11:11:55,144 [myid:] - INFO =A0[main:Environment@9= 8] >>>>> - Server environment:java.vendor=3DSun Microsystems Inc. >>>>> remote: 2012-05-25 11:11:55,144 [myid:] - INFO =A0[main:Environment@9= 8] >>>>> - Server environment:java.home=3D/usr/lib/jvm/java-1.6.0-openjdk-1.6.= 0.0/jre >>>>> remote: 2012-05-25 11:11:55,145 [myid:] - INFO =A0[main:Environment@9= 8] >>>>> - Server environment:java.class.path=3D/var/lib/stickshift/80d2b6413c= e14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../build/classes:/var/= lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/b= in/../build/lib/*.jar:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/= zks1/repo/zookeeper-3.4.3/bin/../lib/slf4j-log4j12-1.6.1.jar:/var/lib/stick= shift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../lib= /slf4j-api-1.6.1.jar:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/z= ks1/repo/zookeeper-3.4.3/bin/../lib/netty-3.2.2.Final.jar:/var/lib/stickshi= ft/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../lib/lo= g4j-1.2.15.jar:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/re= po/zookeeper-3.4.3/bin/../lib/jline-0.9.94.jar:/var/lib/stickshift/80d2b641= 3ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../zookeeper-3.4.3.j= ar:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper= -3.4.3/bin/../src/java/lib/*.jar:/var/lib/stickshift/80d2b6413ce14d3eb25c86= e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../conf: >>>>> remote: 2012-05-25 11:11:55,146 [myid:] - INFO =A0[main:Environment@9= 8] >>>>> - Server environment:java.library.path=3D/usr/lib/jvm/java-1.6.0-open= jdk-1.6.0.0/jre/lib/i386/server:/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0/jre= /lib/i386:/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0/jre/../lib/i386:/usr/java= /packages/lib/i386:/lib:/usr/lib >>>>> remote: 2012-05-25 11:11:55,147 [myid:] - INFO =A0[main:Environment@9= 8] >>>>> - Server environment:java.io.tmpdir=3D/tmp >>>>> remote: 2012-05-25 11:11:55,151 [myid:] - INFO =A0[main:Environment@9= 8] >>>>> - Server environment:java.compiler=3D >>>>> remote: 2012-05-25 11:11:55,152 [myid:] - INFO =A0[main:Environment@9= 8] >>>>> - Server environment:os.name=3DLinux >>>>> remote: 2012-05-25 11:11:55,153 [myid:] - INFO =A0[main:Environment@9= 8] >>>>> - Server environment:os.arch=3Di386 >>>>> remote: 2012-05-25 11:11:55,154 [myid:] - INFO =A0[main:Environment@9= 8] >>>>> - Server environment:os.version=3D2.6.32-220.17.1.el6.x86_64 >>>>> remote: 2012-05-25 11:11:55,155 [myid:] - INFO =A0[main:Environment@9= 8] >>>>> - Server environment:user.name=3D80d2b6413ce14d3eb25c86e33f69c275 >>>>> remote: 2012-05-25 11:11:55,230 [myid:] - INFO =A0[main:Environment@9= 8] >>>>> - Server environment:user.home=3D/var/lib/stickshift/80d2b6413ce14d3e= b25c86e33f69c275 >>>>> remote: 2012-05-25 11:11:55,231 [myid:] - INFO =A0[main:Environment@9= 8] >>>>> - Server environment:user.dir=3D/var/lib/stickshift/80d2b6413ce14d3eb= 25c86e33f69c275/zks1/runtime/repo/zookeeper-3.4.3 >>>>> remote: 2012-05-25 11:11:55,241 [myid:] - INFO >>>>> [main:ZooKeeperServer@733] - tickTime set to 2000 >>>>> remote: 2012-05-25 11:11:55,242 [myid:] - INFO >>>>> [main:ZooKeeperServer@742] - minSessionTimeout set to -1 >>>>> remote: 2012-05-25 11:11:55,242 [myid:] - INFO >>>>> [main:ZooKeeperServer@751] - maxSessionTimeout set to -1 >>>>> remote: 2012-05-25 11:11:55,345 [myid:] - INFO >>>>> [main:NIOServerCnxnFactory@110] - binding to port /127.3.241.1:8080 >>>>> remote: 2012-05-25 11:11:55,443 [myid:] - INFO >>>>> [main:FileTxnSnapLog@238] - Snapshotting: 0x0 to >>>>> ./zk_data/version-2/snapshot.0 >>>>> >>>>> The client log: >>>>> $ bin/zkCli.sh -server zks1-davidosgi.rhcloud.com:80 >>>>> Connecting to zks1-davidosgi.rhcloud.com:80 >>>>> 2012-05-25 22:16:29,123 [myid:] - INFO =A0[main:Environment@98] - Cli= ent >>>>> environment:zookeeper.version=3D3.4.3-1240972, built on 02/06/2012 10= :48 >>>>> GMT >>>>> 2012-05-25 22:16:29,129 [myid:] - INFO =A0[main:Environment@98] - Cli= ent >>>>> environment:host.name=3D192.168.1.2 >>>>> 2012-05-25 22:16:29,129 [myid:] - INFO =A0[main:Environment@98] - Cli= ent >>>>> environment:java.version=3D1.6.0_31 >>>>> 2012-05-25 22:16:29,130 [myid:] - INFO =A0[main:Environment@98] - Cli= ent >>>>> environment:java.vendor=3DApple Inc. >>>>> 2012-05-25 22:16:29,130 [myid:] - INFO =A0[main:Environment@98] - Cli= ent >>>>> environment:java.home=3D/System/Library/Java/JavaVirtualMachines/1.6.= 0.jdk/Contents/Home >>>>> 2012-05-25 22:16:29,131 [myid:] - INFO =A0[main:Environment@98] - Cli= ent >>>>> environment:java.class.path=3D/Users/david/proj/zk-test/zookeeper-3.4= .3/bin/../build/classes:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../bu= ild/lib/*.jar:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../lib/slf4j-lo= g4j12-1.6.1.jar:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../lib/slf4j-= api-1.6.1.jar:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../lib/netty-3.= 2.2.Final.jar:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../lib/log4j-1.= 2.15.jar:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../lib/jline-0.9.94.= jar:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../zookeeper-3.4.3.jar:/U= sers/david/proj/zk-test/zookeeper-3.4.3/bin/../src/java/lib/*.jar:/Users/da= vid/proj/zk-test/zookeeper-3.4.3/bin/../conf: >>>>> 2012-05-25 22:16:29,135 [myid:] - INFO =A0[main:Environment@98] - Cli= ent >>>>> environment:java.library.path=3D.:/Library/Java/Extensions:/System/Li= brary/Java/Extensions:/usr/lib/java >>>>> 2012-05-25 22:16:29,135 [myid:] - INFO =A0[main:Environment@98] - Cli= ent >>>>> environment:java.io.tmpdir=3D/var/folders/s9/8mc72j514lgb0mj11czx585m= 0000gq/T/ >>>>> 2012-05-25 22:16:29,136 [myid:] - INFO =A0[main:Environment@98] - Cli= ent >>>>> environment:java.compiler=3D >>>>> 2012-05-25 22:16:29,136 [myid:] - INFO =A0[main:Environment@98] - Cli= ent >>>>> environment:os.name=3DMac OS X >>>>> 2012-05-25 22:16:29,136 [myid:] - INFO =A0[main:Environment@98] - Cli= ent >>>>> environment:os.arch=3Dx86_64 >>>>> 2012-05-25 22:16:29,137 [myid:] - INFO =A0[main:Environment@98] - Cli= ent >>>>> environment:os.version=3D10.7.4 >>>>> 2012-05-25 22:16:29,137 [myid:] - INFO =A0[main:Environment@98] - Cli= ent >>>>> environment:user.name=3Ddavid >>>>> 2012-05-25 22:16:29,137 [myid:] - INFO =A0[main:Environment@98] - Cli= ent >>>>> environment:user.home=3D/Users/david >>>>> 2012-05-25 22:16:29,138 [myid:] - INFO =A0[main:Environment@98] - Cli= ent >>>>> environment:user.dir=3D/Users/david/proj/zk-test/zookeeper-3.4.3 >>>>> 2012-05-25 22:16:29,139 [myid:] - INFO =A0[main:ZooKeeper@433] - >>>>> Initiating client connection, >>>>> connectString=3Dzks1-davidosgi.rhcloud.com:80 sessionTimeout=3D30000 >>>>> watcher=3Dorg.apache.zookeeper.ZooKeeperMain$MyWatcher@5210f6d3 >>>>> Welcome to ZooKeeper! >>>>> 2012-05-25 22:16:29,582 [myid:] - INFO >>>>> [main-SendThread():ClientCnxn$SendThread@933] - Opening socket >>>>> connection to server /23.22.86.141:80 >>>>> 2012-05-25 22:16:29,591 [myid:] - WARN >>>>> [main-SendThread(ec2-23-22-86-141.compute-1.amazonaws.com:80):ZooKeep= erSaslClient@123] >>>>> - SecurityException: java.lang.SecurityException: Unable to locate a >>>>> login configuration occurred when trying to find JAAS configuration. >>>>> 2012-05-25 22:16:29,591 [myid:] - INFO >>>>> [main-SendThread(ec2-23-22-86-141.compute-1.amazonaws.com:80):ZooKeep= erSaslClient@125] >>>>> - Client will not SASL-authenticate because the default JAAS >>>>> configuration section 'Client' could not be found. If you are not >>>>> using SASL, you may ignore this. On the other hand, if you expected >>>>> SASL to work, please fix your JAAS configuration. >>>>> JLine support is enabled >>>>> [zk: zks1-davidosgi.rhcloud.com:80(CONNECTING) 0] 2012-05-25 >>>>> 22:16:29,735 [myid:] - INFO >>>>> [main-SendThread(ec2-23-22-86-141.compute-1.amazonaws.com:80):ClientC= nxn$SendThread@846] >>>>> - Socket connection established to >>>>> ec2-23-22-86-141.compute-1.amazonaws.com/23.22.86.141:80, initiating >>>>> session >>>>> >>>>> [zk: zks1-davidosgi.rhcloud.com:80(CONNECTING) 0] ls / >>>>> 2012-05-25 22:16:59,733 [myid:] - INFO >>>>> [main-SendThread(ec2-23-22-86-141.compute-1.amazonaws.com:80):ClientC= nxn$SendThread@1051] >>>>> - Client session timed out, have not heard from server in 30000ms for >>>>> sessionid 0x0, closing socket connection and attempting reconnect >>>>> Exception in thread "main" >>>>> org.apache.zookeeper.KeeperException$ConnectionLossException: >>>>> KeeperErrorCode =3D ConnectionLoss for / >>>>> =A0 =A0 =A0 =A0at org.apache.zookeeper.KeeperException.create(KeeperE= xception.java:99) >>>>> =A0 =A0 =A0 =A0at org.apache.zookeeper.KeeperException.create(KeeperE= xception.java:51) >>>>> =A0 =A0 =A0 =A0at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeepe= r.java:1448) >>>>> =A0 =A0 =A0 =A0at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeepe= r.java:1476) >>>>> =A0 =A0 =A0 =A0at org.apache.zookeeper.ZooKeeperMain.processZKCmd(Zoo= KeeperMain.java:717) >>>>> =A0 =A0 =A0 =A0at org.apache.zookeeper.ZooKeeperMain.processCmd(ZooKe= eperMain.java:593) >>>>> =A0 =A0 =A0 =A0at org.apache.zookeeper.ZooKeeperMain.executeLine(ZooK= eeperMain.java:365) >>>>> =A0 =A0 =A0 =A0at org.apache.zookeeper.ZooKeeperMain.run(ZooKeeperMai= n.java:323) >>>>> =A0 =A0 =A0 =A0at org.apache.zookeeper.ZooKeeperMain.main(ZooKeeperMa= in.java:282) >>>>> david@pop ~/proj/zk-test/zookeeper-3.4.3 $ bin/zkCli.sh -server >>>>> zks1-davidosgi.rhcloud.com:80