zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Bosschaert <david.bosscha...@gmail.com>
Subject Re: Using zookeeper in a hightly restricted (cloud) environment
Date Mon, 11 Jun 2012 19:47:51 GMT
Hi Patrick,

This was in relation to getting it to work on RedHat OpenShift. I'll
write a blog article about it in the not too distant future. I'll ping
back with the link when that's done. We could then possibly put the
link into one of the ZK pages...

Best regards,

David

On 8 June 2012 23:15, Patrick Hunt <phunt@apache.org> wrote:
> Sweet. Perhaps you could update the FAQ or troubleshooting guide with
> your findings (ie the problem and how you debugged/fixed it), if you
> think it's appropriate?
>
> https://cwiki.apache.org/confluence/display/ZOOKEEPER/Troubleshooting
> https://cwiki.apache.org/confluence/display/ZOOKEEPER/FAQ
>
> Patrick
>
> On Thu, Jun 7, 2012 at 2:17 PM, David Bosschaert
> <david.bosschaert@gmail.com> wrote:
>> Thanks for confirming this Patrick, this helped me isolate the problem
>> I was having.
>> It turned out that the problem was outside of Zookeeper and I have now
>> successfully been able to use Zookeeper in my environment.
>>
>> Thanks again,
>>
>> David
>>
>> On 30 May 2012 17:45, Patrick Hunt <phunt@apache.org> wrote:
>>> This should work fine. Notice that while the connection is established
>>> the session is never established, eventually it times out.
>>>
>>> Have you tried running a 4 letter word and see if that at least works?
>>> http://zookeeper.apache.org/doc/r3.4.3/zookeeperAdmin.html#sc_zkCommands
>>> (try 'stat' for example)
>>>
>>> Was there anything further in the server log after the client
>>> attempted to connect? There should have been a message that the client
>>> was attempting to connect, if this is not the case then perhaps the
>>> firewall is blocking. Try the 4lw or using telnet to verify that your
>>> client can get past the firewall to the socket bound by the server.
>>>
>>> Patrick
>>>
>>> On Fri, May 25, 2012 at 2:23 PM, David Bosschaert
>>> <david.bosschaert@gmail.com> wrote:
>>>> Hi,
>>>>
>>>> I'm trying to use zookeeper in a cloud environment where I have an
>>>> extremely limited set of server ports available.
>>>> The setup is as follows. I want to run the ZooKeeper server inside my
>>>> cloud node so that it can be accessed from outside. In the node I can
>>>> open a Server Socket on an IP that looks like this 127.3.241.1:8080.
>>>> Port 8080 is then only Server Port I can open, no other ServerSockets
>>>> are allowed.
>>>> From the outside this gets translated into some host.somedomain.com:80
>>>> (port 80!).
>>>>
>>>> The zkServer starts up fairly happy (I set
>>>> clientPortAddress=127.3.241.1 and clientPort=8080) - see the log
>>>> below.
>>>>
>>>> However, when I start the client console
>>>> $ bin/zkCli.sh -server host.somedomain.com:80
>>>> it does not connect properly, and 'ls /' just hangs. I've also pasted
>>>> the log below
>>>>
>>>> My question is: is it possible to use ZooKeeper in such a restricted
>>>> environment at all? If so how should I configure it?
>>>>
>>>> Thanks,
>>>>
>>>> David
>>>>
>>>> Here's the server log:
>>>> remote: JMX disabled by user request
>>>> remote: Using config:
>>>> /var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../conf/zoo.cfg
>>>> remote: 2012-05-25 11:11:54,642 [myid:] - INFO
>>>> [main:QuorumPeerConfig@101] - Reading configuration from:
>>>> /var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../conf/zoo.cfg
>>>> remote: 2012-05-25 11:11:54,735 [myid:] - INFO
>>>> [main:DatadirCleanupManager@78] - autopurge.snapRetainCount set to 3
>>>> remote: 2012-05-25 11:11:54,736 [myid:] - INFO
>>>> [main:DatadirCleanupManager@79] - autopurge.purgeInterval set to 0
>>>> remote: 2012-05-25 11:11:54,737 [myid:] - INFO
>>>> [main:DatadirCleanupManager@101] - Purge task is not scheduled.
>>>> remote: 2012-05-25 11:11:54,738 [myid:] - WARN
>>>> [main:QuorumPeerMain@113] - Either no config or no quorum defined in
>>>> config, running  in standalone mode
>>>> remote: 2012-05-25 11:11:55,053 [myid:] - INFO
>>>> [main:QuorumPeerConfig@101] - Reading configuration from:
>>>> /var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../conf/zoo.cfg
>>>> remote: 2012-05-25 11:11:55,054 [myid:] - INFO
>>>> [main:ZooKeeperServerMain@95] - Starting server
>>>> remote: 2012-05-25 11:11:55,142 [myid:] - INFO  [main:Environment@98]
>>>> - Server environment:zookeeper.version=3.4.3-1240972, built on
>>>> 02/06/2012 10:48 GMT
>>>> remote: 2012-05-25 11:11:55,143 [myid:] - INFO  [main:Environment@98]
>>>> - Server environment:host.name=ex-std-node68.prod.rhcloud.com
>>>> remote: 2012-05-25 11:11:55,143 [myid:] - INFO  [main:Environment@98]
>>>> - Server environment:java.version=1.6.0_22
>>>> remote: 2012-05-25 11:11:55,144 [myid:] - INFO  [main:Environment@98]
>>>> - Server environment:java.vendor=Sun Microsystems Inc.
>>>> remote: 2012-05-25 11:11:55,144 [myid:] - INFO  [main:Environment@98]
>>>> - Server environment:java.home=/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0/jre
>>>> remote: 2012-05-25 11:11:55,145 [myid:] - INFO  [main:Environment@98]
>>>> - Server environment:java.class.path=/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../build/classes:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../build/lib/*.jar:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../lib/slf4j-log4j12-1.6.1.jar:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../lib/slf4j-api-1.6.1.jar:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../lib/netty-3.2.2.Final.jar:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../lib/log4j-1.2.15.jar:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../lib/jline-0.9.94.jar:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../zookeeper-3.4.3.jar:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../src/java/lib/*.jar:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../conf:
>>>> remote: 2012-05-25 11:11:55,146 [myid:] - INFO  [main:Environment@98]
>>>> - Server environment:java.library.path=/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0/jre/lib/i386/server:/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0/jre/lib/i386:/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0/jre/../lib/i386:/usr/java/packages/lib/i386:/lib:/usr/lib
>>>> remote: 2012-05-25 11:11:55,147 [myid:] - INFO  [main:Environment@98]
>>>> - Server environment:java.io.tmpdir=/tmp
>>>> remote: 2012-05-25 11:11:55,151 [myid:] - INFO  [main:Environment@98]
>>>> - Server environment:java.compiler=<NA>
>>>> remote: 2012-05-25 11:11:55,152 [myid:] - INFO  [main:Environment@98]
>>>> - Server environment:os.name=Linux
>>>> remote: 2012-05-25 11:11:55,153 [myid:] - INFO  [main:Environment@98]
>>>> - Server environment:os.arch=i386
>>>> remote: 2012-05-25 11:11:55,154 [myid:] - INFO  [main:Environment@98]
>>>> - Server environment:os.version=2.6.32-220.17.1.el6.x86_64
>>>> remote: 2012-05-25 11:11:55,155 [myid:] - INFO  [main:Environment@98]
>>>> - Server environment:user.name=80d2b6413ce14d3eb25c86e33f69c275
>>>> remote: 2012-05-25 11:11:55,230 [myid:] - INFO  [main:Environment@98]
>>>> - Server environment:user.home=/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275
>>>> remote: 2012-05-25 11:11:55,231 [myid:] - INFO  [main:Environment@98]
>>>> - Server environment:user.dir=/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/runtime/repo/zookeeper-3.4.3
>>>> remote: 2012-05-25 11:11:55,241 [myid:] - INFO
>>>> [main:ZooKeeperServer@733] - tickTime set to 2000
>>>> remote: 2012-05-25 11:11:55,242 [myid:] - INFO
>>>> [main:ZooKeeperServer@742] - minSessionTimeout set to -1
>>>> remote: 2012-05-25 11:11:55,242 [myid:] - INFO
>>>> [main:ZooKeeperServer@751] - maxSessionTimeout set to -1
>>>> remote: 2012-05-25 11:11:55,345 [myid:] - INFO
>>>> [main:NIOServerCnxnFactory@110] - binding to port /127.3.241.1:8080
>>>> remote: 2012-05-25 11:11:55,443 [myid:] - INFO
>>>> [main:FileTxnSnapLog@238] - Snapshotting: 0x0 to
>>>> ./zk_data/version-2/snapshot.0
>>>>
>>>> The client log:
>>>> $ bin/zkCli.sh -server zks1-davidosgi.rhcloud.com:80
>>>> Connecting to zks1-davidosgi.rhcloud.com:80
>>>> 2012-05-25 22:16:29,123 [myid:] - INFO  [main:Environment@98] - Client
>>>> environment:zookeeper.version=3.4.3-1240972, built on 02/06/2012 10:48
>>>> GMT
>>>> 2012-05-25 22:16:29,129 [myid:] - INFO  [main:Environment@98] - Client
>>>> environment:host.name=192.168.1.2
>>>> 2012-05-25 22:16:29,129 [myid:] - INFO  [main:Environment@98] - Client
>>>> environment:java.version=1.6.0_31
>>>> 2012-05-25 22:16:29,130 [myid:] - INFO  [main:Environment@98] - Client
>>>> environment:java.vendor=Apple Inc.
>>>> 2012-05-25 22:16:29,130 [myid:] - INFO  [main:Environment@98] - Client
>>>> environment:java.home=/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home
>>>> 2012-05-25 22:16:29,131 [myid:] - INFO  [main:Environment@98] - Client
>>>> environment:java.class.path=/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../build/classes:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../build/lib/*.jar:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../lib/slf4j-log4j12-1.6.1.jar:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../lib/slf4j-api-1.6.1.jar:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../lib/netty-3.2.2.Final.jar:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../lib/log4j-1.2.15.jar:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../lib/jline-0.9.94.jar:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../zookeeper-3.4.3.jar:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../src/java/lib/*.jar:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../conf:
>>>> 2012-05-25 22:16:29,135 [myid:] - INFO  [main:Environment@98] - Client
>>>> environment:java.library.path=.:/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java
>>>> 2012-05-25 22:16:29,135 [myid:] - INFO  [main:Environment@98] - Client
>>>> environment:java.io.tmpdir=/var/folders/s9/8mc72j514lgb0mj11czx585m0000gq/T/
>>>> 2012-05-25 22:16:29,136 [myid:] - INFO  [main:Environment@98] - Client
>>>> environment:java.compiler=<NA>
>>>> 2012-05-25 22:16:29,136 [myid:] - INFO  [main:Environment@98] - Client
>>>> environment:os.name=Mac OS X
>>>> 2012-05-25 22:16:29,136 [myid:] - INFO  [main:Environment@98] - Client
>>>> environment:os.arch=x86_64
>>>> 2012-05-25 22:16:29,137 [myid:] - INFO  [main:Environment@98] - Client
>>>> environment:os.version=10.7.4
>>>> 2012-05-25 22:16:29,137 [myid:] - INFO  [main:Environment@98] - Client
>>>> environment:user.name=david
>>>> 2012-05-25 22:16:29,137 [myid:] - INFO  [main:Environment@98] - Client
>>>> environment:user.home=/Users/david
>>>> 2012-05-25 22:16:29,138 [myid:] - INFO  [main:Environment@98] - Client
>>>> environment:user.dir=/Users/david/proj/zk-test/zookeeper-3.4.3
>>>> 2012-05-25 22:16:29,139 [myid:] - INFO  [main:ZooKeeper@433] -
>>>> Initiating client connection,
>>>> connectString=zks1-davidosgi.rhcloud.com:80 sessionTimeout=30000
>>>> watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@5210f6d3
>>>> Welcome to ZooKeeper!
>>>> 2012-05-25 22:16:29,582 [myid:] - INFO
>>>> [main-SendThread():ClientCnxn$SendThread@933] - Opening socket
>>>> connection to server /23.22.86.141:80
>>>> 2012-05-25 22:16:29,591 [myid:] - WARN
>>>> [main-SendThread(ec2-23-22-86-141.compute-1.amazonaws.com:80):ZooKeeperSaslClient@123]
>>>> - SecurityException: java.lang.SecurityException: Unable to locate a
>>>> login configuration occurred when trying to find JAAS configuration.
>>>> 2012-05-25 22:16:29,591 [myid:] - INFO
>>>> [main-SendThread(ec2-23-22-86-141.compute-1.amazonaws.com:80):ZooKeeperSaslClient@125]
>>>> - Client will not SASL-authenticate because the default JAAS
>>>> configuration section 'Client' could not be found. If you are not
>>>> using SASL, you may ignore this. On the other hand, if you expected
>>>> SASL to work, please fix your JAAS configuration.
>>>> JLine support is enabled
>>>> [zk: zks1-davidosgi.rhcloud.com:80(CONNECTING) 0] 2012-05-25
>>>> 22:16:29,735 [myid:] - INFO
>>>> [main-SendThread(ec2-23-22-86-141.compute-1.amazonaws.com:80):ClientCnxn$SendThread@846]
>>>> - Socket connection established to
>>>> ec2-23-22-86-141.compute-1.amazonaws.com/23.22.86.141:80, initiating
>>>> session
>>>>
>>>> [zk: zks1-davidosgi.rhcloud.com:80(CONNECTING) 0] ls /
>>>> 2012-05-25 22:16:59,733 [myid:] - INFO
>>>> [main-SendThread(ec2-23-22-86-141.compute-1.amazonaws.com:80):ClientCnxn$SendThread@1051]
>>>> - Client session timed out, have not heard from server in 30000ms for
>>>> sessionid 0x0, closing socket connection and attempting reconnect
>>>> Exception in thread "main"
>>>> org.apache.zookeeper.KeeperException$ConnectionLossException:
>>>> KeeperErrorCode = ConnectionLoss for /
>>>>        at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
>>>>        at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
>>>>        at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1448)
>>>>        at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1476)
>>>>        at org.apache.zookeeper.ZooKeeperMain.processZKCmd(ZooKeeperMain.java:717)
>>>>        at org.apache.zookeeper.ZooKeeperMain.processCmd(ZooKeeperMain.java:593)
>>>>        at org.apache.zookeeper.ZooKeeperMain.executeLine(ZooKeeperMain.java:365)
>>>>        at org.apache.zookeeper.ZooKeeperMain.run(ZooKeeperMain.java:323)
>>>>        at org.apache.zookeeper.ZooKeeperMain.main(ZooKeeperMain.java:282)
>>>> david@pop ~/proj/zk-test/zookeeper-3.4.3 $ bin/zkCli.sh -server
>>>> zks1-davidosgi.rhcloud.com:80

Mime
View raw message