Mailing-List: contact user-help@zookeeper.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@zookeeper.apache.org
MIME-Version: 1.0
In-Reply-To: <1505863086.1396299.1111744976.7DFF5241@webmail.messagingengine.com>
References: <96654c1c-84df-aaa2-6c07-1e0135f936ae@servicemesh.com> <1505863086.1396299.1111744976.7DFF5241@webmail.messagingengine.com>
From: Mark Johnson <mark.johnson@servicemesh.com>
Date: Wed, 20 Sep 2017 01:17:47 -0500
Message-ID: <CABMR8WPg2s1AND3NySR6cGgR6wXJhBC1t9QCpiSjRZGfN305pA@mail.gmail.com>
Subject: Re: Problem creating Ephemeral znodes
To: user@zookeeper.apache.org
Content-Type: multipart/alternative; boundary="001a1146b324462f42055998efa8"
archived-at: Wed, 20 Sep 2017 06:17:57 -0000

--001a1146b324462f42055998efa8
Content-Type: text/plain; charset="UTF-8"

Hi Abraham,

I believe I'm using nio - from the log:

2017-09-19 15:25:51,761 [myid:010000001020] - INFO
[main:NIOServerCnxnFactory@673] - Configuring NIO connection handler with
10s sessionless connection timeout, 1 selector thread(s), 8 worker threads,
and 64 kB direct buffers.

And here's my zoo.cfg:

tickTime=2000
initLimit=100
syncLimit=5
dataDir=/var/lib/zookeeper
clientPort=2181
autopurge.snapRetainCount=3
# Purge task interval in hours
autopurge.purgeInterval=1
minSessionTimeout=4000
maxSessionTimeout=60000
standaloneEnabled=false
reconfigEnabled=true
skipACL=yes
dynamicConfigFile=/opt/zookeeper/conf/zoo.cfg.dynamic

*However, the problem is not removal of the ephemeral nodes, it's that
they're not getting created in the first place,*  When I use the CLI to
create an ephemeral node via:

create -e /testnode

It's not marked as an ephemeral node and hangs around forever.  From stat
/testnode:

ephemeralOwner = 0x0

Sometimes Zookeeper will come up and create ephemeral nodes and they work
correctly (deleted after the connection is terminated).  But usually (with
the version I'm using- 3.5.3-beta2) they are not created.

Are you familiar with the code?  I've downloaded the source and have been
looking for the place where the znodes are actually created but have not
found that section.  If you can point me to the right area of the code I
can probably troubleshoot this issue.

Thanks for any light you can shed on this problem!

Mark


On Tue, Sep 19, 2017 at 6:18 PM, Abraham Fine <afine@apache.org> wrote:

> Hi Mark-
>
> I'm assuming you are using nio and not netty, please let me know if that
> is not the case.
>
> Have you possibly set zookeeper.nio.sessionlessCnxnTimeout? This value
> is 10s by default, so it may take a little more than ten seconds by
> default for ephemeral nodes to be deleted.
>
> Thanks,
> Abe
>
>
> On Tue, Sep 19, 2017, at 08:16, Mark Johnson wrote:
> > Hi,
> >
> > I'm running version 3.5.3-BETA of Zookeeper in a Docker container and
> > usually cannot create Ephemeral znodes, either through the CLI or though
> > a Java program.  It appears the owner is not set.  Here's an example
> > session:
> >
> > ============================================================
> ================================================
> > [root@Appliance ~]# docker exec -it 9b7cd1e80b5b /bin/bash
> > zookeeper@9b7cd1e80b5b:/opt/zookeeper-3.5.3-beta$ bin/zkCli.sh
> > Connecting to localhost:2181
> > 2017-09-19 10:03:31,373 [myid:] - INFO  [main:Environment@109] - Client
> > environment:zookeeper.version=3.5.3-beta-8ce24f9e675cbefffb8f21a47e06b4
> 2864475a60,
> > built on 04/03/2017 16:19 GMT
> > 2017-09-19 10:03:31,378 [myid:] - INFO  [main:Environment@109] - Client
> > environment:host.name=9b7cd1e80b5b
> > 2017-09-19 10:03:31,378 [myid:] - INFO  [main:Environment@109] - Client
> > environment:java.version=1.8.0_144
> > 2017-09-19 10:03:31,380 [myid:] - INFO  [main:Environment@109] - Client
> > environment:java.vendor=Oracle Corporation
> > 2017-09-19 10:03:31,380 [myid:] - INFO  [main:Environment@109] - Client
> > environment:java.home=/usr/lib/jvm/java-8-oracle/jre
> > 2017-09-19 10:03:31,380 [myid:] - INFO  [main:Environment@109] - Client
> > environment:java.class.path=/opt/zookeeper-3.5.3-beta/bin/.
> ./build/classes:/opt/zookeeper-3.5.3-beta/bin/../build/lib/*.jar:/opt/
> zookeeper-3.5.3-beta/bin/../lib/slf4j-log4j12-1.7.5.jar
> > :/opt/zookeeper-3.5.3-beta/bin/../lib/slf4j-api-1.7.5.
> jar:/opt/zookeeper-3.5.3-beta/bin/../lib/netty-3.10.5.Final.
> jar:/opt/zookeeper-3.5.3-beta/bin/../lib/log4j-1.2.17.jar:/
> opt/zookeeper-3.5.3-beta/bin/../lib/jline-2.11.jar:/opt/
> zookeeper-3.5.3-beta/bin/..
> > /lib/jetty-util-9.2.18.v20160721.jar:/opt/zookeeper-
> 3.5.3-beta/bin/../lib/jetty-servlet-9.2.18.v20160721.jar:/
> opt/zookeeper-3.5.3-beta/bin/../lib/jetty-server-9.2.18.
> v20160721.jar:/opt/zookeeper-3.5.3-beta/bin/../lib/jetty-
> security-9.2.18.v20160721.jar:/op
> > t/zookeeper-3.5.3-beta/bin/../lib/jetty-io-9.2.18.v20160721.
> jar:/opt/zookeeper-3.5.3-beta/bin/../lib/jetty-http-9.2.18.
> v20160721.jar:/opt/zookeeper-3.5.3-beta/bin/../lib/javax.
> servlet-api-3.1.0.jar:/opt/zookeeper-3.5.3-beta/bin/../
> lib/jackson-mapper-asl-1.
> > 9.11.jar:/opt/zookeeper-3.5.3-beta/bin/../lib/jackson-core-
> asl-1.9.11.jar:/opt/zookeeper-3.5.3-beta/bin/../lib/commons-
> cli-1.2.jar:/opt/zookeeper-3.5.3-beta/bin/../zookeeper-3.5.
> 3-beta.jar:/opt/zookeeper-3.5.3-beta/bin/../src/java/lib/*.
> jar:/opt/zookeeper-
> > 3.5.3-beta/bin/../conf:
> > 2017-09-19 10:03:31,380 [myid:] - INFO  [main:Environment@109] - Client
> > environment:java.library.path=/usr/java/packages/lib/amd64:/
> usr/lib64:/lib64:/lib:/usr/lib
> > 2017-09-19 10:03:31,380 [myid:] - INFO  [main:Environment@109] - Client
> > environment:java.io.tmpdir=/tmp
> > 2017-09-19 10:03:31,380 [myid:] - INFO  [main:Environment@109] - Client
> > environment:java.compiler=<NA>
> > 2017-09-19 10:03:31,380 [myid:] - INFO  [main:Environment@109] - Client
> > environment:os.name=Linux
> > 2017-09-19 10:03:31,380 [myid:] - INFO  [main:Environment@109] - Client
> > environment:os.arch=amd64
> > 2017-09-19 10:03:31,381 [myid:] - INFO  [main:Environment@109] - Client
> > environment:os.version=3.10.0-514.26.2.el7.x86_64
> > 2017-09-19 10:03:31,381 [myid:] - INFO  [main:Environment@109] - Client
> > environment:user.name=zookeeper
> > 2017-09-19 10:03:31,381 [myid:] - INFO  [main:Environment@109] - Client
> > environment:user.home=/home/zookeeper
> > 2017-09-19 10:03:31,381 [myid:] - INFO  [main:Environment@109] - Client
> > environment:user.dir=/opt/zookeeper-3.5.3-beta
> > 2017-09-19 10:03:31,381 [myid:] - INFO  [main:Environment@109] - Client
> > environment:os.memory.free=233MB
> > 2017-09-19 10:03:31,382 [myid:] - INFO  [main:Environment@109] - Client
> > environment:os.memory.max=240MB
> > 2017-09-19 10:03:31,383 [myid:] - INFO  [main:Environment@109] - Client
> > environment:os.memory.total=240MB
> > 2017-09-19 10:03:31,386 [myid:] - INFO  [main:ZooKeeper@865] -
> > Initiating client connection, connectString=localhost:2181
> > sessionTimeout=30000
> > watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@28ba21f3
> > 2017-09-19 10:03:31,397 [myid:] - INFO  [main:ClientCnxnSocket@236] -
> > jute.maxbuffer value is 4194304 Bytes
> > 2017-09-19 10:03:31,410 [myid:localhost:2181] - INFO
> > [main-SendThread(localhost:2181):ClientCnxn$SendThread@1113] - Opening
> > socket connection to server localhost/127.0.0.1:2181. Will not attempt
> > to authenticate using SASL (unknown error)
> > Welcome to ZooKeeper!
> > JLine support is enabled
> > 2017-09-19 10:03:31,493 [myid:localhost:2181] - INFO
> > [main-SendThread(localhost:2181):ClientCnxn$SendThread@948] - Socket
> > connection established, initiating session, client: /127.0.0.1:44882,
> > server: localhost/127.0.0.1:2181
> > 2017-09-19 10:03:31,506 [myid:localhost:2181] - INFO
> > [main-SendThread(localhost:2181):ClientCnxn$SendThread@1381] - Session
> > establishment complete on server localhost/127.0.0.1:2181, sessionid =
> > 0xeb0000480b3c005c, negotiated timeout = 30000
> >
> > WATCHER::
> >
> > WatchedEvent state:SyncConnected type:None path:null
> > [zk: localhost:2181(CONNECTED) 0] create -e /testnode
> > Created /testnode
> > [zk: localhost:2181(CONNECTED) 1] stat /testnode
> > cZxid = 0x2000002c4
> > ctime = Tue Sep 19 10:03:50 CDT 2017
> > mZxid = 0x2000002c4
> > mtime = Tue Sep 19 10:03:50 CDT 2017
> > pZxid = 0x2000002c4
> > cversion = 0
> > dataVersion = 0
> > aclVersion = 0
> > *ephemeralOwner = 0x0*
> > dataLength = 0
> > numChildren = 0
> > [zk: localhost:2181(CONNECTED) 2]
> > ============================================================
> ==============================================
> >
> > If I disconnect and reconnect, the znode is still present as well. If I
> > restart Zookeeper, many times this will resolve itself somehow and
> > creation of ephemeral nodes will work correctly.  I have also tried this
> > in a clustered configuration and get cases where one Zookeeper will
> > create ephemeral nodes correctly and another Zookeeper will exhibit the
> > above issue.
> >
> > Please let me know what other information would be of use or other
> > things I could be looking for.
> >
> > Any help appreciated.
> >
> > Mark
> >
> >
> >
>

--001a1146b324462f42055998efa8--