Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 9F672200B4F for ; Tue, 12 Jul 2016 01:01:50 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 9DEBD160A7D; Mon, 11 Jul 2016 23:01:50 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 68F2B160A78 for ; Tue, 12 Jul 2016 01:01:49 +0200 (CEST) Received: (qmail 34863 invoked by uid 500); 11 Jul 2016 23:01:48 -0000 Mailing-List: contact user-help@curator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@curator.apache.org Delivered-To: mailing list user@curator.apache.org Received: (qmail 34853 invoked by uid 99); 11 Jul 2016 23:01:48 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 11 Jul 2016 23:01:48 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id E91DFC72D4 for ; Mon, 11 Jul 2016 23:01:47 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.199 X-Spam-Level: * X-Spam-Status: No, score=1.199 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, WEIRD_PORT=0.001] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (1024-bit key) header.d=radiantlogic.com Received: from mx2-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id zX87EyutkpQc for ; Mon, 11 Jul 2016 23:01:45 +0000 (UTC) Received: from mail-oi0-f48.google.com (mail-oi0-f48.google.com [209.85.218.48]) by mx2-lw-eu.apache.org (ASF Mail Server at mx2-lw-eu.apache.org) with ESMTPS id E636B5F240 for ; Mon, 11 Jul 2016 23:01:44 +0000 (UTC) Received: by mail-oi0-f48.google.com with SMTP id w18so26697742oiw.3 for ; Mon, 11 Jul 2016 16:01:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=radiantlogic.com; s=google; h=mime-version:from:date:message-id:subject:to; bh=4Scu13DHHjwyGZXTsLjHuZzc9zBVYC30YwxZcBQXXPM=; b=GboTpE7vLtc8BlqTGKBS+K1XbQSIBecfHzJGpUXN9Tf5q89QSpTLOTENq114wJ66fm zVBEHpeGTeOezzFXZt+ktZKmuGxx7LwAcDmVu5iM8Ubb8a1lhTJ8c5F7pmfafPK99fUV 34WNI/op58t0HoNfgtHIVn4XwTO7V9qMB/tYU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=4Scu13DHHjwyGZXTsLjHuZzc9zBVYC30YwxZcBQXXPM=; b=Fw1dJxgM7+YPkvXX9oG9WZJaNyMEBFy6gXLvYhi+nGU1a6BJeDqyz5B8sEO6TpKkzt v+iEJm6kJrBfBVSbzt3d1bxxJE5gvwUqxPO0WVmTWBRu+bVOWWuIBHW5rZZ04Ah0Lyaj 4akClstgFJcF64h0Oe0WzreyxWUCIJDRKZqA1xUlJz4xsCmJyEbUbM33DM+DnxLBKqA3 18bLAOwXqSVNH+Jbvh4mEXV3NeOEiz8ey5bhxZcpNGVFtpn+hHB06sDPX8bQ8OAPPRV8 HM3dg+YahU0z656jKds00OXpKTW3g+VGvpka5R+eDd4ttVn81PfLteKu/VUSZ2LobiWB i8/Q== X-Gm-Message-State: ALyK8tLC/E6AlMjPRzk8XtkUX2oJ7dBVzvJJu5DPjhpDxxc/0Opy3txVZBD+L+ZGgqEBz/gTkyeyRVR4L6aG14YkAK3ZnkN+Ovj4tvhYxkriEf434hgMeDj8cR40Ng0W4aolNGmujlNbnwCU1kpU5zfjJDGKWJ4qdpzWfiX6DOsRw/rR7R0iei3u1KR3bEJsukA793jkFGyUO9TIG9lEipfuPuJdArP6VT9NERJHc2CSa0cv X-Received: by 10.157.6.3 with SMTP id 3mr12244524otn.143.1468278098287; Mon, 11 Jul 2016 16:01:38 -0700 (PDT) MIME-Version: 1.0 Received: by 10.202.74.199 with HTTP; Mon, 11 Jul 2016 16:01:37 -0700 (PDT) From: Benjamin Jaton Date: Mon, 11 Jul 2016 16:01:37 -0700 Message-ID: Subject: Curator never goes through when ZK is read-only To: "user@curator.apache.org" Content-Type: multipart/alternative; boundary=94eb2c0924ba7b03c005376422dd archived-at: Mon, 11 Jul 2016 23:01:50 -0000 --94eb2c0924ba7b03c005376422dd Content-Type: text/plain; charset=UTF-8 I have a 3 nodes ZK cluster , readonly mode is enabled. 2 nodes are down, so the third one is in read-only (verified by using the ZK API manually). I'm using this piece of code: Builder curatorClientBuilder = CuratorFrameworkFactory.builder() .connectString("QA-E8WIN11:2181,QA-E8WIN12:2181") .sessionTimeoutMs(45000).connectionTimeoutMs(15000) .retryPolicy(new RetryNTimes(3, 5000)).canBeReadOnly(true); CuratorFramework client = curatorClientBuilder.build(); client.start(); client.getZookeeperClient().blockUntilConnectedOrTimedOut(); System.out.println("Successfully established the connection with ZooKeeper"); client.getData().forPath("/"); System.out.println("Done."); When curator pick the host that is UP first, it goes through very quickly. When it picks the host that is down first (QA-E8WIN12), it never goes through. Am I missing something here? Below is the log: 2016-07-11 15:46:45,733 INFO org.apache.curator.framework.imps.CuratorFrameworkImpl:235 - Starting 2016-07-11 15:46:45,743 INFO org.apache.zookeeper.ZooKeeper:109 - Client environment:zookeeper.version=3.5.1-alpha-1748895, built on 06/17/2016 22:24 GMT 2016-07-11 15:46:45,743 INFO org.apache.zookeeper.ZooKeeper:109 - Client environment:host.name=w-rli09-ben 2016-07-11 15:46:45,744 INFO org.apache.zookeeper.ZooKeeper:109 - Client environment:java.version=1.8.0_11 2016-07-11 15:46:45,744 INFO org.apache.zookeeper.ZooKeeper:109 - Client environment:java.vendor=Oracle Corporation 2016-07-11 15:46:45,744 INFO org.apache.zookeeper.ZooKeeper:109 - Client environment:java.home=/usr/local/apps/jdk1.8.0_11/jre 2016-07-11 15:46:45,744 INFO org.apache.zookeeper.ZooKeeper:109 - Client environment:java.class.path=(..) 2016-07-11 15:46:45,746 INFO org.apache.zookeeper.ZooKeeper:109 - Client environment:java.io.tmpdir=/tmp 2016-07-11 15:46:45,746 INFO org.apache.zookeeper.ZooKeeper:109 - Client environment:java.compiler= 2016-07-11 15:46:45,747 INFO org.apache.zookeeper.ZooKeeper:109 - Client environment:os.name=Linux 2016-07-11 15:46:45,747 INFO org.apache.zookeeper.ZooKeeper:109 - Client environment:os.arch=amd64 2016-07-11 15:46:45,747 INFO org.apache.zookeeper.ZooKeeper:109 - Client environment:os.version=3.8.0-44-generic 2016-07-11 15:46:45,747 INFO org.apache.zookeeper.ZooKeeper:109 - Client environment:user.name=benji 2016-07-11 15:46:45,747 INFO org.apache.zookeeper.ZooKeeper:109 - Client environment:user.home=(..) 2016-07-11 15:46:45,747 INFO org.apache.zookeeper.ZooKeeper:109 - Client environment:user.dir=(..) 2016-07-11 15:46:45,747 INFO org.apache.zookeeper.ZooKeeper:109 - Client environment:os.memory.free=385MB 2016-07-11 15:46:45,748 INFO org.apache.zookeeper.ZooKeeper:109 - Client environment:os.memory.max=5358MB 2016-07-11 15:46:45,748 INFO org.apache.zookeeper.ZooKeeper:109 - Client environment:os.memory.total=397MB 2016-07-11 15:46:45,750 INFO org.apache.zookeeper.ZooKeeper:716 - Initiating client connection, connectString=QA-E8WIN11:2181,QA-E8WIN12:2181 sessionTimeout=45000 watcher=org.apache.curator.ConnectionState@3e96bacf 2016-07-11 15:46:45,769 INFO org.apache.zookeeper.ClientCnxn:1138 - Opening socket connection to server QA-E8WIN12/10.11.8.232:2181. Will not attempt to authenticate using SASL (unknown error) Successfully established the connection with ZooKeeper 2016-07-11 15:47:08,300 INFO org.apache.zookeeper.ClientCnxn:1251 - Client session timed out, have not heard from server in 22534ms for sessionid 0x0, closing socket connection and attempting reconnect 2016-07-11 15:47:08,403 INFO org.apache.zookeeper.ClientCnxn:1138 - Opening socket connection to server QA-E8WIN11/10.11.8.236:2181. Will not attempt to authenticate using SASL (unknown error) 2016-07-11 15:47:08,406 INFO org.apache.zookeeper.ClientCnxn:980 - Socket connection established, initiating session, client: /10.11.12.4:47004, server: QA-E8WIN11/10.11.8.236:2181 2016-07-11 15:47:08,413 INFO org.apache.zookeeper.ClientCnxn:1400 - Session establishment complete on server QA-E8WIN11/10.11.8.236:2181, sessionid = 0x100024f0936005b, negotiated timeout = 45000 (READ-ONLY mode) 2016-07-11 15:47:08,415 INFO org.apache.zookeeper.ClientCnxn:1297 - Checking server QA-E8WIN12/10.11.8.232:2181 for being r/w. Timeout 200 2016-07-11 15:47:08,420 INFO org.apache.curator.framework.state.ConnectionStateManager:228 - State change: READ_ONLY 2016-07-11 15:49:15,712 INFO org.apache.zookeeper.ClientCnxn:1253 - Unable to read additional data from server sessionid 0x100024f0936005b, likely server has closed socket, closing socket connection and attempting reconnect 2016-07-11 15:49:15,815 INFO org.apache.curator.framework.state.ConnectionStateManager:228 - State change: SUSPENDED 2016-07-11 15:49:17,313 INFO org.apache.zookeeper.ClientCnxn:1138 - Opening socket connection to server QA-E8WIN11/10.11.8.236:2181. Will not attempt to authenticate using SASL (unknown error) 2016-07-11 15:49:17,314 INFO org.apache.zookeeper.ClientCnxn:980 - Socket connection established, initiating session, client: /10.11.12.4:47295, server: QA-E8WIN11/10.11.8.236:2181 2016-07-11 15:49:17,316 INFO org.apache.zookeeper.ClientCnxn:1400 - Session establishment complete on server QA-E8WIN11/10.11.8.236:2181, sessionid = 0x100024f0936005f, negotiated timeout = 45000 (READ-ONLY mode) 2016-07-11 15:49:17,316 INFO org.apache.zookeeper.ClientCnxn:1297 - Checking server QA-E8WIN12/10.11.8.232:2181 for being r/w. Timeout 400 2016-07-11 15:49:17,316 INFO org.apache.curator.framework.state.ConnectionStateManager:228 - State change: READ_ONLY 2016-07-11 15:51:24,607 INFO org.apache.zookeeper.ClientCnxn:1253 - Unable to read additional data from server sessionid 0x100024f0936005f, likely server has closed socket, closing socket connection and attempting reconnect 2016-07-11 15:51:24,713 INFO org.apache.curator.framework.state.ConnectionStateManager:228 - State change: SUSPENDED 2016-07-11 15:51:26,467 INFO org.apache.zookeeper.ClientCnxn:1138 - Opening socket connection to server QA-E8WIN11/10.11.8.236:2181. Will not attempt to authenticate using SASL (unknown error) 2016-07-11 15:51:26,468 INFO org.apache.zookeeper.ClientCnxn:980 - Socket connection established, initiating session, client: /10.11.12.4:47576, server: QA-E8WIN11/10.11.8.236:2181 2016-07-11 15:51:26,470 INFO org.apache.zookeeper.ClientCnxn:1400 - Session establishment complete on server QA-E8WIN11/10.11.8.236:2181, sessionid = 0x100024f09360061, negotiated timeout = 45000 (READ-ONLY mode) 2016-07-11 15:51:26,470 INFO org.apache.zookeeper.ClientCnxn:1297 - Checking server QA-E8WIN12/10.11.8.232:2181 for being r/w. Timeout 800 2016-07-11 15:51:26,470 INFO org.apache.curator.framework.state.ConnectionStateManager:228 - State change: READ_ONLY (...) Thanks Ben --94eb2c0924ba7b03c005376422dd Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
I have a 3 nodes ZK cluster , readonly mode is enabled.2 nodes are down, so the third one is in read-only (verified by using the= ZK API manually).

I'm using this piece of cod= e:

Builder curatorClientBuilder =3D CuratorFrameworkFactory.builder(= )
.connectS= tring("QA-E8WIN11:2181,QA-E8WIN12:2181")
.sessionTimeoutMs(45000).connection= TimeoutMs(15000)
<= /span>.retryPolicy(new RetryNTimes(3, 5000)).canBeReadOnly(true);

Curator= Framework client =3D curatorClientBuilder.build();
client.start();
client.getZookeeperClient().blockU= ntilConnectedOrTimedOut();
System.out.println("Successfully established the connect= ion with ZooKeeper");
client.getData().forPath("/");
System.out.println("Done.");
<= /div>

When curator pick the host that is UP first, it go= es through very quickly. When it picks the host that is down first (QA-E8WI= N12), it never goes through.

Am I missing somethin= g here?

Below is the log:

2016-07-11 15:46:45,733 INFO =C2=A0org.apache.curator.framework.imps.= CuratorFrameworkImpl:235 - Starting
2016-07-11 15:46:45,743 INFO = =C2=A0org.apache.zookeeper.ZooKeeper:109 - Client environment:zookeeper.ver= sion=3D3.5.1-alpha-1748895, built on 06/17/2016 22:24 GMT
2016-07= -11 15:46:45,743 INFO =C2=A0org.apache.zookeeper.ZooKeeper:109 - Client env= ironment:host.name=3Dw-rli09-ben
2016-07-11 15:46:45,744 INFO =C2=A0org.apache.zookeeper.ZooKeeper:109 - Cl= ient environment:java.version=3D1.8.0_11
2016-07-11 15:46:45,744 = INFO =C2=A0org.apache.zookeeper.ZooKeeper:109 - Client environment:java.ven= dor=3DOracle Corporation
2016-07-11 15:46:45,744 INFO =C2=A0org.a= pache.zookeeper.ZooKeeper:109 - Client environment:java.home=3D/usr/local/a= pps/jdk1.8.0_11/jre
2016-07-11 15:46:45,744 INFO =C2=A0org.apache= .zookeeper.ZooKeeper:109 - Client environment:java.class.path=3D(..)
<= div>2016-07-11 15:46:45,746 INFO =C2=A0org.apache.zookeeper.ZooKeeper:109 -= Client environment:java.io.tmpdir=3D/tmp
2016-07-11 15:46:45,746= INFO =C2=A0org.apache.zookeeper.ZooKeeper:109 - Client environment:java.co= mpiler=3D<NA>
2016-07-11 15:46:45,747 INFO =C2=A0org.apache= .zookeeper.ZooKeeper:109 - Client environment:os= .name=3DLinux
2016-07-11 15:46:45,747 INFO =C2=A0org.apache.z= ookeeper.ZooKeeper:109 - Client environment:os.arch=3Damd64
2016-= 07-11 15:46:45,747 INFO =C2=A0org.apache.zookeeper.ZooKeeper:109 - Client e= nvironment:os.version=3D3.8.0-44-generic
2016-07-11 15:46:45,747 = INFO =C2=A0org.apache.zookeeper.ZooKeeper:109 - Client environment:user.name=3Dbenji
2016-07-11 15:46:45,7= 47 INFO =C2=A0org.apache.zookeeper.ZooKeeper:109 - Client environment:user.= home=3D(..)
2016-07-11 15:46:45,747 INFO =C2=A0org.apache.zookeep= er.ZooKeeper:109 - Client environment:user.dir=3D(..)
2016-07-11 = 15:46:45,747 INFO =C2=A0org.apache.zookeeper.ZooKeeper:109 - Client environ= ment:os.memory.free=3D385MB
2016-07-11 15:46:45,748 INFO =C2=A0or= g.apache.zookeeper.ZooKeeper:109 - Client environment:os.memory.max=3D5358M= B
2016-07-11 15:46:45,748 INFO =C2=A0org.apache.zookeeper.ZooKeep= er:109 - Client environment:os.memory.total=3D397MB
2016-07-11 15= :46:45,750 INFO =C2=A0org.apache.zookeeper.ZooKeeper:716 - Initiating clien= t connection, connectString=3DQA-E8WIN11:2181,QA-E8WIN12:2181 sessionTimeou= t=3D45000 watcher=3Dorg.apache.curator.ConnectionState@3e96bacf
2= 016-07-11 15:46:45,769 INFO =C2=A0org.apache.zookeeper.ClientCnxn:1138 - Op= ening socket connection to server QA-E8WIN12/10.11.8.232:2181. Will not attempt to authenticate using SASL (un= known error)
Successfully established the connection with ZooKeep= er
2016-07-11 15:47:08,300 INFO =C2=A0org.apache.zookeeper.Client= Cnxn:1251 - Client session timed out, have not heard from server in 22534ms= for sessionid 0x0, closing socket connection and attempting reconnect
2016-07-11 15:47:08,403 INFO =C2=A0org.apache.zookeeper.ClientCnxn:11= 38 - Opening socket connection to server QA-E8WIN11/10.11.8.236:2181. Will not attempt to authenticate using S= ASL (unknown error)
2016-07-11 15:47:08,406 INFO =C2=A0org.apache= .zookeeper.ClientCnxn:980 - Socket connection established, initiating sessi= on, client: /10.11.12.4:47004, serv= er: QA-E8WIN11/10.11.8.236:2181
2016-07-11 15:47:08,413 INFO =C2=A0org.apache.zookeeper.ClientCnxn:1= 400 - Session establishment complete on server QA-E8WIN11/10.11.8.236:2181, sessionid =3D 0x100024f0936005b, n= egotiated timeout =3D 45000 (READ-ONLY mode)
2016-07-11 15:47:08,= 415 INFO =C2=A0org.apache.zookeeper.ClientCnxn:1297 - Checking server QA-E8= WIN12/10.11.8.232:2181 for being r/= w. Timeout 200
2016-07-11 15:47:08,420 INFO =C2=A0org.apache.cura= tor.framework.state.ConnectionStateManager:228 - State change: READ_ONLY
2016-07-11 15:49:15,712 INFO =C2=A0org.apache.zookeeper.ClientCnxn:= 1253 - Unable to read additional data from server sessionid 0x100024f093600= 5b, likely server has closed socket, closing socket connection and attempti= ng reconnect
2016-07-11 15:49:15,815 INFO =C2=A0org.apache.curato= r.framework.state.ConnectionStateManager:228 - State change: SUSPENDED
2016-07-11 15:49:17,313 INFO =C2=A0org.apache.zookeeper.ClientCnxn:11= 38 - Opening socket connection to server QA-E8WIN11/10.11.8.236:2181. Will not attempt to authenticate using S= ASL (unknown error)
2016-07-11 15:49:17,314 INFO =C2=A0org.apache= .zookeeper.ClientCnxn:980 - Socket connection established, initiating sessi= on, client: /10.11.12.4:47295, serv= er: QA-E8WIN11/10.11.8.236:2181
2016-07-11 15:49:17,316 INFO =C2=A0org.apache.zookeeper.ClientCnxn:1= 400 - Session establishment complete on server QA-E8WIN11/10.11.8.236:2181, sessionid =3D 0x100024f0936005f, n= egotiated timeout =3D 45000 (READ-ONLY mode)
2016-07-11 15:49:17,= 316 INFO =C2=A0org.apache.zookeeper.ClientCnxn:1297 - Checking server QA-E8= WIN12/10.11.8.232:2181 for being r/= w. Timeout 400
2016-07-11 15:49:17,316 INFO =C2=A0org.apache.cura= tor.framework.state.ConnectionStateManager:228 - State change: READ_ONLY
2016-07-11 15:51:24,607 INFO =C2=A0org.apache.zookeeper.ClientCnxn:= 1253 - Unable to read additional data from server sessionid 0x100024f093600= 5f, likely server has closed socket, closing socket connection and attempti= ng reconnect
2016-07-11 15:51:24,713 INFO =C2=A0org.apache.curato= r.framework.state.ConnectionStateManager:228 - State change: SUSPENDED
2016-07-11 15:51:26,467 INFO =C2=A0org.apache.zookeeper.ClientCnxn:11= 38 - Opening socket connection to server QA-E8WIN11/10.11.8.236:2181. Will not attempt to authenticate using S= ASL (unknown error)
2016-07-11 15:51:26,468 INFO =C2=A0org.apache= .zookeeper.ClientCnxn:980 - Socket connection established, initiating sessi= on, client: /10.11.12.4:47576, serv= er: QA-E8WIN11/10.11.8.236:2181
2016-07-11 15:51:26,470 INFO =C2=A0org.apache.zookeeper.ClientCnxn:1= 400 - Session establishment complete on server QA-E8WIN11/10.11.8.236:2181, sessionid =3D 0x100024f09360061, n= egotiated timeout =3D 45000 (READ-ONLY mode)
2016-07-11 15:51:26,= 470 INFO =C2=A0org.apache.zookeeper.ClientCnxn:1297 - Checking server QA-E8= WIN12/10.11.8.232:2181 for being r/= w. Timeout 800
2016-07-11 15:51:26,470 INFO =C2=A0org.apache.cura= tor.framework.state.ConnectionStateManager:228 - State change: READ_ONLY
(...)

Thanks
Ben

--94eb2c0924ba7b03c005376422dd--