From user-return-11170-apmail-zookeeper-user-archive=zookeeper.apache.org@zookeeper.apache.org Tue Nov 28 21:55:05 2017 Return-Path: X-Original-To: apmail-zookeeper-user-archive@www.apache.org Delivered-To: apmail-zookeeper-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id F2E1C17756 for ; Tue, 28 Nov 2017 21:55:05 +0000 (UTC) Received: (qmail 16590 invoked by uid 500); 28 Nov 2017 21:55:05 -0000 Delivered-To: apmail-zookeeper-user-archive@zookeeper.apache.org Received: (qmail 16532 invoked by uid 500); 28 Nov 2017 21:55:05 -0000 Mailing-List: contact user-help@zookeeper.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@zookeeper.apache.org Delivered-To: mailing list user@zookeeper.apache.org Received: (qmail 16521 invoked by uid 99); 28 Nov 2017 21:55:05 -0000 Received: from mail-relay.apache.org (HELO mail-relay.apache.org) (140.211.11.15) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 28 Nov 2017 21:55:05 +0000 Received: from auth2-smtp.messagingengine.com (auth2-smtp.messagingengine.com [66.111.4.228]) by mail-relay.apache.org (ASF Mail Server at mail-relay.apache.org) with ESMTPSA id 5BE0C1A008C for ; Tue, 28 Nov 2017 21:55:03 +0000 (UTC) Received: from compute6.internal (compute6.nyi.internal [10.202.2.46]) by mailauth.nyi.internal (Postfix) with ESMTP id 76F5D209E6 for ; Tue, 28 Nov 2017 16:55:02 -0500 (EST) Received: from web1 ([10.202.2.211]) by compute6.internal (MEProxy); Tue, 28 Nov 2017 16:55:02 -0500 X-ME-Sender: Received: by mailuser.nyi.internal (Postfix, from userid 99) id 41C71940CA; Tue, 28 Nov 2017 16:55:02 -0500 (EST) Message-Id: <1511906102.2211622.1187388960.3BBE084A@webmail.messagingengine.com> From: Abraham Fine To: user@zookeeper.apache.org MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="utf-8" X-Mailer: MessagingEngine.com Webmail Interface - ajax-a169161c Subject: Re: Zookeeper client reverse lookup issue Date: Tue, 28 Nov 2017 16:55:02 -0500 References: <1510960212.3586213.1176415192.1DA76F7D@webmail.messagingengine.com> <1510960307.3586786.1176427824.786AF0AC@webmail.messagingengine.com> <1511210054.1371454.1178944576.38A23BC8@webmail.messagingengine.com> <1511302699.931307.1180343800.3AE7684E@webmail.messagingengine.com> In-Reply-To: Hi Ben- I'm not completely sure what you mean by "similar configuration gating" but I think a system property with a boolean switch would be the appropriate way of handling this. Take a look at " zookeeper.forceSync". Hopefully this helps. Thanks, Abe On Tue, Nov 21, 2017, at 20:06, Ben Wood wrote: > Definitely. > > Can you point me in the direction of similar configuration gating? > > On Tue, Nov 21, 2017 at 2:18 PM, Abraham Fine wrote: > > > I don't think a backport would be possible here (since existing > > deployments may break when upgraded within the 3.4 line). If a fix can > > be created without changing existing ZooKeeper behavior by default that > > would be much more likely to be considered. Perhaps in the form of a > > configuration switch of some kind. > > > > In short we need to consider this from two angles. 3.4 is broken because > > this is unexpected behavior and 3.5 is broken because ZOOKEEPER-2171 > > broke backwards compatibility. So I think we should add in a switch with > > opposite default values for 3.4 and 3.5. > > > > Would you be interested in submitting a patch for this Ben? > > > > Thanks, > > Abe > > > > On Mon, Nov 20, 2017, at 15:11, Ben Wood wrote: > > > Do you think a patch targeting 3.4.11 or 3.4.12 that backports this > > > lookup > > > change (as minimally as possible) would be accepted? > > > > > > In order to do ZK in containers (yes, slightly terrifying), it's not > > > _necessary_ but a lot easier! > > > > > > On Mon, Nov 20, 2017 at 12:34 PM, Abraham Fine wrote: > > > > > > > Hi Ben- > > > > > > > > Unfortunately 3.5 is still in beta and not recommended for production. > > > > > > > > Thanks, > > > > Abe > > > > > > > > On Mon, Nov 20, 2017, at 11:13, Ben Wood wrote: > > > > > Hey Abraham, > > > > > > > > > > We've been using 3.4.11.[ > > > > > > > > > > That's great news the reverse lookup has already been taken out in > > 3.5! > > > > > We > > > > > found another ticket https://issues.apache.org/ > > > > jira/browse/ZOOKEEPER-2858 > > > > > that suggested there were some workarounds but hadn't found that one > > (and > > > > > hadn't had a chance to test 3.5). > > > > > > > > > > This raises an interesting question for us, admitted ZK neophytes, > > is ZK > > > > > 3.5 "safe" to use in production? > > > > > > > > > > On Fri, Nov 17, 2017 at 3:11 PM, Abraham Fine > > wrote: > > > > > > > > > > > This change occurred due to > > > > > > https://issues.apache.org/jira/browse/ZOOKEEPER-2171 > > > > > > > > > > > > On Fri, Nov 17, 2017, at 15:10, Abraham Fine wrote: > > > > > > > Hi Ben- > > > > > > > > > > > > > > What version of ZooKeeper are you using? In my testing it looks > > like > > > > 3.4 > > > > > > > does a reverse lookup when creating the server principal > > > > > > > (https://github.com/apache/zookeeper/blob/branch-3.4/src/ > > > > > > java/main/org/apache/zookeeper/ClientCnxn.java#L1011) > > > > > > > but 3.5/master do not > > > > > > > (https://github.com/apache/zookeeper/blob/branch-3.5/src/ > > > > > > java/main/org/apache/zookeeper/ClientCnxn.java#L1104). > > > > > > > > > > > > > > Let me know if that helps. > > > > > > > > > > > > > > Thanks, > > > > > > > Abe > > > > > > > > > > > > > > On Fri, Nov 17, 2017, at 12:01, Ben Wood wrote: > > > > > > > > Hey Folks, > > > > > > > > > > > > > > > > My team and I are working on a containerized Zookeeper service > > on > > > > top > > > > > > of > > > > > > > > DC/OS. We're running into an issue with Kerberos in the > > following > > > > > > > > scenario. > > > > > > > > > > > > > > > > Simplified, we have a zk server with the DNS address > > zk-server.dcos > > > > > > (e.g. > > > > > > > > the dns address of the ZK task) and actual hostname > > zk-server.aws > > > > > > > > (Shortened here, but really a standard resolvable AWS private > > dns > > > > > > > > address) > > > > > > > > and a kafka broker, kafka.dcos. > > > > > > > > > > > > > > > > We can easily setup our Zookeeper and Kafka services to work > > > > together, > > > > > > > > until we try to enable Kerberos. ZK itself works just fine with > > > > > > Kerberos, > > > > > > > > but the Kafka broker is not able to connect to the ZK server: > > > > > > > > > > > > > > > > 0. kafka.dcos is started with a zk server list of > > zk-server.dcos. > > > > > > > > 1. kafka.dcos starts up, initializing its ZK client. > > > > > > > > 2. kafka.dcos then attempts to retrieve a ticket from the KDC > > in > > > > order > > > > > > to > > > > > > > > talk to zk-server.aws, however the only zk principal known to > > the > > > > kdc > > > > > > is > > > > > > > > zk-server.dcos. > > > > > > > > > > > > > > > > From reading the source ( > > > > > > > > https://github.com/apache/zookeeper/blob/master/src/ > > > > > > java/main/org/apache/zookeeper/client/ > > StaticHostProvider.java#L112) > > > > > > > > it appears that the zk client is winding up with the actual > > > > hostname of > > > > > > > > the > > > > > > > > ZK server. > > > > > > > > > > > > > > > > Being new to the codebase, is this because of a client reverse > > > > lookup? > > > > > > Or > > > > > > > > because the zk server is telling the client about its > > hostname? It > > > > > > > > appears > > > > > > > > to be the former. > > > > > > > > > > > > > > > > Thanks! > > > > > > > > Ben > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > Ben Wood > > > > > Software Engineer - Data Agility > > > > > Mesosphere > > > > > > > > > > > > > > > > -- > > > Ben Wood > > > Software Engineer - Data Agility > > > Mesosphere > > > > > > -- > Ben Wood > Software Engineer - Data Agility > Mesosphere