From user-return-12458-archive-asf-public=cust-asf.ponee.io@zookeeper.apache.org Tue Dec 17 17:58:41 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id 720DB180607 for ; Tue, 17 Dec 2019 18:58:41 +0100 (CET) Received: (qmail 2955 invoked by uid 500); 17 Dec 2019 17:58:40 -0000 Mailing-List: contact user-help@zookeeper.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@zookeeper.apache.org Delivered-To: mailing list user@zookeeper.apache.org Received: (qmail 2943 invoked by uid 99); 17 Dec 2019 17:58:40 -0000 Received: from Unknown (HELO mailrelay1-lw-us.apache.org) (10.10.3.159) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 17 Dec 2019 17:58:40 +0000 Received: from [192.168.1.28] (unknown [84.2.28.26]) by mailrelay1-lw-us.apache.org (ASF Mail Server at mailrelay1-lw-us.apache.org) with ESMTPSA id A5BE010FC for ; Tue, 17 Dec 2019 17:58:39 +0000 (UTC) From: Andor Molnar Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\)) Subject: Re: Zookeeper 3.5 SSL and Kerberos authentication Date: Tue, 17 Dec 2019 18:58:37 +0100 References: <7A3AAC00-4CD8-4197-B64C-6FDCB08DCD4E@gmail.com> <51ACAD67-4CB5-4CF3-B6E4-0A0B25926AE1@apache.org> To: user@zookeeper.apache.org In-Reply-To: Message-Id: <4DE84E3F-A02E-4359-9CAC-21810D3EA5DA@apache.org> X-Mailer: Apple Mail (2.3445.104.11) "We were using early 3.5.3 or something like that.=E2=80=9D Netty stack had a major refactor in 3.5.5 Andor > On 2019. Dec 17., at 16:40, Enrico Olivelli = wrote: >=20 > Il giorno mar 17 dic 2019 alle ore 16:26 Szalay-Bek=C5=91 M=C3=A1t=C3=A9= < > szalay.beko.mate@gmail.com> ha scritto: >=20 >> I added a comment on Jira. This is something we will also need to fix = in my >> company soon. >>=20 >> @enrico you wrote: >>> in my company we set up some ZK with TLS and SASL, using TLS for >> encryption and SASL for auth. We were using early 3.5.3 or something = like >> that. >>=20 >=20 > Unfortunately we do not have that setup anymore, we had to drop it = because > at that time (and still nowadays) from the same JVMs we had also to = connect > to an HBase cluster with ZK 3.4 > that does not support TLS. >=20 > Currently we are using only SASL and not TLS > Sorry >=20 > Enrico >=20 >=20 >>=20 >> According to this, the scenario should work. Maybe we just = misconfigured >> something, or this was something got broken in a later version? Can = you >> share the config you use? Maybe you are setting = `zookeeper.ssl.clientAuth` >> and `zookeeper.ssl.quorum.clientAuth` to `none` or `want` ? >>=20 >> Regards, >> Mate >>=20 >> On Tue, Dec 17, 2019 at 10:48 AM Andor Molnar = wrote: >>=20 >>> Hi Jorn, >>>=20 >>> Sorry for coming back late to this. I=E2=80=99ve just validated the = scenario on >> my >>> test cluster. Looks like the issue is valid: Kerberos auth and SSL = are >>> mutually exclusive currently. When Kerberos is set up and trying to >> connect >>> to secure port I got an infinite loop on client side: >>>=20 >>> 2019-12-17 01:43:30,984 [myid:barbaresco-1.vpc.cloudera.com:2182] - = WARN >>> [Thread-39:Login$1@197] - TGT renewal thread has been interrupted = and >>> will exit. >>> 2019-12-17 01:43:30,987 [myid:barbaresco-1.vpc.cloudera.com:2182] - = INFO >>> [main-SendThread(barbaresco-1.vpc.cloudera.com:2182):Login@302] - = Client >>> successfully logged in. >>> 2019-12-17 01:43:30,987 [myid:barbaresco-1.vpc.cloudera.com:2182] - = INFO >>> [Thread-40:Login$1@135] - TGT refresh thread started. >>> 2019-12-17 01:43:30,987 [myid:barbaresco-1.vpc.cloudera.com:2182] - = INFO >>> = [main-SendThread(barbaresco-1.vpc.cloudera.com:2182):SecurityUtils$1@124 >> ] >>> - Client will use GSSAPI as SASL mechanism. >>> 2019-12-17 01:43:30,988 [myid:barbaresco-1.vpc.cloudera.com:2182] - = INFO >>> [main-SendThread(barbaresco-1.vpc.cloudera.com:2182 >>> ):ClientCnxn$SendThread@1112] - Opening socket connection to server >>> barbaresco-1.vpc.cloudera.com/10.65.25.98:2182. Will attempt to >>> SASL-authenticate using Login Context section 'Client' >>> 2019-12-17 01:43:30,988 [myid:barbaresco-1.vpc.cloudera.com:2182] - = INFO >>> [main-SendThread(barbaresco-1.vpc.cloudera.com:2182 >>> ):ClientCnxn$SendThread@959] - Socket connection established, = initiating >>> session, client: /10.65.25.98:45362, server: >>> barbaresco-1.vpc.cloudera.com/10.65.25.98:2182 >>> 2019-12-17 >>> >>> 01:43:30,989 [myid:barbaresco-1.vpc.cloudera.com:2182] - INFO >>> [Thread-40:Login@320] - TGT valid starting at: Tue Dec 17 >> 01:43:30 >>> PST 2019 >>> 2019-12-17 01:43:30,989 [myid:barbaresco-1.vpc.cloudera.com:2182] - = INFO >>> [Thread-40:Login@321] - TGT expires: Thu Jan 16 >> 01:43:30 >>> PST 2020 >>> 2019-12-17 01:43:30,989 [myid:barbaresco-1.vpc.cloudera.com:2182] - = INFO >>> [Thread-40:Login$1@193] - TGT refresh sleeping until: Fri Jan 10 >> 20:23:33 >>> PST 2020 >>> 2019-12-17 01:43:30,989 [myid:barbaresco-1.vpc.cloudera.com:2182] - = INFO >>> [main-SendThread(barbaresco-1.vpc.cloudera.com:2182 >>> ):ClientCnxn$SendThread@1240] - Unable to read additional data from >>> server sessionid 0x0, likely server has closed socket, closing = socket >>> connection and attempting reconnect >>>=20 >>> And the following error on server side: >>>=20 >>> 2019-12-17 01:43:33,002 INFO >>> org.apache.zookeeper.server.NettyServerCnxnFactory: SSL handler = added for >>> channel: [id: 0xcf37c14b, L:/10.65.25.98:2182 - = R:/10.65.25.98:45380] >>> 2019-12-17 01:43:33,003 ERROR >>> org.apache.zookeeper.server.NettyServerCnxnFactory: Unsuccessful >> handshake >>> with session 0x0 >>> 2019-12-17 01:43:33,003 WARN >>> org.apache.zookeeper.server.NettyServerCnxnFactory: Exception caught >>> io.netty.handler.codec.DecoderException: >>> io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS record: >>>=20 >> = 0000002d000000000000000000000000000075300000000000000000000000100000000000= 000000000000000000000000 >>> at >>>=20 >> = io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecode= r.java:475) >>> at >>>=20 >> = io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecod= er.java:283) >>> at >>>=20 >> = io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractC= hannelHandlerContext.java:374) >>> at >>>=20 >> = io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractC= hannelHandlerContext.java:360) >>> at >>>=20 >> = io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractCha= nnelHandlerContext.java:352) >>> at >>>=20 >> = io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultCha= nnelPipeline.java:1422) >>> at >>>=20 >> = io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractC= hannelHandlerContext.java:374) >>> at >>>=20 >> = io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractC= hannelHandlerContext.java:360) >>> at >>>=20 >> = io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipe= line.java:931) >>> at >>>=20 >> = io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollI= nReady(AbstractEpollStreamChannel.java:792) >>> at >>>=20 >> = io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:483= ) >>> at >>> io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:383) >>> at >>>=20 >> = io.netty.util.concurrent.SingleThreadEventExecutor$6.run(SingleThreadEvent= Executor.java:1044) >>> at >>> = io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) >>> at >>>=20 >> = io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnab= le.java:30) >>> at java.lang.Thread.run(Thread.java:748) >>>=20 >>> I will update the Jira too. >>>=20 >>> Andor >>>=20 >>>=20 >>>=20 >>>=20 >>>=20 >>>> On 2019. Nov 8., at 20:31, J=C3=B6rn Franke = wrote: >>>>=20 >>>> Thanks. Can you please share the configuration file? >>>>=20 >>>> I tried with 3.5.5 - without SSL Kerberos works, but once I = configured >>> client ssl it said authentication fail (I have to check if I can dig = up >> the >>> log files) and as far as I remember this was related to x509 >>> authentication. The certificate and truststore themselves are fine = (I >> think >>> I needed to convert the truststore to jks). >>>> Sorry it was some time ago, I should have separated the log files. >>>> For me it did not matter that the ports are separated, but it = worked on >>> the non-ssl port fine. >>>>=20 >>>>> Am 06.11.2019 um 23:08 schrieb Enrico Olivelli = : >>>>>=20 >>>>> =EF=BB=BFJorn, >>>>> IIRC in my company we set up some ZK with TLS and SASL, using TLS = for >>>>> encryption and SASL for auth. >>>>> We were using early 3.5.3 or something like that. >>>>>=20 >>>>> Do you have a specific error? >>>>>=20 >>>>> I can also add that in 3.6.0 we will have port-unification, this = way >> you >>>>> can configure only one client port and accept plain text and TLS >>> connection >>>>> from clients (this helps the ttransition to TLS) >>>>>=20 >>>>> Enrico >>>>>=20 >>>>> Il mer 6 nov 2019, 22:28 J=C3=B6rn Franke = ha >> scritto: >>>>>=20 >>>>>> Dear all, >>>>>>=20 >>>>>> it seems that ZooKeeper 3.5 with SSL enabled does not support >> Kerberos >>>>>> authentication, but only X509 authentication. Kerberos is used in >> many >>>>>> Enterprise environments and is supported by Apache Solr. Is this = a >>> bug? Or >>>>>> am I missing something? >>>>>>=20 >>>>>>=20 >>>>>> I created a Jira for this: >>>>>> https://issues.apache.org/jira/browse/ZOOKEEPER-3482 >>>>>>=20 >>>>>>=20 >>>>>> thank you. >>>>>>=20 >>>>>> best regards >>>>>>=20 >>>=20 >>>=20 >>=20