From user-return-12831-archive-asf-public=cust-asf.ponee.io@zookeeper.apache.org Wed Jun 10 12:35:23 2020 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id 38EAC18062B for ; Wed, 10 Jun 2020 14:35:23 +0200 (CEST) Received: (qmail 99347 invoked by uid 500); 10 Jun 2020 12:35:22 -0000 Mailing-List: contact user-help@zookeeper.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@zookeeper.apache.org Delivered-To: mailing list user@zookeeper.apache.org Received: (qmail 99327 invoked by uid 99); 10 Jun 2020 12:35:21 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 10 Jun 2020 12:35:21 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id AA4CB1A3371 for ; Wed, 10 Jun 2020 12:35:20 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 0.255 X-Spam-Level: X-Spam-Status: No, score=0.255 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, HTML_MESSAGE=0.2, KAM_SHORT=0.001, NORMAL_HTTP_TO_IP=0.001, NUMERIC_HTTP_ADDR=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001, WEIRD_PORT=0.001] autolearn=disabled Authentication-Results: spamd2-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-he-de.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id D6YSidwd9yWq for ; Wed, 10 Jun 2020 12:35:17 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=2607:f8b0:4864:20::b42; helo=mail-yb1-xb42.google.com; envelope-from=aparajita.1194@gmail.com; receiver= Received: from mail-yb1-xb42.google.com (mail-yb1-xb42.google.com [IPv6:2607:f8b0:4864:20::b42]) by mx1-he-de.apache.org (ASF Mail Server at mx1-he-de.apache.org) with ESMTPS id 25D597F590 for ; Wed, 10 Jun 2020 12:35:16 +0000 (UTC) Received: by mail-yb1-xb42.google.com with SMTP id r18so1079179ybl.5 for ; Wed, 10 Jun 2020 05:35:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=rv4ylPNPqwHbRZJCU/AlcdZADHk64XQ/ZpGLREAZtZQ=; b=P5xj9kMulHkADV1aOxikNUNplUB9cKAbPm8yyyMUhQStHu08vYYhGR1aT9lu7CerEH oIzfIpIy9/n9HH80paR7wMPz1NPOL+xZDRhvy+Igj+qo9oDn1Id8+uJ0y6Eo9OCsj8yY DdTFgvFvWZ7SZzC1NOYciDXGaRXo+MRk/Z5JcHisv0ygpZ1GTrIzc83WckuFbSSMoFWJ fL/v0o2HipiMXnt5+332rA5mOqPkdEt8KlZUYjbTL/WvV81TXxHpZtMGK90kaDXTgBpR zJCTcQOsFVIssJfIkcIj9FuFjfHwKzdgEwwaDNygdCCPvDCPpBlCamLJwKFvWuBzpJAU cPRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=rv4ylPNPqwHbRZJCU/AlcdZADHk64XQ/ZpGLREAZtZQ=; b=GnG1rbI8paXfgflDdJzuOe5tRez+Ox3Ev1qj6aISg5R1tOuoGL8sf/AGISvOr6D2dS GM1pHizRBjbwb7E13RR6q66+oKCry2i9QpXLCa9B7hQcnv4KmOOe9a+LClOZasMsZr4v NYhGlEQ3QrAc5iO8eBBJkuKkuIhGe/gVR3bHAgkDZ5WNSDx578HAJpWuoCKUxFsNUcok BnYqBjqWlIKHQO1clLAeF0rV/JRCkUDjFVqzc3ueBaS1V7IAroKW7trsJHMZOpA5MIGa je/e4SJoIjQH6gwMDoWcChrx++ZLHfDsa65WZ/zUlc4VtskJow2hjCRbwmxqPO1BCRbj Bp7w== X-Gm-Message-State: AOAM532RWWwWLO/5bnzRYNR723jroFeh7NvyiniiW71tx9ulvDpwLru0 aXRUhqcJj7Unl8348SCo6yO24G7+aWZgDnDyhSvkIiPz X-Google-Smtp-Source: ABdhPJxo6ZQddU+hZ9H0WRmIyi/pWKhDvm2vx9ZmoN8E8KbiwGE/Ifb7m9sU6oR9rQdoLlq0xdmAOqfYXZir0St+bsk= X-Received: by 2002:a5b:346:: with SMTP id q6mr5490661ybp.472.1591792513632; Wed, 10 Jun 2020 05:35:13 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Aparajita Singh Date: Wed, 10 Jun 2020 18:05:01 +0530 Message-ID: Subject: Re: Zookeeper client fails during SASL authentication To: user@zookeeper.apache.org Content-Type: multipart/alternative; boundary="00000000000054cfed05a7ba134d" --00000000000054cfed05a7ba134d Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable > > Hi, > > I am trying to migrate an unauthenticated zookeeper cluster to a kerberos > authenticated one. For the time being SSL is disabled. I have configured > the server and client as described below but when SASL is enabled I am > unable to retreive data using zookeeper shell client from the zookeeper > server. Could I get some help in understanding why this is failing? > > server.log snippet > > 2020-06-10 17:09:01,263 - INFO [NIOServerCxn.Factory: > 0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@197] - Accepted socket > connection from /127.0.0.1:44994 > > 2020-06-10 17:09:01,264 - INFO [NIOServerCxn.Factory: > 0.0.0.0/0.0.0.0:2181:NIOServerCnxn@827] - Processing mntr command from / > 127.0.0.1:44994 > > 2020-06-10 17:09:01,265 - INFO [Thread-5:NIOServerCnxn@1007] - Closed > socket connection for client /127.0.0.1:44994 (no session established for > client) > > 2020-06-10 17:09:26,647 - INFO [main:Environment@100] - Client > environment:zookeeper.version=3D3.4.6-169--1, built on 02/10/2016 05:49 G= MT > > 2020-06-10 17:09:26,649 - INFO [main:Environment@100] - Client > environment:host.name=3Dstage-kdc-zk-ivy > > 2020-06-10 17:09:26,649 - INFO [main:Environment@100] - Client > environment:java.version=3D1.8.0_172 > > 2020-06-10 17:09:26,651 - INFO [main:Environment@100] - Client > environment:java.vendor=3DOracle Corporation > > 2020-06-10 17:09:26,651 - INFO [main:Environment@100] - Client > environment:java.home=3D/usr/lib/jvm/oracle-java8-jdk-amd64/jre > > 2020-06-10 17:09:26,651 - INFO [main:Environment@100] - Client > environment:java.class.path=3D/usr/hdp/2.4.0.0-169/zookeeper/bin/../build= /classes:/usr/hdp/2.4.0.0-169/zookeeper/bin/../build/lib/*.jar:/usr/hdp/2.4= .0.0-169/zookeeper/bin/../lib/xercesMinimal-1.9.6.2.jar:/usr/hdp/2.4.0.0-16= 9/zookeeper/bin/../lib/wagon-provider-api-2.4.jar:/usr/hdp/2.4.0.0-169/zook= eeper/bin/../lib/wagon-http-shared4-2.4.jar:/usr/hdp/2.4.0.0-169/zookeeper/= bin/../lib/wagon-http-shared-1.0-beta-6.jar:/usr/hdp/2.4.0.0-169/zookeeper/= bin/../lib/wagon-http-lightweight-1.0-beta-6.jar:/usr/hdp/2.4.0.0-169/zooke= eper/bin/../lib/wagon-http-2.4.jar:/usr/hdp/2.4.0.0-169/zookeeper/bin/../li= b/wagon-file-1.0-beta-6.jar:/usr/hdp/2.4.0.0-169/zookeeper/bin/../lib/slf4j= -log4j12-1.6.1.jar:/usr/hdp/2.4.0.0-169/zookeeper/bin/../lib/slf4j-api-1.6.= 1.jar:/usr/hdp/2.4.0.0-169/zookeeper/bin/../lib/plexus-utils-3.0.8.jar:/usr= /hdp/2.4.0.0-169/zookeeper/bin/../lib/plexus-interpolation-1.11.jar:/usr/hd= p/2.4.0.0-169/zookeeper/bin/../lib/plexus-container-default-1.0-alpha-9-sta= ble-1.jar:/usr/hdp/2.4.0.0-169/zookeeper/bin/../lib/netty-3.7.0.Final.jar:/= usr/hdp/2.4.0.0-169/zookeeper/bin/../lib/nekohtml-1.9.6.2.jar:/usr/hdp/2.4.= 0.0-169/zookeeper/bin/../lib/maven-settings-2.2.1.jar:/usr/hdp/2.4.0.0-169/= zookeeper/bin/../lib/maven-repository-metadata-2.2.1.jar:/usr/hdp/2.4.0.0-1= 69/zookeeper/bin/../lib/maven-project-2.2.1.jar:/usr/hdp/2.4.0.0-169/zookee= per/bin/../lib/maven-profile-2.2.1.jar:/usr/hdp/2.4.0.0-169/zookeeper/bin/.= ./lib/maven-plugin-registry-2.2.1.jar:/usr/hdp/2.4.0.0-169/zookeeper/bin/..= /lib/maven-model-2.2.1.jar:/usr/hdp/2.4.0.0-169/zookeeper/bin/../lib/maven-= error-diagnostics-2.2.1.jar:/usr/hdp/2.4.0.0-169/zookeeper/bin/../lib/maven= -artifact-manager-2.2.1.jar:/usr/hdp/2.4.0.0-169/zookeeper/bin/../lib/maven= -artifact-2.2.1.jar:/usr/hdp/2.4.0.0-169/zookeeper/bin/../lib/maven-ant-tas= ks-2.1.3.jar:/usr/hdp/2.4.0.0-169/zookeeper/bin/../lib/log4j-1.2.16.jar:/us= r/hdp/2.4.0.0-169/zookeeper/bin/../lib/jsoup-1.7.1.jar:/usr/hdp/2.4.0.0-169= /zookeeper/bin/../lib/jline-0.9.94.jar:/usr/hdp/2.4.0.0-169/zookeeper/bin/.= ./lib/httpcore-4.2.3.jar:/usr/hdp/2.4.0.0-169/zookeeper/bin/../lib/httpclie= nt-4.2.3.jar:/usr/hdp/2.4.0.0-169/zookeeper/bin/../lib/commons-logging-1.1.= 1.jar:/usr/hdp/2.4.0.0-169/zookeeper/bin/../lib/commons-io-2.2.jar:/usr/hdp= /2.4.0.0-169/zookeeper/bin/../lib/commons-codec-1.6.jar:/usr/hdp/2.4.0.0-16= 9/zookeeper/bin/../lib/classworlds-1.1-alpha-2.jar:/usr/hdp/2.4.0.0-169/zoo= keeper/bin/../lib/backport-util-concurrent-3.1.jar:/usr/hdp/2.4.0.0-169/zoo= keeper/bin/../lib/apache-log4j-extras-1.2.17.jar:/usr/hdp/2.4.0.0-169/zooke= eper/bin/../lib/ant-launcher-1.8.0.jar:/usr/hdp/2.4.0.0-169/zookeeper/bin/.= ./lib/ant-1.8.0.jar:/usr/hdp/2.4.0.0-169/zookeeper/bin/../zookeeper-3.4.6.2= .4.0.0-169.jar:/usr/hdp/2.4.0.0-169/zookeeper/bin/../src/java/lib/*.jar:/us= r/hdp/2.4.0.0-169/zookeeper/conf::/usr/hdp/2.4.0.0-169/zookeeper/conf:/usr/= hdp/2.4.0.0-169/zookeeper/zookeeper.jar:/usr/hdp/2.4.0.0-169/zookeeper/zook= eeper-3.4.6.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/slf4j-log4j1= 2-1.6.1.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/slf4j-api-1.6.1.jar:/usr/hdp= /2.4.0.0-169/zookeeper/lib/classworlds-1.1-alpha-2.jar:/usr/hdp/2.4.0.0-169= /zookeeper/lib/maven-model-2.2.1.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/htt= pcore-4.2.3.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/plexus-container-default= -1.0-alpha-9-stable-1.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/ant-launcher-1= .8.0.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/plexus-utils-3.0.8.jar:/usr/hdp= /2.4.0.0-169/zookeeper/lib/jline-0.9.94.jar:/usr/hdp/2.4.0.0-169/zookeeper/= lib/wagon-http-2.4.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/maven-settings-2.= 2.1.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/log4j-1.2.16.jar:/usr/hdp/2.4.0.= 0-169/zookeeper/lib/netty-3.7.0.Final.jar:/usr/hdp/2.4.0.0-169/zookeeper/li= b/commons-codec-1.6.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/commons-io-2.2.j= ar:/usr/hdp/2.4.0.0-169/zookeeper/lib/nekohtml-1.9.6.2.jar:/usr/hdp/2.4.0.0= -169/zookeeper/lib/backport-util-concurrent-3.1.jar:/usr/hdp/2.4.0.0-169/zo= okeeper/lib/apache-log4j-extras-1.2.17.jar:/usr/hdp/2.4.0.0-169/zookeeper/l= ib/ant-1.8.0.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/xercesMinimal-1.9.6.2.j= ar:/usr/hdp/2.4.0.0-169/zookeeper/lib/commons-logging-1.1.1.jar:/usr/hdp/2.= 4.0.0-169/zookeeper/lib/httpclient-4.2.3.jar:/usr/hdp/2.4.0.0-169/zookeeper= /lib/maven-profile-2.2.1.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/maven-error= -diagnostics-2.2.1.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/maven-project-2.2= .1.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/jsoup-1.7.1.jar:/usr/hdp/2.4.0.0-= 169/zookeeper/lib/plexus-interpolation-1.11.jar:/usr/hdp/2.4.0.0-169/zookee= per/lib/maven-plugin-registry-2.2.1.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/= wagon-http-shared-1.0-beta-6.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/maven-r= epository-metadata-2.2.1.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/wagon-http-= lightweight-1.0-beta-6.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/maven-ant-tas= ks-2.1.3.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/wagon-http-shared4-2.4.jar:= /usr/hdp/2.4.0.0-169/zookeeper/lib/wagon-provider-api-2.4.jar:/usr/hdp/2.4.= 0.0-169/zookeeper/lib/maven-artifact-manager-2.2.1.jar:/usr/hdp/2.4.0.0-169= /zookeeper/lib/maven-artifact-2.2.1.jar:/usr/hdp/2.4.0.0-169/zookeeper/lib/= wagon-file-1.0-beta-6.jar:/usr/share/zookeeper/* > > 2020-06-10 17:09:26,651 - INFO [main:Environment@100] - Client > environment:java.library.path=3D/usr/java/packages/lib/amd64:/usr/lib64:/= lib64:/lib:/usr/lib > > 2020-06-10 17:09:26,651 - INFO [main:Environment@100] - Client > environment:java.io.tmpdir=3D/tmp > > 2020-06-10 17:09:26,651 - INFO [main:Environment@100] - Client > environment:java.compiler=3D > > 2020-06-10 17:09:26,651 - INFO [main:Environment@100] - Client > environment:os.name=3DLinux > > 2020-06-10 17:09:26,652 - INFO [main:Environment@100] - Client > environment:os.arch=3Damd64 > > 2020-06-10 17:09:26,652 - INFO [main:Environment@100] - Client > environment:os.version=3D4.9.0-9-amd64 > > 2020-06-10 17:09:26,652 - INFO [main:Environment@100] - Client > environment:user.name=3Droot > > 2020-06-10 17:09:26,652 - INFO [main:Environment@100] - Client > environment:user.home=3D/root > > 2020-06-10 17:09:26,652 - INFO [main:Environment@100] - Client > environment:user.dir=3D/home/aparajita.singh > > 2020-06-10 17:09:26,653 - INFO [main:ZooKeeper@438] - Initiating client > connection, connectString=3Dstage-kdc-zk-ivy sessionTimeout=3D30000 > watcher=3Dorg.apache.zookeeper.ZooKeeperMain$MyWatcher@379619aa > > 2020-06-10 17:09:26,752 - INFO > [main-SendThread(stage-kdc-zk-ivy:2181):Login@293] - successfully logged > in. > > 2020-06-10 17:09:26,753 - INFO [Thread-0:Login$1@127] - TGT refresh > thread started. > > 2020-06-10 17:09:26,757 - INFO > [main-SendThread(stage-kdc-zk-ivy:2181):ZooKeeperSaslClient$1@285] - > Client will use GSSAPI as SASL mechanism. > > 2020-06-10 17:09:26,758 - INFO [Thread-0:Login@301] - TGT valid starting > at: Wed Jun 10 15:17:21 IST 2020 > > 2020-06-10 17:09:26,758 - INFO [Thread-0:Login@302] - TGT expires: > Thu Jun 11 15:17:21 IST 2020 > > 2020-06-10 17:09:26,758 - INFO [Thread-0:Login$1@181] - TGT refresh > sleeping until: Thu Jun 11 11:17:04 IST 2020 > > 2020-06-10 17:09:26,799 - INFO > [main-SendThread(stage-kdc-zk-ivy:2181):ClientCnxn$SendThread@1019] - > Opening socket connection to server stage-kdc-zk-ivy/10.33.203.225:2181. > Will attempt to SASL-authenticate using Login Context section 'Client' > > 2020-06-10 17:09:26,854 - INFO [NIOServerCxn.Factory: > 0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@197] - Accepted socket > connection from /10.33.203.225:45018 > > 2020-06-10 17:09:26,854 - INFO > [main-SendThread(stage-kdc-zk-ivy:2181):ClientCnxn$SendThread@864] - > Socket connection established to stage-kdc-zk-ivy/10.33.203.225:2181, > initiating session > > 2020-06-10 17:09:26,856 - INFO [NIOServerCxn.Factory: > 0.0.0.0/0.0.0.0:2181:ZooKeeperServer@868] - Client attempting to > establish new session at /10.33.203.225:45018 > > 2020-06-10 17:09:26,859 - INFO [CommitProcessor:88:ZooKeeperServer@617] > - Established session 0x58729e0540980002 with negotiated timeout 30000 fo= r > client /10.33.203.225:45018 > > 2020-06-10 17:09:26,861 - INFO > [main-SendThread(stage-kdc-zk-ivy:2181):ClientCnxn$SendThread@1279] - > Session establishment complete on server stage-kdc-zk-ivy/ > 10.33.203.225:2181, sessionid =3D 0x58729e0540980002, negotiated timeout = =3D > 30000 > > 2020-06-10 17:09:27,007 - WARN [NIOServerCxn.Factory: > 0.0.0.0/0.0.0.0:2181:ZooKeeperServer@969] - Client failed to SASL > authenticate: javax.security.sasl.SaslException: GSS initiate failed > [Caused by GSSException: Failure unspecified at GSS-API level (Mechanism > level: Invalid argument (400) - Cannot find key of appropriate type to > decrypt AP REP - AES256 CTS mode with HMAC SHA1-96)] > > 2020-06-10 17:09:27,007 - WARN [NIOServerCxn.Factory: > 0.0.0.0/0.0.0.0:2181:ZooKeeperServer@975] - Closing client connection due > to SASL authentication failure. > > 2020-06-10 17:09:27,007 - INFO [NIOServerCxn.Factory: > 0.0.0.0/0.0.0.0:2181:NIOServerCnxn@1007] - Closed socket connection for > client /10.33.203.225:45018 which had sessionid 0x58729e0540980002 > > 2020-06-10 17:09:27,008 - ERROR [NIOServerCxn.Factory: > 0.0.0.0/0.0.0.0:2181:NIOServerCnxn@178] - Unexpected Exception: > > java.nio.channels.CancelledKeyException > > at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73) > > at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:77) > > at > org.apache.zookeeper.server.NIOServerCnxn.sendBuffer(NIOServerCnxn.java:1= 51) > > at > org.apache.zookeeper.server.NIOServerCnxn.sendResponse(NIOServerCnxn.java= :1081) > > at > org.apache.zookeeper.server.ZooKeeperServer.processPacket(ZooKeeperServer= .java:936) > > at > org.apache.zookeeper.server.NIOServerCnxn.readRequest(NIOServerCnxn.java:= 373) > > at > org.apache.zookeeper.server.NIOServerCnxn.readPayload(NIOServerCnxn.java:= 200) > > at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:244) > > at > org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory= .java:208) > > at java.lang.Thread.run(Thread.java:748) > > 2020-06-10 17:09:27,008 - INFO > [main-SendThread(stage-kdc-zk-ivy:2181):ClientCnxn$SendThread@1142] - > Unable to read additional data from server sessionid 0x58729e0540980002, > likely server has closed socket, closing socket connection and attempting > reconnect > > 2020-06-10 17:09:27,008 - WARN [NIOServerCxn.Factory: > 0.0.0.0/0.0.0.0:2181:NIOServerCnxn@346] - Exception causing close of > session 0x58729e0540980002 due to java.nio.channels.CancelledKeyException > > 2020-06-10 17:10:01,317 - INFO [NIOServerCxn.Factory: > 0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@197] - Accepted socket > connection from /127.0.0.1:45004 > > 2020-06-10 17:10:01,318 - INFO [NIOServerCxn.Factory: > 0.0.0.0/0.0.0.0:2181:NIOServerCnxn@827] - Processing mntr command from / > 127.0.0.1:45004 > > > > zookeeper shell client output > > aparajita.singh@stage-kdc-zk-ivy:~$ sudo > /usr/hdp/2.4.0.0-169/zookeeper/bin/zookeeper-client -server > stage-kdc-zk-ivy get /test2 > > log4j:WARN Large window sizes are not allowed. > > log4j:WARN MaxIndex reduced to 13. > > Connecting to stage-kdc-zk-ivy > > Debug is true storeKey false useTicketCache true useKeyTab true > doNotPrompt true ticketCache is /tmp/krb5cc_0 isInitiator true KeyTab is > /etc/krb5.keytab refreshKrb5Config is false principal is > zookeeper/stage-kdc-zk-ivy@stage.fdp.kafka tryFirstPass is false > useFirstPass is false storePass is false clearPass is false > > Acquire TGT from Cache > > Principal is zookeeper/stage-kdc-zk-ivy@stage.fdp.kafka > > null credentials from Ticket Cache > > principal is zookeeper/stage-kdc-zk-ivy@stage.fdp.kafka > > Will use keytab > > Commit Succeeded > > > > WATCHER:: > > > WatchedEvent state:SyncConnected type:None path:null > > > WATCHER:: > > > WatchedEvent state:Disconnected type:None path:null > > Exception in thread "main" > org.apache.zookeeper.KeeperException$ConnectionLossException: > KeeperErrorCode =3D ConnectionLoss for /test2 > > at org.apache.zookeeper.KeeperException.create(KeeperException.java:99) > > at org.apache.zookeeper.KeeperException.create(KeeperException.java:51) > > at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155) > > at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1184) > > at org.apache.zookeeper.ZooKeeperMain.processZKCmd(ZooKeeperMain.java:717= ) > > at org.apache.zookeeper.ZooKeeperMain.processCmd(ZooKeeperMain.java:591) > > at org.apache.zookeeper.ZooKeeperMain.run(ZooKeeperMain.java:354) > > at org.apache.zookeeper.ZooKeeperMain.main(ZooKeeperMain.java:282) > > zoo.cfg > > #setACL=3DFalse > > autopurge.snapRetainCount=3D30 > > tickTime=3D2000 > > dataDir=3D/grid/1/var/lib/zookeeper > > zookeeper_jmx_port=3D9009 > > initLimit=3D100 > > syncLimit=3D5 > > autopurge.purgeInterval=3D24 > > clientPort=3D2181 > > globalOutstandingLimit=3D5000 > > maxClientCnxns=3D2000 > > server.99=3Dstage-kdc-zk-harley:2888:3888 > > server.88=3Dstage-kdc-zk-ivy:2888:3888 > > server.77=3Dstage-kdc-zk-2face:2888:3888 > > > authProvider.1=3Dorg.apache.zookeeper.server.auth.SASLAuthenticationProvi= der > > requireClientAuthScheme=3Dsasl > > > quorum.auth.enableSasl=3Dtrue > > quorum.auth.learnerRequireSasl=3Dtrue > > quorum.auth.serverRequireSasl=3Dtrue > > quorum.auth.kerberos.servicePrincipal=3Dhost/stage-kdc-zk-ivy@stage.fdp.k= afka > > quorum.cnxn.threads.size=3D20 > > > > java.env > > SERVER_JVMFLAGS=3D"${SERVER_JVMFLAGS} > -Djava.security.auth.login.config=3D/home/aparajita.singh/jaas/jaas.conf > -Dzookeeper.authProvider.sasl=3Dorg.apache.zookeeper.server.auth.SASLAuth= enticationProvider > -Dsun.security.krb5.debug=3Dtrue" > > CLIENT_JVMFLAGS=3D"${CLIENT_JVMFLAGS} > -Djava.security.auth.login.config=3D/home/aparajita.singh/jaas/client.con= f > -Dzookeeper.authProvider.sasl=3Dorg.apache.zookeeper.server.auth.SASLAuth= enticationProvider > -Dsun.security.krb5.debug=3Dtrue" > > > /home/aparajita.singh/jaas/jaas.conf > > // Zookeeper server authentication > > Server { > > com.sun.security.auth.module.Krb5LoginModule required > > useKeyTab=3Dtrue > > useTicketCache=3Dfalse > > //ticketCache=3D"/tmp/krb5cc_0" > > renewTicket=3Dtrue > > doNotPrompt=3Dtrue > > debug=3Dtrue > > keyTab=3D"/etc/krb5.keytab" > > serviceName=3D"host" > > principal=3D"host/stage-kdc-zk-ivy@stage.fdp.kafka"; > > }; > > > // Zookeeper quorum server authentication > > QuorumServer { > > com.sun.security.auth.module.Krb5LoginModule required > > useKeyTab=3Dtrue > > useTicketCache=3Dfalse > > //ticketCache=3D"/tmp/krb5cc_0" > > renewTicket=3Dtrue > > doNotPrompt=3Dtrue > > debug=3Dtrue > > keyTab=3D"/etc/krb5.keytab" > > serviceName=3D"host" > > principal=3D"host/stage-kdc-zk-ivy@stage.fdp.kafka"; > > }; > > > // Zookeeper learner authentication > > QuorumLearner { > > com.sun.security.auth.module.Krb5LoginModule required > > useKeyTab=3Dtrue > > useTicketCache=3Dfalse > > //ticketCache=3D"/tmp/krb5cc_0" > > renewTicket=3Dtrue > > doNotPrompt=3Dtrue > > debug=3Dtrue > > keyTab=3D"/etc/krb5.keytab" > > serviceName=3D"host" > > principal=3D"host/stage-kdc-zk-ivy@stage.fdp.kafka"; > > }; > > > > /home/aparajita.singh/jaas/client.conf > > // Zookeeper client authentication > > Client { > > com.sun.security.auth.module.Krb5LoginModule required > > useKeyTab=3Dtrue > > useTicketCache=3Dtrue > > ticketCache=3D"/tmp/krb5cc_0" > > renewTicket=3Dtrue > > doNotPrompt=3Dtrue > > debug=3Dtrue > > keyTab=3D"/etc/krb5.keytab" > > serviceName=3D"zookeeper" > > principal=3D"zookeeper/stage-kdc-zk-ivy@stage.fdp.kafka"; > > }; > > > Using kinit command I am able to generate the TGT for both principals. As > per the zookeeper server log, the TGT can be generated as expected. The > keytab file is accessible to all system users for now. > > aparajita.singh@stage-kdc-zk-ivy:~$ sudo /krb5/bin/kinit > zookeeper/stage-kdc-zk-ivy@stage.fdp.kafka -k -t /etc/krb5.keytab > > aparajita.singh@stage-kdc-zk-ivy:~$ sudo /krb5/bin/kinit > host/stage-kdc-zk-ivy@stage.fdp.kafka -k -t /etc/krb5.keytab > > > -- > Thanks, > Aparajita > --00000000000054cfed05a7ba134d--