From user-return-12526-archive-asf-public=cust-asf.ponee.io@zookeeper.apache.org Wed Jan 15 12:01:27 2020 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id 0546F18065E for ; Wed, 15 Jan 2020 13:01:26 +0100 (CET) Received: (qmail 28653 invoked by uid 500); 15 Jan 2020 12:01:25 -0000 Mailing-List: contact user-help@zookeeper.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@zookeeper.apache.org Delivered-To: mailing list user@zookeeper.apache.org Received: (qmail 28641 invoked by uid 99); 15 Jan 2020 12:01:25 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 15 Jan 2020 12:01:25 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 3BC761A31FD for ; Wed, 15 Jan 2020 12:01:24 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 0.502 X-Spam-Level: X-Spam-Status: No, score=0.502 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, HTML_MESSAGE=0.2, KAM_LOTSOFHASH=0.25, NUMERIC_HTTP_ADDR=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001, WEIRD_PORT=0.001] autolearn=disabled Authentication-Results: spamd2-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-ec2-va.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id mMLvXvIrB9v7 for ; Wed, 15 Jan 2020 12:01:19 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=209.85.217.50; helo=mail-vs1-f50.google.com; envelope-from=jain.arpit6@gmail.com; receiver= Received: from mail-vs1-f50.google.com (mail-vs1-f50.google.com [209.85.217.50]) by mx1-ec2-va.apache.org (ASF Mail Server at mx1-ec2-va.apache.org) with ESMTPS id E4715BC535 for ; Wed, 15 Jan 2020 12:01:18 +0000 (UTC) Received: by mail-vs1-f50.google.com with SMTP id x18so10275352vsq.4 for ; Wed, 15 Jan 2020 04:01:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=aBCS1+Mo3lfLd1/V2C64hLXtuafvP9Kf/7qgay2ty0k=; b=lrUOM6fIAAMnNzYzbXrJ7bLjmor+9qioNDSK7xc9rGDPndYPqRjzxxnG1As1rwa31O VOQpWpimMXxTOkN8xrA3CDV4DZo278QMWKnKTw7eOc7K5sxu+vEY9FB6sHi1c7VDYLti 6IpS5D7YAreo516WjmL+yF5O8yGI+Wp6Wjy/0Pm/hWZehb6FISTdeR17jcUxXTY8pE96 bfmkHCGtuqL1LAonb62MLIcROsVQTGjfnGbZncg90S7w/x1AtPXmIveScMZdD+e1+5/g ms+nNCf7dcuuQVyZe0mBly9wEtgqfIHIwsfUjEZiwdZrJKE+OTi8lSjHcWG9vLqRfMS3 4z8A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=aBCS1+Mo3lfLd1/V2C64hLXtuafvP9Kf/7qgay2ty0k=; b=hjpzuNhntiaL3g9EXrS6gj2jPLyezbcdGCxwrZgB+k1XAGar91ievltPpa5ORShehE OUpcPkBjmg7PCagKjucK4UfgCXIFWxLTAOK/TPboQd9nsrUmsXCxSF9O9R0Zygdx2h1z t3J2bFFOCKZZ7y1w2i22LHk0yVf7hOlZnrHJ81uDIT5JO5o5WYCw9enyj2iu0ebyrk98 xifLTsIO3NJbNMWaFqRVZh1MdILquWf26bv0NR+xTIPmSztRKWIC2KC4TfqwElN2iKud Ix6+X5lP4f9XnUQEU2tPr3ML0b/Fwmy2pePFS8nRUw0RS0q1uETMxy0bhO+2DIpm8tJk lozA== X-Gm-Message-State: APjAAAWKNRO9lZFkKgCbzm4lu46UMrzrxk4AecZi49e4D5VTM1zJFqW5 GIOZsasjHJ2CFqcG/5VpKuBfBeh5ozC/uJv3JZ4= X-Google-Smtp-Source: APXvYqzD8bmlg7+GlOzx2T1SRUWgGR1Poy8ex0FFSLNcxX+N/k67OFkH45o2SWQach4JB4zj/F+/M6xE5iw8/C+6TKk= X-Received: by 2002:a05:6102:d4:: with SMTP id u20mr4219115vsp.27.1579089678347; Wed, 15 Jan 2020 04:01:18 -0800 (PST) MIME-Version: 1.0 References: <87h814n37k.fsf@sinenomine.net> <87blrcmow9.fsf@sinenomine.net> <877e20mjh7.fsf@sinenomine.net> In-Reply-To: From: Arpit Jain Date: Wed, 15 Jan 2020 12:01:05 +0000 Message-ID: Subject: Re: Zookeeper and curator SASL authentication To: =?UTF-8?B?U3phbGF5LUJla8WRIE3DoXTDqQ==?= Cc: Damien Diederen , Enrico Olivelli , UserZooKeeper Content-Type: multipart/alternative; boundary="00000000000058b059059c2c777c" --00000000000058b059059c2c777c Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable I have asked in Curator mailing list as well but not much help. I am able to set ACL with sasl scheme by using zkCli.sh client in Zookeeper server. The idea is to use Curator to set the ACLs so that only my client application can access its Znodes. On Wed, Jan 15, 2020 at 9:21 AM Szalay-Bek=C5=91 M=C3=A1t=C3=A9 wrote: > I am not sure what is wrong with the code... I am not familiar with > Curator. I can try to google / reproduce this and see what is wrong, but = it > will take a while for me. So first I would ask the others, maybe there is > someone who knows both ZooKeeper SASL and Curator and can help you more i= n > this mailing list. If noone replies, then I will try to setup a dummy > project with Curator to test this. > > Did you also ask around the Curator mailing list maybe? Would it help if = I > send you code about setting the ACLs using plain ZooKeeper (and no Curato= r)? > > On Tue, Jan 14, 2020 at 2:48 PM Arpit Jain wrote: > >> Thanks for the clarification. >> I am able to authenticate client with Zookeeper. However, when I started >> to set ACLs with the same client, I get error messages. This is how I am >> creating curator client for setting ACLs >> >> CuratorFrameworkFactory.Builder builder =3D >> >> CuratorFrameworkFactory.builder().connectString( >> coordinatorHosts).retryPolicy(retryPolicy) >> >> .connectionTimeoutMs(coordinatorConnectionTimeout >> ).sessionTimeoutMs(coordinatorSessionTimeout); >> >> final CuratorFramework curatorFramework =3D >> >> builder.authorization("sasl", "zkclient/zoo1@EXAMPLE.COM" >> .getBytes()).aclProvider(new ACLProvider() { >> >> @Override >> >> public List getDefaultAcl() { >> >> return ZooDefs.Ids.CREATOR_ALL_ACL; >> >> } >> >> >> @Override >> >> public List getAclForPath(String path) { >> >> return ZooDefs.Ids.CREATOR_ALL_ACL; >> >> } >> >> }).build(); >> >> >> I see below logs in Zookeeper node: >> >> >> >> >> >> *2020-01-14 13:27:53,174 [myid:1] - INFO >> [NIOWorkerThread-3:SaslServerCallbackHandler@120] - Successfully >> authenticated client: authenticationID=3Dzkclient/zoo1@EXAMPLE.COM >> ; authorizationID=3Dzkclient/zoo1@EXAMPLE.COM >> .2020-01-14 13:27:53,175 [myid:1] - INFO >> [NIOWorkerThread-3:SaslServerCallbackHandler@136] - Setting authorizedI= D: >> zkclient/zoo1@EXAMPLE.COM 2020-01-14 13:27:53,175 >> [myid:1] - INFO [NIOWorkerThread-3:ZooKeeperServer@1170] - adding SASL >> authorization for authorizationID: zkclient/zoo1@EXAMPLE.COM >> 2020-01-14 13:27:53,182 [myid:1] - INFO >> [NIOWorkerThread-7:ZooKeeperServer@1095] - got auth packet >> /172.30.0.6:36658 2020-01-14 13:27:53,183 [myid= :1] >> - WARN [NIOWorkerThread-7:ZooKeeperServer@1123] - Authentication failed >> for scheme: sasl* >> >> Is this not the correct way to do it ? >> >> >> >> On Tue, Jan 14, 2020 at 11:52 AM Szalay-Bek=C5=91 M=C3=A1t=C3=A9 < >> szalay.beko.mate@gmail.com> wrote: >> >>> The system property name is a bit misleading... this parameter is >>> actually specifies the username used in the ZooKeeper server principal. >>> (in your case the server principal is: zookeeper/zoo1@EXAMPLE.COM) >>> AFAIK the ZooKeeper client (after authenticated as zkclient/ >>> zoo1@EXAMPLE.COM in Kerberos based on the jaas.conf file) needs to know >>> the ZooKeeper server principal in order to ask for a specific token fro= m >>> kerberos which can be read by the ZooKeeper server. >>> >>> In 3.5.5 (or 3.5.6) you can use the zookeeper.sasl.client.username >>> parameter (plus some other parameters) to configure how the server >>> principal will be determined by the client. >>> See: >>> https://github.com/apache/zookeeper/blob/c11b7e26bc554b8523dc929761dd28= 808913f091/zookeeper-server/src/main/java/org/apache/zookeeper/SaslServerPr= incipal.java#L48 >>> >>> In future releases (3.5.7, 3.6, ...) you can also use >>> the zookeeper.server.principal parameter (a much better name I think) t= o >>> use a fix server principal name in the client. >>> See: >>> https://github.com/apache/zookeeper/blob/1c5d135d74f16275876c024401dc2d= e92909b20a/zookeeper-server/src/main/java/org/apache/zookeeper/SaslServerPr= incipal.java#L50 >>> >>> On Mon, Jan 13, 2020 at 6:03 PM Arpit Jain >>> wrote: >>> >>>> Does this user name have to be "Zookeeper" >>>> (-Dzookeeper.sasl.client.username=3Dzookeeper) always ? >>>> And the client principal name is different than this username..Correct >>>> me if I am wrong ? >>>> >>>> On Mon, Jan 13, 2020 at 4:58 PM Arpit Jain >>>> wrote: >>>> >>>>> Thanks you so much ! >>>>> It worked finally. I had to change >>>>> -Dzookeeper.sasl.client.username=3Dzookeeper parameter. >>>>> >>>>> On Mon, Jan 13, 2020 at 4:40 PM Szalay-Bek=C5=91 M=C3=A1t=C3=A9 < >>>>> szalay.beko.mate@gmail.com> wrote: >>>>> >>>>>> You are using 3.5.5 or 3.5.6, right? >>>>>> I think you need to specify: >>>>>> -Dzookeeper.sasl.client.username=3Dzookeeper >>>>>> can you give it a try? If it doesn't work then I can take a deeper >>>>>> look (also we can enable some debug logging) >>>>>> >>>>>> On Mon, Jan 13, 2020 at 5:31 PM Arpit Jain >>>>>> wrote: >>>>>> >>>>>>> Hi >>>>>>> >>>>>>> I have Kerberos, Zookeeper and my application (using curator) >>>>>>> running in 3 docker containers with ZK SASL authentication enabled.= The ZK >>>>>>> can login to Kerberos and starts successfully. >>>>>>> >>>>>>> The ZK server principal is zookeeper/zoo1@EXAMPLE.COM >>>>>>> The client principal is : zkclient/zoo1@EXAMPLE.COM >>>>>>> >>>>>>> While starting my application, I am seeing failure while obtaining >>>>>>> TGS. >>>>>>> See the log at Kerberos side: >>>>>>> >>>>>>> >>>>>>> >>>>>>> *Jan 13 15:22:19 kdc krb5kdc[20](info): AS_REQ (2 etypes {18 17}) >>>>>>> 172.30.0.6 : NEEDED_PREAUTH: zkclient/zoo1@EXAMP= LE.COM >>>>>>> for krbtgt/EXAMPLE.COM@EXAMPLE.COM >>>>>>> , Additional pre-authentication requiredJa= n 13 >>>>>>> 15:22:19 kdc krb5kdc[20](info): AS_REQ (2 etypes {18 17}) 172.30.0.= 6 >>>>>>> : ISSUE: authtime 1578928939, etypes {rep=3D18 t= kt=3D18 >>>>>>> ses=3D18}, zkclient/zoo1@EXAMPLE.COM for >>>>>>> krbtgt/EXAMPLE.COM@EXAMPLE.COM Jan 13 15:2= 2:19 kdc >>>>>>> krb5kdc[20](info): TGS_REQ (4 etypes {18 17 16 23}) 172.30.0.6 >>>>>>> : ISSUE: authtime 1578928939, etypes {rep=3D18 t= kt=3D18 >>>>>>> ses=3D18}, zkclient/zoo1@EXAMPLE.COM for >>>>>>> zkclient/zoo1@EXAMPLE.COM * >>>>>>> >>>>>>> However, if I use the zkCli.sh to login to Zookeeper, it >>>>>>> successfully logs in. See the log on Kerberos side. See the differe= nce in >>>>>>> the last line while requesting TGS. >>>>>>> >>>>>>> >>>>>>> >>>>>>> *Jan 13 15:26:14 kdc krb5kdc[20](info): AS_REQ (2 etypes {18 17}) >>>>>>> 172.30.0.3 : NEEDED_PREAUTH: zkclient/zoo1@EXAMP= LE.COM >>>>>>> for krbtgt/EXAMPLE.COM@EXAMPLE.COM >>>>>>> , Additional pre-authentication requiredJa= n 13 >>>>>>> 15:26:14 kdc krb5kdc[20](info): AS_REQ (2 etypes {18 17}) 172.30.0.= 3 >>>>>>> : ISSUE: authtime 1578929174, etypes {rep=3D18 t= kt=3D18 >>>>>>> ses=3D18}, zkclient/zoo1@EXAMPLE.COM for >>>>>>> krbtgt/EXAMPLE.COM@EXAMPLE.COM Jan 13 15:2= 6:14 kdc >>>>>>> krb5kdc[20](info): TGS_REQ (4 etypes {18 17 16 23}) 172.30.0.3 >>>>>>> : ISSUE: authtime 1578929174, etypes {rep=3D18 t= kt=3D18 >>>>>>> ses=3D18}, zkclient/zoo1@EXAMPLE.COM for >>>>>>> zookeeper/zoo1@EXAMPLE.COM * >>>>>>> >>>>>>> The client section in JAAS config file is same in both the cases bu= t >>>>>>> the server it is looking for is different in both the cases. >>>>>>> Could someone suggest why there is a difference ? >>>>>>> >>>>>>> Thanks >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Mon, Jan 13, 2020 at 4:17 PM Szalay-Bek=C5=91 M=C3=A1t=C3=A9 < >>>>>>> szalay.beko.mate@gmail.com> wrote: >>>>>>> >>>>>>>> Also please note, that the >>>>>>>> 'Configuration.getConfiguration().refresh()' will reload only the >>>>>>>> jaas.config. >>>>>>>> If you also need to reload the kerberos client config, then you ca= n >>>>>>>> add the "refreshKrb5Config=3Dtrue" line to your jaas.conf file. Th= is will >>>>>>>> trigger to reload the krb.cfg file as well if needed. >>>>>>>> >>>>>>>> I am just working on a PR where I had to use both of these: >>>>>>>> https://github.com/apache/zookeeper/pull/1204/files#diff-0c01d3c68= a71c68701d778cc556c8e0a >>>>>>>> >>>>>>>> Cheers, >>>>>>>> Mate >>>>>>>> >>>>>>>> On Thu, Jan 9, 2020 at 10:02 PM Damien Diederen < >>>>>>>> ddiederen@sinenomine.net> wrote: >>>>>>>> >>>>>>>>> >>>>>>>>> Hi Enrico, >>>>>>>>> >>>>>>>>> > There is a method to force JAAS to reload the system property. >>>>>>>>> > >>>>>>>>> > Something like Configuration.getConfiguration().refresh() >>>>>>>>> >>>>>>>>> Great to know! Thanks! >>>>>>>>> >>>>>>>>> > You have to call that method after changing the system property >>>>>>>>> >>>>>>>>> Cheers, -D >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> > Il gio 9 gen 2020, 20:05 Damien Diederen < >>>>>>>>> ddiederen@sinenomine.net> ha >>>>>>>>> > scritto: >>>>>>>>> > >>>>>>>>> >> >>>>>>>>> >> Hi Arpit, M=C3=A1t=C3=A9, >>>>>>>>> >> >>>>>>>>> >> Arpit wrote: >>>>>>>>> >> >>>>>>>>> >> > The solution is to pass JAAS file >>>>>>>>> >> > with -Djava.security.auth.login.config=3D/path/to/jaas.conf= . >>>>>>>>> >> >>>>>>>>> >> Okay=E2=80=94good. >>>>>>>>> >> >>>>>>>>> >> > Using System.setProperty does not work for me. >>>>>>>>> >> >>>>>>>>> >> Ah, I see. And I'm not surprised; I think M=C3=A1t=C3=A9 is o= n the right >>>>>>>>> track: >>>>>>>>> >> >>>>>>>>> >> >> I also faced this exception not long ago. I think it is an >>>>>>>>> edge case, >>>>>>>>> >> most >>>>>>>>> >> >> probably you have something else, but still... maybe it >>>>>>>>> helps: >>>>>>>>> >> >> >>>>>>>>> >> >> I tried to write a unit test which dynamically generated >>>>>>>>> multiple >>>>>>>>> >> >> jaas.conf files. Then I was setting the >>>>>>>>> >> >> java.security.auth.login.config system property to the >>>>>>>>> config file I >>>>>>>>> >> needed >>>>>>>>> >> >> in the given testcase, and when I tried to establish a >>>>>>>>> ZooKeeper >>>>>>>>> >> connection >>>>>>>>> >> >> in the unit test, I also got the same exception that you go= t. >>>>>>>>> >> >> >>>>>>>>> >> >> The problem was, that the security configuration file I >>>>>>>>> referred in the >>>>>>>>> >> >> java.security.auth.login.config system property file was >>>>>>>>> read only once, >>>>>>>>> >> >> then stored in memory. And it haven't got reloaded, even if >>>>>>>>> the file (or >>>>>>>>> >> >> its path in the system property) changed. >>>>>>>>> >> >>>>>>>>> >> My understanding is that the property is read very early after >>>>>>>>> "VM boot" >>>>>>>>> >> (the first time any class tries to access the >>>>>>>>> java.security.Provider): >>>>>>>>> >> the resource it points to is parsed at that point, and the >>>>>>>>> property >>>>>>>>> >> "never" checked again. >>>>>>>>> >> >>>>>>>>> >> (It *may* be possible to flush the "Spi" or something, but it'= s >>>>>>>>> clearly >>>>>>>>> >> not the kind of usage it was designed for.) >>>>>>>>> >> >>>>>>>>> >> >> Maybe the best in this case is to >>>>>>>>> >> >> specify separate JAAS config sections for each tests and us= e >>>>>>>>> a single >>>>>>>>> >> >> JAAS.conf file per JVM. >>>>>>>>> >> >>>>>>>>> >> That's probably the easiest if the set is enumerable. >>>>>>>>> >> >>>>>>>>> >> "Real dynamism" might require overriding the "Spi" or >>>>>>>>> "Provider," but >>>>>>>>> >> that's probably overkill for a few tests. >>>>>>>>> >> >>>>>>>>> >> (Now that I think of it=E2=80=A6 our tests are already run und= er the >>>>>>>>> JMockit >>>>>>>>> >> agent, so live-patching JAAS methods using mockit.MockUp might >>>>>>>>> be >>>>>>>>> >> another option :) >>>>>>>>> >> >>>>>>>>> >> Anyway. It looks like setting the property externally worked >>>>>>>>> for Arpit. >>>>>>>>> >> >>>>>>>>> >> Cheers, -D >>>>>>>>> >> >>>>>>>>> >>>>>>>> --00000000000058b059059c2c777c--