zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stevo Slavić <ssla...@gmail.com>
Subject Re: How to determine cause of InvalidACL
Date Mon, 04 Jul 2016 01:30:33 GMT
Thanks for the tips to both!

Still struggling with this.

If I got it right, issue it seems is in different response of ZooKeeper
server (3.4.6) for same request - when authentication is on or off,
ZooKeeper responds differently when trying to create a node that already
exists.

Not using Kerberos, using DIGEST-MD5 as SASL mechanism.

I'm trying to enable just ZooKeeper authentication, between Kafka brokers
and ZooKeeper, and an app and ZooKeeper. InvalidACL is not thrown and
everything works OK, without authentication enabled.
App has logic to explicitly create topic using AdminUtils.createTopic with
custom topic configuration. That fails now with InvalidACL. No custom ACL
is being used in the calls.

What I can see from stack trace and Kafka source code, when trying to write
topic configuration data
https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/admin/AdminUtils.scala#L436,
underlying ZkUtils call first gets NoNode for the topic config path
"/config/topics/foo" and then tries to create parent and that fails with
InvalidACL
https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/utils/ZkUtils.scala#L424

Will check with Kafka community, why does the AdminUtils, when tying to
update topic config node data assumes that it got NoNode because parent
node doesn't exist - checked manually and /config/topics exists (likely
broker creates it on startup) and has ACLs set (world can read, and Kafka's
user can cdrwa). App trying to create the topic explicitly has same JAAS
config file as Kafka broker, and zkUtils passed to AdminUtils.createTopic
has isSecure set to true, so it shouldn't be permissions or authentication
issue.

Without authentication, same ZkUtils call to create parent fails with
ZkNodeExistsException but that exception is swallowed by ZkClient (see
https://github.com/sgroschupf/zkclient/blob/master/src/main/java/org/I0Itec/zkclient/ZkClient.java#L295
) so ZkUtils never sees it. While it seems with ZooKeeper 3.4.6 server and
authentication on, client trying to create a node that already exists gets
InvalidACL, so topic creation fails - if true, is this inconsistency in
response a feature or a bug in ZooKeeper?

Kind regards,
Stevo Slavic.

On Sat, Jul 2, 2016 at 2:40 AM, Raúl Gutiérrez Segalés <rgs@itevenworks.net>
wrote:

> Another way to debug this is by using zk-dump [0] to dump requests/replies
> and see if the expected ACL is being send across the wire.
>
> -rgs
>
> [0] https://github.com/twitter/zktraffic
> On Jul 1, 2016 7:03 AM, "Flavio Junqueira" <fpj@apache.org> wrote:
>
> > Hi Stevo,
> >
> > The logs indicate whether it has connected and authorized properly. Make
> > sure your keytab file is all good.
> >
> > One other suggestion is to try with the zkCli against the ensemble before
> > you try to run Kafka to make sure that your configuration is correct.
> >
> > -Flavio
> >
> > > On 01 Jul 2016, at 14:48, Stevo Slavić <sslavic@gmail.com> wrote:
> > >
> > > Hello Apache ZooKeeper community,
> > >
> > > As subject states, I'm having difficulties determining what could be
> > > causing InvalidACL to be thrown, any help would be appreciated.
> > >
> > > Kafka's ZkUtils I assume works OK and sends default ACLs with create
> path
> > > request
> > >
> >
> https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/utils/ZkUtils.scala#L80
> > > but server replies with InvalidACL error. Even at trace log level I
> > cannot
> > > see any useful information in the console.
> > >
> > > Kind regards,
> > > Stevo Slavic.
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message