From dev-return-107148-archive-asf-public=cust-asf.ponee.io@kafka.apache.org Wed Sep 4 10:39:26 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id 1B9151804BB for ; Wed, 4 Sep 2019 12:39:25 +0200 (CEST) Received: (qmail 88588 invoked by uid 500); 4 Sep 2019 15:22:08 -0000 Mailing-List: contact dev-help@kafka.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@kafka.apache.org Delivered-To: mailing list dev@kafka.apache.org Received: (qmail 88574 invoked by uid 99); 4 Sep 2019 15:22:07 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 04 Sep 2019 15:22:07 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id E936AC1D07 for ; Wed, 4 Sep 2019 10:39:22 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.801 X-Spam-Level: * X-Spam-Status: No, score=1.801 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=disabled Authentication-Results: spamd4-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-he-de.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id WK0fL6DV2rmb for ; Wed, 4 Sep 2019 10:39:16 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=2607:f8b0:4864:20::d41; helo=mail-io1-xd41.google.com; envelope-from=rajinisivaram@gmail.com; receiver= Received: from mail-io1-xd41.google.com (mail-io1-xd41.google.com [IPv6:2607:f8b0:4864:20::d41]) by mx1-he-de.apache.org (ASF Mail Server at mx1-he-de.apache.org) with ESMTPS id 4F8447DD5B for ; Wed, 4 Sep 2019 10:39:15 +0000 (UTC) Received: by mail-io1-xd41.google.com with SMTP id b10so43195005ioj.2 for ; Wed, 04 Sep 2019 03:39:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=OJwq2YgrUofjPGfoiS/26AO37HV0NmAJBOZ1teUWjds=; b=eo0jfHjyJ0XUd07Ekd2a7ZT8WZDj/F4aikWnIMsBz+bjgfxvj/ss/3GLxPPQs2l+y9 9Cl+OzIT+3YIAGQocvoi8sMvgmKJLKcUM4/g/52b0/Npq35LvYEpma3GtfBvo9jOxXKe OdgWLYiPAQtKwudWYLXFZLPwvkRq5k1afzpn1IltjcN0pJIPe6xh+PKh3I2cmuxOH3Nc rj6+C7umN/0YhBap12r4Iguuhzgpi239wdmNXFKbtWVP/RtFoIx6DD0Dxnptf+ni9twf 1EbaJOFFAnJ3VU1MZd1zDGk2gTBZ1Z8nJW1Y6svIWiE25V1Uuu29daHC1Dt1DCNpFVHB UNWg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=OJwq2YgrUofjPGfoiS/26AO37HV0NmAJBOZ1teUWjds=; b=XkTtLGlzsc32ZTgHYnWN+He1h0xkD4vmrgXFS80+GMGBkSUJabbdj60eHngqYdRMbf 5a6hu3Xz7Jz9bK4A5FcvjnYnIkgg33K7mICk2a9WNkveRqD7YJPfSZvt8u9xpNdoLcr4 6kGnI1Dh6Vl9bJEC/G1KobqeXcPyxLClzD8lgJLdwC44yRXI+TeA/io0QmoiQoMUjNjd +uHQWjVOeC23xttR3k9blpuq/ws8/jGS/shlqsOzsQRMXkNRL3FrWf5Rsfvz90cjSeSh HeqXFfaLgp+r+QPp3HFmNralgx2swc3WpUWNJK1TssBKaT0eYOKjigs7vqgViKuVRWb5 FPPg== X-Gm-Message-State: APjAAAXUTsEFbEhr2R22ZF5PPojetKhwOIj1jGfEz4fu+xtP3x7TCaiq pASXpQa5OzvRArYzwrL73mo1jq7VVbHSZ7pBkmb8jWO5 X-Google-Smtp-Source: APXvYqyPr8Kvxo65HHkg/jdYGDWhomy1Bp33//VlENg3ZqC+lQU4fn0RKawyELSc/Qm/5444RwR34OMS6xJ3MkZMXcg= X-Received: by 2002:a02:9668:: with SMTP id c95mr9665705jai.4.1567593553369; Wed, 04 Sep 2019 03:39:13 -0700 (PDT) MIME-Version: 1.0 References: <48CB3DB7-709A-4479-8EDB-A4D95B26728E@apache.org> <2FB0D77D-5E49-4232-B7DE-BD83780C9CFD@apache.org> In-Reply-To: From: Rajini Sivaram Date: Wed, 4 Sep 2019 11:39:01 +0100 Message-ID: Subject: Re: [DISCUSS] KIP-504 - Add new Java Authorizer Interface To: dev Content-Type: multipart/alternative; boundary="000000000000e6a0d20591b7d029" --000000000000e6a0d20591b7d029 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi Colin & Don, I have submitted a PR to help with reviewing the new API: https://github.com/apache/kafka/pull/7293/files. - Authorizer.java contains the updated API and includes javadoc update including the threading model. - AclAuthorizer.scala changes in the PR should give an idea of the implementation changes for custom authorizers that want to continue to b= e synchronous. - KafkaApis.scala continues to use the API synchronously for now. It can be updated later. Thank you, Rajini On Wed, Sep 4, 2019 at 9:38 AM Rajini Sivaram wrote: > Hi Don, > > Thanks for your note. > > 1) The intention is to avoid blocking in the calling thread. We already > have several requests that are put into a purgatory when waiting for remo= te > communication, for example produce request waiting for replication. Since > we have a limited number of request threads in the broker, we want to mak= e > progress with other requests, while one is waiting on any form of remote > communication. > > 2) Async management calls will be useful with the default authorizer when > KIP-500 removes ZK and we rely on Kafka instead. Our current ZK-based > implementation as well as any custom implementations that don't want to b= e > async will just need to return a sync'ed value. So instead of returning ` > value`, the code would just return `CompletableFuture.completedFuture > (value)`. So it would be just a single line change in the implementation > with the new API. The caller would treat completedFuture exactly as it > does today, processing the request synchronously without using a purgator= y. > > 3) For implementations that return a completedFuture as described in 2), > the behaviour would remain exactly the same. No additional threads or > purgatory will be used for this case. So there would be no performance > penalty. For implementations that return a future that is not complete, w= e > prioritise running more requests concurrently. So in a deployment with a > large number of clients, we would improve performance by allowing other > requests to be processed on the request threads while some are waiting fo= r > authorization metadata. > > 4) I was concerned about this too. The goal is to make the API flexible > enough to handle large scale deployments in future when caching all > authorization metadata in each broker is not viable. Using an async API > that returns CompletionStage, the caller has the option to handle the > result synchronously or asynchronously, so we don't necessarily need to > update the calling code right away. Custom authorizers using the async AP= I > have full control over whether authorization is performed in-line since > completedFuture will always be handled synchronously. We do need to > update KafkaApis to take advantage of the asynchronous API to improve > scale. Even though this is a big change, since we will be doing the same > for all requests, it shouldn't be too hard to maintain since the same > pattern will be used for all requests. > > Regards, > > Rajini > > On Tue, Sep 3, 2019 at 11:48 PM Don Bosco Durai wrote: > >> Hi Rajini >> >> Help me understand this a bit more. >> >> 1. For all practical purpose, without authorization you can't go to the >> next step. The calling code needs to block anyway. So should we just let >> the implementation code do the async part? >> 2. If you feel management calls need to be async, then we should conside= r >> another set of async APIs. I don't feel we should complicate it further = ( >> 3. Another concern I have is wrt performance. Kafka has been built to >> handle 1000s per second requests. Not sure whether making it async will = add >> any unnecessary overhead. >> 4. How much complication would this add on the calling side? And is it >> worth it? >> >> Thanks >> >> Bosco >> >> >> =EF=BB=BFOn 9/3/19, 8:50 AM, "Rajini Sivaram" = wrote: >> >> Hi all, >> >> Ismael brought up a point that it will be good to make the Authorize= r >> interface asynchronous to avoid blocking request threads during remo= te >> operations. >> >> 1) Since we want to support different backends for authorization >> metadata, >> making createAcls() and deleteAcls() asynchronous makes sense since >> these >> always involve remote operations. When KIP-500 removes ZooKeeper, we >> would >> want to move ACLs to Kafka and async updates will avoid unnecessary >> blocking. >> 2) For authorize() method, we currently use cached ACLs in the >> built-in >> authorizers, so synchronous authorise operations work well now. But >> async >> authorize() would support this scenario as well as authorizers in >> large >> organisations where an LRU cache would enable a smaller cache even >> when the >> backend holds a large amount of ACLs for infrequently used resources >> or >> users who don't use the system frequently. >> >> For both cases, the built-in authorizer will continue to be >> synchronous, >> using CompletableFuture.completedFuture() to return the actual >> result. But >> async API will make custom authorizer implementations more flexible.= I >> would like to know if there are any concerns with these changes befo= re >> updating the KIP. >> >> *Proposed API:* >> public interface Authorizer extends Configurable, Closeable { >> >> Map> start(AuthorizerServerInfo >> serverInfo); >> List> >> authorize(AuthorizableRequestContext requestContext, List >> actions); >> List> >> createAcls(AuthorizableRequestContext requestContext, List >> aclBindings); >> List> >> deleteAcls(AuthorizableRequestContext requestContext, >> List aclBindingFilters); >> CompletionStage> acls(AclBindingFilter >> filter); >> } >> >> >> Thank you, >> >> Rajini >> >> On Sun, Aug 18, 2019 at 6:25 PM Don Bosco Durai >> wrote: >> >> > Hi Rajini >> > >> > Thanks for clarifying. I am good for now. >> > >> > Regards >> > >> > Bosco >> > >> > >> > =EF=BB=BFOn 8/16/19, 11:30 AM, "Rajini Sivaram" >> wrote: >> > >> > Hi Don, >> > >> > That should be fine. I guess Ranger loads policies from the >> database >> > synchronously when the authorizer is configured, similar to >> > SimpleAclAuthorizer loading from ZooKeeper. Ranger can continu= e >> to load >> > synchronously from `configure()` or `start()` and return an >> empty map >> > from >> > `start()`. That would retain the existing behaviour.. When the >> same >> > database stores policies for all listeners and the policies ar= e >> not >> > stored >> > in Kafka, there is no value in making the load asynchronous. >> > >> > Regards, >> > >> > Rajini >> > >> > >> > On Fri, Aug 16, 2019 at 6:43 PM Don Bosco Durai < >> bosco@apache.org> >> > wrote: >> > >> > > Hi Rajini >> > > >> > > Assuming this doesn't affect custom plugins, I don't have an= y >> > concerns >> > > regarding this. >> > > >> > > I do have one question regarding: >> > > >> > > "For authorizers that don=E2=80=99t store metadata in ZooKee= per, >> ensure that >> > > authorizer metadata for each listener is available before >> starting >> > up the >> > > listener. This enables different authorization metadata >> stores for >> > > different listeners." >> > > >> > > Since Ranger uses its own database for storing policies, do >> you >> > anticipate >> > > any issues? >> > > >> > > Thanks >> > > >> > > Bosco >> > > >> > > >> > > =EF=BB=BFOn 8/16/19, 6:49 AM, "Rajini Sivaram" < >> rajinisivaram@gmail.com> >> > wrote: >> > > >> > > Hi all, >> > > >> > > I made another change to the KIP. The KIP was originally >> > proposing to >> > > extend SimpleAclAuthorizer to also implement the new API >> (in >> > addition >> > > to >> > > the existing API). But since we use the new API when >> available, >> > this >> > > can >> > > break custom authorizers that extend this class and >> override >> > specific >> > > methods of the old API. To avoid breaking any existing >> custom >> > > implementations that extend this class, particularly >> because it >> > is in >> > > the >> > > public package kafka.security.auth, the KIP now proposes >> to >> > retain the >> > > old >> > > authorizer as-is. SimpleAclAuthorizer will continue to >> > implement the >> > > old >> > > API, but will be deprecated. A new authorizer >> implementation >> > > kafka.security.authorizer.AclAuthorizer will be added fo= r >> the >> > new API >> > > (this >> > > will not be in the public package). >> > > >> > > Please let me know if you have any concerns. >> > > >> > > Regards, >> > > >> > > Rajini >> > > >> > > >> > > On Fri, Aug 16, 2019 at 8:48 AM Rajini Sivaram < >> > > rajinisivaram@gmail.com> >> > > wrote: >> > > >> > > > Thanks Colin. >> > > > >> > > > If there are no other concerns, I will start vote late= r >> today. >> > Many >> > > thanks >> > > > to every one for the feedback. >> > > > >> > > > Regards, >> > > > >> > > > Rajini >> > > > >> > > > >> > > > On Thu, Aug 15, 2019 at 11:57 PM Colin McCabe < >> > cmccabe@apache.org> >> > > wrote: >> > > > >> > > >> Thanks, Rajini. It looks good to me. >> > > >> >> > > >> best, >> > > >> Colin >> > > >> >> > > >> >> > > >> On Thu, Aug 15, 2019, at 11:37, Rajini Sivaram wrote: >> > > >> > Hi Colin, >> > > >> > >> > > >> > Thanks for the review. I have updated the KIP to >> move the >> > > interfaces for >> > > >> > request context and server info to the authorizer >> package. >> > These >> > > are now >> > > >> > called AuthorizableRequestContext and >> AuthorizerServerInfo. >> > > Endpoint is >> > > >> now >> > > >> > a class in org.apache.kafka.common to make it >> reusable >> > since we >> > > already >> > > >> > have multiple implementations of it. I have removed >> > requestName >> > > from the >> > > >> > request context interface since authorizers can >> distinguish >> > > follower >> > > >> fetch >> > > >> > and consumer fetch from the operation being >> authorized. So >> > 16-bit >> > > >> request >> > > >> > type should be sufficient for audit logging. Also >> replaced >> > > AuditFlag >> > > >> with >> > > >> > two booleans as you suggested. >> > > >> > >> > > >> > Can you take another look and see if the KIP is >> ready for >> > voting? >> > > >> > >> > > >> > Thanks for all your help! >> > > >> > >> > > >> > Regards, >> > > >> > >> > > >> > Rajini >> > > >> > >> > > >> > On Wed, Aug 14, 2019 at 8:59 PM Colin McCabe < >> > cmccabe@apache.org> >> > > >> wrote: >> > > >> > >> > > >> > > Hi Rajini, >> > > >> > > >> > > >> > > I think it would be good to rename >> KafkaRequestContext to >> > > something >> > > >> like >> > > >> > > AuthorizableRequestContext, and put it in the >> > > >> > > org.apache.kafka.server.authorizer namespace. If >> we put >> > it in >> > > the >> > > >> > > org.apache.kafka.common namespace, then it's not >> really >> > clear >> > > that >> > > >> it's >> > > >> > > part of the Authorizer API. Since this class is >> purely an >> > > interface, >> > > >> with >> > > >> > > no concrete implementation of anything, there's >> nothing >> > common >> > > to >> > > >> really >> > > >> > > reuse in any case. We definitely don't want >> someone to >> > > accidentally >> > > >> add or >> > > >> > > remove methods because they think this is just >> another >> > internal >> > > class >> > > >> used >> > > >> > > for requests. >> > > >> > > >> > > >> > > The BrokerInfo class is a nice improvement. It >> looks >> > like it >> > > will be >> > > >> > > useful for passing in information about the >> context we're >> > > running >> > > >> in. It >> > > >> > > would be nice to call this class ServerInfo rathe= r >> than >> > > BrokerInfo, >> > > >> since >> > > >> > > we will want to run the authorizer on controllers >> as well >> > as on >> > > >> brokers, >> > > >> > > and the controller may run as a separate process >> post >> > KIP-500. >> > > I also >> > > >> > > think that this class should be in the >> > > >> org.apache.kafka.server.authorizer >> > > >> > > namespace. Again, it is an interface, not a >> concrete >> > > implementation, >> > > >> and >> > > >> > > it's an interface that is very specifically for t= he >> > authorizer. >> > > >> > > >> > > >> > > I agree that we probably don't have enough >> information >> > > preserved for >> > > >> > > requests currently to always know what entity mad= e >> them. >> > So we >> > > can >> > > >> leave >> > > >> > > that out for now (except in the special case of >> Fetch). >> > > Perhaps we >> > > >> can add >> > > >> > > this later if it's needed. >> > > >> > > >> > > >> > > I understand the intention behind AuthorizationMo= de >> > (which is >> > > now >> > > >> called >> > > >> > > AuditFlag in the latest revision). But it still >> feels >> > > complex. What >> > > >> if we >> > > >> > > just had two booleans in Action: logSuccesses and >> > logFailures? >> > > That >> > > >> seems >> > > >> > > to cover all the cases here. MANDATORY_AUTHORIZE >> =3D true, >> > true. >> > > >> > > OPTIONAL_AUTHORIZE =3D true, false. FILTER =3D t= rue, >> false. >> > > >> LIST_AUTHORIZED =3D >> > > >> > > false, false. Would there be anything lost versu= s >> having >> > the >> > > enum? >> > > >> > > >> > > >> > > best, >> > > >> > > Colin >> > > >> > > >> > > >> > > >> > > >> > > On Wed, Aug 14, 2019, at 06:29, Mickael Maison >> wrote: >> > > >> > > > Hi Rajini, >> > > >> > > > >> > > >> > > > Thanks for the KIP! >> > > >> > > > I really like that authorize() will be able to >> take a >> > batch of >> > > >> > > > requests, this will speed up many >> implementations! >> > > >> > > > >> > > >> > > > On Tue, Aug 13, 2019 at 5:57 PM Rajini Sivaram = < >> > > >> rajinisivaram@gmail.com> >> > > >> > > wrote: >> > > >> > > > > >> > > >> > > > > Thanks David! I have fixed the typo. >> > > >> > > > > >> > > >> > > > > Also made a couple of changes to make the >> context >> > > interfaces more >> > > >> > > generic. >> > > >> > > > > KafkaRequestContext now returns the 16-bit AP= I >> key as >> > Colin >> > > >> suggested >> > > >> > > as >> > > >> > > > > well as the friendly name used in metrics >> which are >> > useful >> > > in >> > > >> audit >> > > >> > > logs. >> > > >> > > > > `Authorizer#start` is now provided a generic >> > `BrokerInfo` >> > > >> interface >> > > >> > > that >> > > >> > > > > gives cluster id, broker id and endpoint >> information. >> > The >> > > generic >> > > >> > > interface >> > > >> > > > > can potentially be used in other broker >> plugins in >> > future >> > > and >> > > >> provides >> > > >> > > > > dynamically generated configs like broker id >> and ports >> > > which are >> > > >> > > currently >> > > >> > > > > not available to plugins unless these configs >> are >> > statically >> > > >> > > configured. >> > > >> > > > > Please let me know if there are any concerns. >> > > >> > > > > >> > > >> > > > > Regards, >> > > >> > > > > >> > > >> > > > > Rajini >> > > >> > > > > >> > > >> > > > > On Tue, Aug 13, 2019 at 4:30 PM David Jacot < >> > > djacot@confluent.io> >> > > >> > > wrote: >> > > >> > > > > >> > > >> > > > > > Hi Rajini, >> > > >> > > > > > >> > > >> > > > > > Thank you for the update! It looks good to >> me. >> > There is a >> > > typo >> > > >> in the >> > > >> > > > > > `AuditFlag` enum: `MANDATORY_AUTHOEIZE` -> >> > > >> `MANDATORY_AUTHORIZE`. >> > > >> > > > > > >> > > >> > > > > > Regards, >> > > >> > > > > > David >> > > >> > > > > > >> > > >> > > > > > On Mon, Aug 12, 2019 at 2:54 PM Rajini >> Sivaram < >> > > >> > > rajinisivaram@gmail.com> >> > > >> > > > > > wrote: >> > > >> > > > > > >> > > >> > > > > > > Hi David, >> > > >> > > > > > > >> > > >> > > > > > > Thanks for reviewing the KIP! Since >> questions >> > about >> > > >> `authorization >> > > >> > > mode` >> > > >> > > > > > > and `count` have come up multiple times, = I >> have >> > renamed >> > > both. >> > > >> > > > > > > >> > > >> > > > > > > 1) Renamed `count` to >> `resourceReferenceCount`. >> > It is >> > > the >> > > >> number >> > > >> > > of times >> > > >> > > > > > > the resource being authorized is referenc= ed >> > within the >> > > >> request. >> > > >> > > > > > > >> > > >> > > > > > > 2) Renamed `AuthorizationMode` to >> `AuditFlag`. It >> > is >> > > provided >> > > >> to >> > > >> > > improve >> > > >> > > > > > > audit logging in the authorizer. The enum >> values >> > have >> > > javadoc >> > > >> which >> > > >> > > > > > > indicate how the authorization result is >> used in >> > each >> > > of the >> > > >> modes >> > > >> > > to >> > > >> > > > > > > enable authorizers to log audit messages >> at the >> > right >> > > severity >> > > >> > > level. >> > > >> > > > > > > >> > > >> > > > > > > Regards, >> > > >> > > > > > > >> > > >> > > > > > > Rajini >> > > >> > > > > > > >> > > >> > > > > > > On Mon, Aug 12, 2019 at 5:54 PM David >> Jacot < >> > > >> djacot@confluent.io> >> > > >> > > wrote: >> > > >> > > > > > > >> > > >> > > > > > > > Hi Rajini, >> > > >> > > > > > > > >> > > >> > > > > > > > Thank you for the KIP. Overall, it look= s >> good >> > to me. >> > > I have >> > > >> few >> > > >> > > > > > > > questions/suggestions: >> > > >> > > > > > > > >> > > >> > > > > > > > 1. It is hard to grasp what >> `Action#count` is >> > for. I >> > > guess I >> > > >> > > understand >> > > >> > > > > > > > where you want to go with it but it too= k >> me a >> > while to >> > > >> figure it >> > > >> > > out. >> > > >> > > > > > > > Perhaps, we could come up with a better >> name >> > than >> > > `count`? >> > > >> > > > > > > > >> > > >> > > > > > > > 2. I had a hard time trying to >> understand the >> > > >> `AuthorizationMode` >> > > >> > > > > > > concept, >> > > >> > > > > > > > especially wrt. the OPTIONAL one. My >> > understanding is >> > > that >> > > >> an >> > > >> > > ACL is >> > > >> > > > > > > either >> > > >> > > > > > > > defined or not. Could you elaborate a >> bit more >> > on >> > > that? >> > > >> > > > > > > > >> > > >> > > > > > > > Thanks, >> > > >> > > > > > > > David >> > > >> > > > > > > > >> > > >> > > > > > > > On Fri, Aug 9, 2019 at 10:26 PM Don >> Bosco Durai >> > < >> > > >> > > bosco@apache.org> >> > > >> > > > > > > wrote: >> > > >> > > > > > > > >> > > >> > > > > > > > > Hi Rajini >> > > >> > > > > > > > > >> > > >> > > > > > > > > 3.2 - This makes sense. Thanks for >> clarifying. >> > > >> > > > > > > > > >> > > >> > > > > > > > > Rest looks fine. Once the >> implementations are >> > done, >> > > it >> > > >> will be >> > > >> > > more >> > > >> > > > > > > clear >> > > >> > > > > > > > > on the different values RequestType >> and Mode. >> > > >> > > > > > > > > >> > > >> > > > > > > > > Thanks >> > > >> > > > > > > > > >> > > >> > > > > > > > > Bosco >> > > >> > > > > > > > > >> > > >> > > > > > > > > >> > > >> > > > > > > > > =EF=BB=BFOn 8/9/19, 5:08 AM, "Rajini = Sivaram" < >> > > >> rajinisivaram@gmail.com >> > > >> > > > >> > > >> > > > > > wrote: >> > > >> > > > > > > > > >> > > >> > > > > > > > > Hi Don, >> > > >> > > > > > > > > >> > > >> > > > > > > > > Thanks for the suggestions. A few >> > responses >> > > below: >> > > >> > > > > > > > > >> > > >> > > > > > > > > 3.1 Can rename and improve docs i= f >> we keep >> > > this. Let's >> > > >> > > finish the >> > > >> > > > > > > > > discussion on Colin's suggestions >> > regarding this >> > > >> first. >> > > >> > > > > > > > > 3.2 No, I was thinking of some >> requests >> > that >> > > have an >> > > >> old >> > > >> > > way of >> > > >> > > > > > > > > authorizing >> > > >> > > > > > > > > and a new way where we have >> retained the >> > old >> > > way for >> > > >> > > backward >> > > >> > > > > > > > > compatibility. One example is >> > Cluster:Create >> > > >> permission to >> > > >> > > create >> > > >> > > > > > > > > topics. >> > > >> > > > > > > > > We have replaced this with >> fine-grained >> > topic >> > > create >> > > >> > > access using >> > > >> > > > > > > > > Topic:Create >> > > >> > > > > > > > > for topic patterns. But we still >> check if >> > user >> > > has >> > > >> > > Cluster:Create >> > > >> > > > > > > > > first. If >> > > >> > > > > > > > > Denied, the deny is ignored and w= e >> check >> > > >> Topic:Create. We >> > > >> > > dont >> > > >> > > > > > want >> > > >> > > > > > > > to >> > > >> > > > > > > > > log >> > > >> > > > > > > > > DENY for Cluster:Create at INFO >> level for >> > this, >> > > since >> > > >> this >> > > >> > > is >> > > >> > > > > > not a >> > > >> > > > > > > > > mandatory ACL for creating topics= . >> We >> > will get >> > > >> appropriate >> > > >> > > logs >> > > >> > > > > > > from >> > > >> > > > > > > > > the >> > > >> > > > > > > > > subsequent Topic:Create in this >> case. >> > > >> > > > > > > > > 3.3 They are not quite the same. >> FILTER >> > implies >> > > that >> > > >> user >> > > >> > > > > > actually >> > > >> > > > > > > > > requested access to and performed >> some >> > > operation on >> > > >> the >> > > >> > > filtered >> > > >> > > > > > > > > resources. >> > > >> > > > > > > > > LIST_AUTHORZED did not result in >> any >> > actual >> > > access. >> > > >> User >> > > >> > > simply >> > > >> > > > > > > > wanted >> > > >> > > > > > > > > to >> > > >> > > > > > > > > know what they are allowed to >> access. >> > > >> > > > > > > > > 3.4 Request types are Produce, >> JoinGroup, >> > > OffsetCommit >> > > >> > > etc. So >> > > >> > > > > > that >> > > >> > > > > > > > is >> > > >> > > > > > > > > different from authorization mode= , >> > operation >> > > etc. >> > > >> > > > > > > > > >> > > >> > > > > > > > > >> > > >> > > > > > > > > On Thu, Aug 8, 2019 at 11:36 PM >> Don Bosco >> > Durai >> > > < >> > > >> > > > > > bosco@apache.org> >> > > >> > > > > > > > > wrote: >> > > >> > > > > > > > > >> > > >> > > > > > > > > > Hi Rajini >> > > >> > > > > > > > > > >> > > >> > > > > > > > > > Thanks for clarifying. This is >> very >> > helpful... >> > > >> > > > > > > > > > >> > > >> > > > > > > > > > #1 - On the Ranger side, we >> should be >> > able to >> > > handle >> > > >> > > multiple >> > > >> > > > > > > > > requests at >> > > >> > > > > > > > > > the same time. I was just not >> sure how >> > much >> > > >> processing >> > > >> > > overhead >> > > >> > > > > > > > will >> > > >> > > > > > > > > be >> > > >> > > > > > > > > > there on the Broker side to >> split and >> > then >> > > >> consolidate >> > > >> > > the >> > > >> > > > > > > results. >> > > >> > > > > > > > > If it >> > > >> > > > > > > > > > is negligible, then this is th= e >> better >> > way. >> > > It will >> > > >> > > make it >> > > >> > > > > > > future >> > > >> > > > > > > > > proof. >> > > >> > > > > > > > > > #2 - I agree, having a single >> "start" >> > call >> > > makes it >> > > >> > > cleaner. >> > > >> > > > > > The >> > > >> > > > > > > > > > Authorization implementation >> will only >> > have >> > > to do >> > > >> the >> > > >> > > > > > > > initialization >> > > >> > > > > > > > > only >> > > >> > > > > > > > > > once. >> > > >> > > > > > > > > > #3.1 - Thanks for the >> clarification. I >> > think >> > > it >> > > >> makes >> > > >> > > sense to >> > > >> > > > > > > have >> > > >> > > > > > > > > this. >> > > >> > > > > > > > > > The term "Mode" might not be >> explicit >> > enough. >> > > >> > > Essentially it >> > > >> > > > > > > seems >> > > >> > > > > > > > > you want >> > > >> > > > > > > > > > the Authorizer to know the >> > Intent/Purpose of >> > > the >> > > >> > > authorize call >> > > >> > > > > > > and >> > > >> > > > > > > > > let the >> > > >> > > > > > > > > > Authorizer decide what to log a= s >> Audit >> > event. >> > > >> Changing >> > > >> > > the >> > > >> > > > > > > > > class/field name >> > > >> > > > > > > > > > or giving more documentation >> will do. >> > > >> > > > > > > > > > #3.2 - Regarding the option >> "OPTIONAL". >> > Are >> > > you >> > > >> thinking >> > > >> > > from >> > > >> > > > > > > > > chaining >> > > >> > > > > > > > > > multiple Authorizer? If so, I >> am not >> > sure >> > > whether >> > > >> the >> > > >> > > Broker >> > > >> > > > > > > would >> > > >> > > > > > > > > have >> > > >> > > > > > > > > > enough information to make that >> > decision. I >> > > feel the >> > > >> > > Authorizer >> > > >> > > > > > > > will >> > > >> > > > > > > > > be the >> > > >> > > > > > > > > > one who would have that >> knowledge. E.g. >> > in >> > > Ranger >> > > >> we have >> > > >> > > > > > > explicit >> > > >> > > > > > > > > deny, >> > > >> > > > > > > > > > which means no matter what, the >> request >> > > should be >> > > >> denied >> > > >> > > for >> > > >> > > > > > the >> > > >> > > > > > > > > user/group >> > > >> > > > > > > > > > or condition. So if you are >> thinking of >> > > chaining >> > > >> > > Authorizers, >> > > >> > > > > > > then >> > > >> > > > > > > > > > responses should have the third >> state, >> > e.g. >> > > >> > > "DENIED_FINAL", in >> > > >> > > > > > > > which >> > > >> > > > > > > > > case >> > > >> > > > > > > > > > if there is an Authorization >> chain, it >> > will >> > > be stop >> > > >> and >> > > >> > > the >> > > >> > > > > > > request >> > > >> > > > > > > > > will be >> > > >> > > > > > > > > > denied and if it is just denied= , >> then >> > you >> > > might fall >> > > >> > > back to >> > > >> > > > > > next >> > > >> > > > > > > > > > authorizer. If we don't have >> chaining of >> > > >> Authorizing in >> > > >> > > mind, >> > > >> > > > > > > then >> > > >> > > > > > > > we >> > > >> > > > > > > > > > should reconsider OPTIONAL for >> now. Or >> > > clarify under >> > > >> > > which >> > > >> > > > > > > scenario >> > > >> > > > > > > > > > OPTIONAL will be called. >> > > >> > > > > > > > > > #3.3 Regarding, FILTER v/s >> > LIST_AUTHORIZED, >> > > isn't >> > > >> > > > > > > LIST_AUTHORIZED a >> > > >> > > > > > > > > > special case for "FILTER"? >> > > >> > > > > > > > > > #3.4 KafkaRequestContext. >> requestType() >> > v/s >> > > Action. >> > > >> > > > > > > > > authorizationMode. I >> > > >> > > > > > > > > > am not sure about the overlap o= r >> > ambiguity. >> > > >> > > > > > > > > > #4 - Cool. This is good, it wil= l >> be less >> > > stress on >> > > >> the >> > > >> > > > > > > Authorizer. >> > > >> > > > > > > > > Ranger >> > > >> > > > > > > > > > already supports the "count" >> concept >> > and also >> > > has >> > > >> > > batching >> > > >> > > > > > > > > capability to >> > > >> > > > > > > > > > aggregate similar requests to >> reduce the >> > > number of >> > > >> audit >> > > >> > > logs >> > > >> > > > > > to >> > > >> > > > > > > > > write. We >> > > >> > > > > > > > > > should be able to pass this >> through. >> > > >> > > > > > > > > > #5 - Assuming if the object >> instance is >> > going >> > > out of >> > > >> > > scope, we >> > > >> > > > > > > > > should be >> > > >> > > > > > > > > > fine. Not a super important ask= . >> Ranger >> > is >> > > already >> > > >> > > catching >> > > >> > > > > > KILL >> > > >> > > > > > > > > signal for >> > > >> > > > > > > > > > clean up. >> > > >> > > > > > > > > > >> > > >> > > > > > > > > > Thanks again. These are good >> > enhancements. >> > > >> Appreciate >> > > >> > > your >> > > >> > > > > > > efforts >> > > >> > > > > > > > > here. >> > > >> > > > > > > > > > >> > > >> > > > > > > > > > Bosco >> > > >> > > > > > > > > > >> > > >> > > > > > > > > > >> > > >> > > > > > > > > > >> > > >> > > > > > > > > > =EF=BB=BFOn 8/8/19, 2:03 AM, "R= ajini >> Sivaram" < >> > > >> > > rajinisivaram@gmail.com >> > > >> > > > > > > >> > > >> > > > > > > > > wrote: >> > > >> > > > > > > > > > >> > > >> > > > > > > > > > Hi Don, >> > > >> > > > > > > > > > >> > > >> > > > > > > > > > Thanks for reviewing the KI= P. >> > > >> > > > > > > > > > >> > > >> > > > > > > > > > 1. I had this originally as >> a single >> > > Action, but >> > > >> > > thought it >> > > >> > > > > > > may >> > > >> > > > > > > > > be >> > > >> > > > > > > > > > useful >> > > >> > > > > > > > > > to support batched authoriz= e >> calls >> > as >> > > well and >> > > >> keep >> > > >> > > it >> > > >> > > > > > > > > consistent with >> > > >> > > > > > > > > > other methods. Single >> requests can >> > contain >> > > >> multiple >> > > >> > > topics. >> > > >> > > > > > > For >> > > >> > > > > > > > > > example a >> > > >> > > > > > > > > > produce request can contain >> records >> > for >> > > several >> > > >> > > partitions >> > > >> > > > > > of >> > > >> > > > > > > > > different >> > > >> > > > > > > > > > topics. Broker could >> potentially >> > > authorize these >> > > >> > > together. >> > > >> > > > > > > For >> > > >> > > > > > > > > > SimpleAclAuthorizer, batche= d >> > authorize >> > > methods >> > > >> don't >> > > >> > > > > > provide >> > > >> > > > > > > > any >> > > >> > > > > > > > > > optimisation since lookup i= s >> based >> > on >> > > resources >> > > >> > > followed by >> > > >> > > > > > > the >> > > >> > > > > > > > > > matching >> > > >> > > > > > > > > > logic. But some authorizers >> may >> > manage >> > > ACLs by >> > > >> user >> > > >> > > > > > principal >> > > >> > > > > > > > > rather >> > > >> > > > > > > > > > than >> > > >> > > > > > > > > > resource and may be able to >> optimize >> > > batched >> > > >> > > requests. I am >> > > >> > > > > > > ok >> > > >> > > > > > > > > with >> > > >> > > > > > > > > > using >> > > >> > > > > > > > > > single Action if this is >> likely to >> > cause >> > > issues. >> > > >> > > > > > > > > > 2. If you have two >> listeners, one >> > for >> > > >> inter-broker >> > > >> > > traffic >> > > >> > > > > > > and >> > > >> > > > > > > > > another >> > > >> > > > > > > > > > for >> > > >> > > > > > > > > > external clients, start >> method is >> > invoked >> > > twice, >> > > >> > > once for >> > > >> > > > > > > each >> > > >> > > > > > > > > > listener. On >> > > >> > > > > > > > > > second thought, that may be >> > confusing and >> > > a >> > > >> single >> > > >> > > start() >> > > >> > > > > > > > > invocation >> > > >> > > > > > > > > > that >> > > >> > > > > > > > > > provides all listener >> information >> > and >> > > returns >> > > >> > > multiple >> > > >> > > > > > > futures >> > > >> > > > > > > > > would be >> > > >> > > > > > > > > > better. Will update the KIP= . >> > > >> > > > > > > > > > 3. A typical example is a >> consumer >> > > subscribing >> > > >> to a >> > > >> > > regex >> > > >> > > > > > > > > pattern. We >> > > >> > > > > > > > > > request all topic metadata >> from the >> > > broker in >> > > >> order >> > > >> > > to >> > > >> > > > > > decide >> > > >> > > > > > > > > whether >> > > >> > > > > > > > > > the >> > > >> > > > > > > > > > pattern matches, expecting = to >> > receive a >> > > list of >> > > >> > > authorised >> > > >> > > > > > > > > topics. The >> > > >> > > > > > > > > > user >> > > >> > > > > > > > > > is not asking to subscribe >> to an >> > > unauthorized >> > > >> topic. >> > > >> > > If >> > > >> > > > > > there >> > > >> > > > > > > > > are 10000 >> > > >> > > > > > > > > > topics in the cluster and >> the user >> > has >> > > access >> > > >> to 100 >> > > >> > > of >> > > >> > > > > > them, >> > > >> > > > > > > > at >> > > >> > > > > > > > > the >> > > >> > > > > > > > > > moment >> > > >> > > > > > > > > > we log 9900 DENIED log >> entries at >> > INFO >> > > level in >> > > >> > > > > > > > > SimpleAclAuthorizer. >> > > >> > > > > > > > > > The >> > > >> > > > > > > > > > proposal is to authorize th= is >> > request with >> > > >> > > > > > > > > AuthorizationMode.FILTER, so >> > > >> > > > > > > > > > that authorizers can log >> resources >> > that >> > > are >> > > >> filtered >> > > >> > > out at >> > > >> > > > > > > > > lower level >> > > >> > > > > > > > > > like DEBUG since this is no= t >> an >> > attempt to >> > > >> access >> > > >> > > > > > > unauthorized >> > > >> > > > > > > > > > resources. >> > > >> > > > > > > > > > Brokers already handle thes= e >> > differently >> > > since >> > > >> no >> > > >> > > > > > > authorization >> > > >> > > > > > > > > error >> > > >> > > > > > > > > > is >> > > >> > > > > > > > > > returned to the client in >> these >> > cases. >> > > Providing >> > > >> > > > > > > authorization >> > > >> > > > > > > > > mode to >> > > >> > > > > > > > > > authorizers enables >> authorizer >> > > implementations >> > > >> to >> > > >> > > generate >> > > >> > > > > > > > > better audit >> > > >> > > > > > > > > > logs. >> > > >> > > > > > > > > > 4. Each request may contain >> multiple >> > > instances >> > > >> of >> > > >> > > the same >> > > >> > > > > > > > > authorizable >> > > >> > > > > > > > > > resource. For example a >> produce >> > request >> > > may >> > > >> contain >> > > >> > > records >> > > >> > > > > > > for >> > > >> > > > > > > > > 10 >> > > >> > > > > > > > > > partitions of the same >> topic. At the >> > > moment, we >> > > >> > > invoke >> > > >> > > > > > > > authorize >> > > >> > > > > > > > > > method 10 >> > > >> > > > > > > > > > times. The proposal is to >> invoke it >> > once >> > > with >> > > >> > > count=3D10. The >> > > >> > > > > > > > > count is >> > > >> > > > > > > > > > provided to authorizer just >> for >> > audit >> > > logging >> > > >> > > purposes. >> > > >> > > > > > > > > > 5. Authorizer implements >> Closeable, >> > so you >> > > >> could use >> > > >> > > > > > close() >> > > >> > > > > > > to >> > > >> > > > > > > > > flush >> > > >> > > > > > > > > > audits? >> > > >> > > > > > > > > > >> > > >> > > > > > > > > > On Thu, Aug 8, 2019 at 7:01 >> AM Don >> > Bosco >> > > Durai < >> > > >> > > > > > > > bosco@apache.org >> > > >> > > > > > > > > > >> > > >> > > > > > > > > > wrote: >> > > >> > > > > > > > > > >> > > >> > > > > > > > > > > Rajini >> > > >> > > > > > > > > > > >> > > >> > > > > > > > > > > Thanks for putting this >> together. >> > It is >> > > >> looking >> > > >> > > good. I >> > > >> > > > > > > have >> > > >> > > > > > > > > few >> > > >> > > > > > > > > > > questions... >> > > >> > > > > > > > > > > >> > > >> > > > > > > > > > > 1. >> List >> > > authorize(..., >> > > >> > > List >> > > >> > > > > > > > > actions). >> > > >> > > > > > > > > > Do you >> > > >> > > > > > > > > > > see a scenario where the >> broker >> > will >> > > call >> > > >> > > authorize for >> > > >> > > > > > > > > multiple >> > > >> > > > > > > > > > topics at >> > > >> > > > > > > > > > > the same time? I can >> understand >> > that >> > > during >> > > >> > > > > > > creating/deleting >> > > >> > > > > > > > > ACLS, >> > > >> > > > > > > > > > > multiple permissions for >> multiple >> > > resources >> > > >> might >> > > >> > > be >> > > >> > > > > > done. >> > > >> > > > > > > > For >> > > >> > > > > > > > > > authorize >> > > >> > > > > > > > > > > call, would this be a >> case? And >> > does the >> > > >> Authorize >> > > >> > > > > > > > > implementation >> > > >> > > > > > > > > > will be >> > > >> > > > > > > > > > > able to do performance >> > optimization >> > > because of >> > > >> > > this? Or >> > > >> > > > > > > > should >> > > >> > > > > > > > > we >> > > >> > > > > > > > > > just keep >> > > >> > > > > > > > > > > it simple? I don't see it >> as an >> > issue >> > > from >> > > >> Apache >> > > >> > > Ranger >> > > >> > > > > > > > side, >> > > >> > > > > > > > > but >> > > >> > > > > > > > > > just >> > > >> > > > > > > > > > > checking to see whether w= e >> need >> > to be >> > > aware of >> > > >> > > something. >> > > >> > > > > > > > > > > 2. Should I assume that t= he >> > > SecurityProtocol >> > > >> passed >> > > >> > > > > > during >> > > >> > > > > > > > > start and >> > > >> > > > > > > > > > the >> > > >> > > > > > > > > > > one return by >> > > >> > > KafkaRequestContext.securityProtocol() will >> > > >> > > > > > > be >> > > >> > > > > > > > > the >> > > >> > > > > > > > > > same? >> > > >> > > > > > > > > > > CompletableFuture >> > start(String >> > > >> listenerName, >> > > >> > > > > > > > > SecurityProtocol >> > > >> > > > > > > > > > > securityProtocol); >> > > >> > > > > > > > > > > >> > KafkaRequestContext.securityProtocol() >> > > >> > > > > > > > > > > 3. What is the purpose of >> > > AuthorizationMode? >> > > >> How >> > > >> > > does the >> > > >> > > > > > > > > broker >> > > >> > > > > > > > > > decide >> > > >> > > > > > > > > > > what mode to use when the >> > authorize() >> > > method >> > > >> is >> > > >> > > called? >> > > >> > > > > > > > > > > 4. Can we clarify "count" >> in >> > Action a >> > > bit >> > > >> more? >> > > >> > > How is it >> > > >> > > > > > > > used? >> > > >> > > > > > > > > > > 5. Do you feel having >> "stop" >> > along with >> > > >> "start" be >> > > >> > > > > > helpful? >> > > >> > > > > > > > > E.g. In >> > > >> > > > > > > > > > Ranger >> > > >> > > > > > > > > > > we try to optimize the >> Audit >> > writing by >> > > >> caching >> > > >> > > the logs >> > > >> > > > > > > for >> > > >> > > > > > > > a >> > > >> > > > > > > > > fixed >> > > >> > > > > > > > > > > interval. But when the >> Broker >> > > terminates, we >> > > >> do a >> > > >> > > forced >> > > >> > > > > > > > flush. >> > > >> > > > > > > > > > Having an >> > > >> > > > > > > > > > > explicit "stop" might giv= e >> us a >> > formal >> > > way to >> > > >> > > flush our >> > > >> > > > > > > > audits. >> > > >> > > > > > > > > > > >> > > >> > > > > > > > > > > Thanks >> > > >> > > > > > > > > > > >> > > >> > > > > > > > > > > Bosco >> > > >> > > > > > > > > > > >> > > >> > > > > > > > > > > =EF=BB=BFOn 8/7/19, 3:59 = PM, >> "Rajini >> > Sivaram" < >> > > >> > > > > > > > rajinisivaram@gmail.com >> > > >> > > > > > > > > > >> > > >> > > > > > > > > > wrote: >> > > >> > > > > > > > > > > >> > > >> > > > > > > > > > > Hi Ron/Harsha/Satish, >> > > >> > > > > > > > > > > >> > > >> > > > > > > > > > > Thanks for reviewing >> the KIP! >> > > >> > > > > > > > > > > >> > > >> > > > > > > > > > > We should perhaps hav= e >> a wider >> > > discussion >> > > >> > > outside >> > > >> > > > > > this >> > > >> > > > > > > > KIP >> > > >> > > > > > > > > for >> > > >> > > > > > > > > > > refactoring >> > > >> > > > > > > > > > > clients so that other= s >> who >> > are not >> > > >> following >> > > >> > > this KIP >> > > >> > > > > > > > also >> > > >> > > > > > > > > > notice the >> > > >> > > > > > > > > > > discussion. Satish, >> would you >> > like >> > > to >> > > >> start a >> > > >> > > > > > > discussion >> > > >> > > > > > > > > thread >> > > >> > > > > > > > > > on dev? >> > > >> > > > > > > > > > > >> > > >> > > > > > > > > > > Regards, >> > > >> > > > > > > > > > > >> > > >> > > > > > > > > > > Rajini >> > > >> > > > > > > > > > > >> > > >> > > > > > > > > > > >> > > >> > > > > > > > > > > On Wed, Aug 7, 2019 a= t >> 6:21 PM >> > > Satish >> > > >> Duggana < >> > > >> > > > > > > > > > > satish.duggana@gmail.com> >> > > >> > > > > > > > > > > wrote: >> > > >> > > > > > > > > > > >> > > >> > > > > > > > > > > > I felt the same nee= d >> when >> > we want >> > > to >> > > >> add a >> > > >> > > > > > pluggable >> > > >> > > > > > > > API >> > > >> > > > > > > > > for >> > > >> > > > > > > > > > core >> > > >> > > > > > > > > > > > server >> functionality. This >> > does >> > > not >> > > >> need to >> > > >> > > be part >> > > >> > > > > > > of >> > > >> > > > > > > > > this >> > > >> > > > > > > > > > KIP, it >> > > >> > > > > > > > > > > > can be a separate >> KIP. I can >> > > contribute >> > > >> those >> > > >> > > > > > > > refactoring >> > > >> > > > > > > > > > changes if >> > > >> > > > > > > > > > > > others are OK with >> that. >> > > >> > > > > > > > > > > > >> > > >> > > > > > > > > > > > It is better to hav= e >> a >> > structure >> > > like >> > > >> below. >> > > >> > > > > > > > > > > > >> > > >> > > > > > > > > > > > kafka-common: >> > > >> > > > > > > > > > > > common classes whic= h >> can be >> > used >> > > in any >> > > >> of >> > > >> > > the >> > > >> > > > > > other >> > > >> > > > > > > > > modules >> > > >> > > > > > > > > > in Kafka >> > > >> > > > > > > > > > > > like client, >> > Kafka-server-common >> > > and >> > > >> server >> > > >> > > etc. >> > > >> > > > > > > > > > > > >> > > >> > > > > > > > > > > > kafka-client-common= : >> > > >> > > > > > > > > > > > common classes whic= h >> can be >> > used >> > > in the >> > > >> > > client >> > > >> > > > > > > module. >> > > >> > > > > > > > > This >> > > >> > > > > > > > > > can be >> > > >> > > > > > > > > > > > part of client modu= le >> > itself. >> > > >> > > > > > > > > > > > >> > > >> > > > > > > > > > > > kafka-server-common= : >> > > >> > > > > > > > > > > > classes required >> only for >> > > kafka-server. >> > > >> > > > > > > > > > > > >> > > >> > > > > > > > > > > > Thanks. >> > > >> > > > > > > > > > > > Satish. >> > > >> > > > > > > > > > > > >> > > >> > > > > > > > > > > > On Wed, Aug 7, 2019 >> at 9:28 >> > PM >> > > Harsha >> > > >> > > Chintalapani >> > > >> > > > > > < >> > > >> > > > > > > > > > kafka@harsha.io> >> > > >> > > > > > > > > > > > wrote: >> > > >> > > > > > > > > > > > > >> > > >> > > > > > > > > > > > > Thanks for the KI= P >> Rajini. >> > > >> > > > > > > > > > > > > Quick thought, it >> would >> > be good >> > > to >> > > >> have a >> > > >> > > common >> > > >> > > > > > > > module >> > > >> > > > > > > > > > outside of >> > > >> > > > > > > > > > > > clients >> > > >> > > > > > > > > > > > > that only applies >> to >> > server side >> > > >> > > interfaces & >> > > >> > > > > > > > changes. >> > > >> > > > > > > > > It >> > > >> > > > > > > > > > looks >> > > >> > > > > > > > > > > like we >> > > >> > > > > > > > > > > > are >> > > >> > > > > > > > > > > > > increasingly in >> favor of >> > using >> > > Java >> > > >> > > interface for >> > > >> > > > > > > > > pluggable >> > > >> > > > > > > > > > > modules on >> > > >> > > > > > > > > > > > the >> > > >> > > > > > > > > > > > > broker side. >> > > >> > > > > > > > > > > > > >> > > >> > > > > > > > > > > > > Thanks, >> > > >> > > > > > > > > > > > > Harsha >> > > >> > > > > > > > > > > > > >> > > >> > > > > > > > > > > > > >> > > >> > > > > > > > > > > > > On Tue, Aug 06, >> 2019 at >> > 2:31 PM, >> > > >> Rajini >> > > >> > > Sivaram < >> > > >> > > > > > > > > > > rajinisivaram@gmail.com >> > > >> > > > > > > > > > > > > >> > > >> > > > > > > > > > > > > wrote: >> > > >> > > > > > > > > > > > > >> > > >> > > > > > > > > > > > > > Hi all, >> > > >> > > > > > > > > > > > > > >> > > >> > > > > > > > > > > > > > I have created = a >> KIP to >> > > replace the >> > > >> Scala >> > > >> > > > > > > > Authorizer >> > > >> > > > > > > > > API >> > > >> > > > > > > > > > with a >> > > >> > > > > > > > > > > new >> > > >> > > > > > > > > > > > Java >> > > >> > > > > > > > > > > > > > API: >> > > >> > > > > > > > > > > > > > >> > > >> > > > > > > > > > > > > > - >> > > >> > > > > > > > > > > > > > >> > > >> > > > > > > >> > https://cwiki.apache.org/confluence/display/KAFKA/ >> > > >> > > > > > > > > > > > > > >> > > >> > > KIP-504+-+Add+new+Java+Authorizer+Interface >> > > >> > > > > > > > > > > > > > >> > > >> > > > > > > > > > > > > > This is >> replacement for >> > KIP-50 >> > > >> which was >> > > >> > > > > > accepted >> > > >> > > > > > > > > but never >> > > >> > > > > > > > > > > merged. >> > > >> > > > > > > > > > > > Apart >> > > >> > > > > > > > > > > > > > from moving to = a >> Java >> > API >> > > consistent >> > > >> > > with other >> > > >> > > > > > > > > pluggable >> > > >> > > > > > > > > > > interfaces >> > > >> > > > > > > > > > > > in the >> > > >> > > > > > > > > > > > > > broker, KIP-504 >> also >> > attempts >> > > to >> > > >> address >> > > >> > > known >> > > >> > > > > > > > > limitations >> > > >> > > > > > > > > > in the >> > > >> > > > > > > > > > > > > > authorizer. If >> you have >> > come >> > > across >> > > >> other >> > > >> > > > > > > > > limitations that >> > > >> > > > > > > > > > you >> > > >> > > > > > > > > > > would >> > > >> > > > > > > > > > > > like >> > > >> > > > > > > > > > > > > > to see addresse= d >> in the >> > new >> > > API, >> > > >> please >> > > >> > > raise >> > > >> > > > > > > these >> > > >> > > > > > > > > on the >> > > >> > > > > > > > > > > discussion >> > > >> > > > > > > > > > > > > > thread so that >> we can >> > > consider those >> > > >> > > too. All >> > > >> > > > > > > > > suggestions >> > > >> > > > > > > > > > and >> > > >> > > > > > > > > > > feedback >> > > >> > > > > > > > > > > > are >> > > >> > > > > > > > > > > > > > welcome. >> > > >> > > > > > > > > > > > > > >> > > >> > > > > > > > > > > > > > Thank you, >> > > >> > > > > > > > > > > > > > >> > > >> > > > > > > > > > > > > > Rajini >> > > >> > > > > > > > > > > > > > >> > > >> > > > > > > > > > > > >> > > >> > > > > > > > > > > >> > > >> > > > > > > > > > > >> > > >> > > > > > > > > > > >> > > >> > > > > > > > > > > >> > > >> > > > > > > > > > >> > > >> > > > > > > > > > >> > > >> > > > > > > > > > >> > > >> > > > > > > > > > >> > > >> > > > > > > > > >> > > >> > > > > > > > > >> > > >> > > > > > > > > >> > > >> > > > > > > > > >> > > >> > > > > > > > >> > > >> > > > > > > >> > > >> > > > > > >> > > >> > > > >> > > >> > > >> > > >> > >> > > >> >> > > > >> > > >> > > >> > > >> > > >> > > >> > >> > >> > >> > >> > >> >> >> >> --000000000000e6a0d20591b7d029--