kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ismael Juma <ism...@juma.me.uk>
Subject Re: [DISCUSS] KIP-117: Add a public AdministrativeClient API for Kafka admin operations
Date Wed, 08 Feb 2017 15:49:31 GMT
Thanks for elaborating Jay. I totally agree that we have to be very careful
in how we use our complexity budget. Easier said than done when people
don't agree on what is complex and what is simple. :) For example, I think
batch APIs are a significant source of complexity as you have to do a bunch
of ceremony to group things before sending the request and error handling
becomes more complex due to partial failures (things like `Try` or other
mechanisms that serve a similar role are then needed).

Maybe a way forward is to write API usage examples to help validate that
the suggested API is indeed easy to use.

Ismael

On Wed, Feb 8, 2017 at 4:40 AM, Jay Kreps <jay@confluent.io> wrote:

> Totally agree on CompletableFuture. Also agree with some of the rough edges
> on the Consumer.
>
> I don't have much of a leg to stand on with the splitting vs not splitting
> thing, really hard to argue one or the other is better. I guess the one
> observation in watching us try to make good public apis over the years is I
> am kind of in favor of a particular kind of simple. In particular I think
> since the bar is sooo high in support and docs and the community of users
> so broad in the range of their capabilities, it makes it so there is a lot
> of value in dead simple interfaces that don't have a lot of conceptual
> weight, don't introduce a lot of new classes or concepts or general
> patterns that must be understood to use them correctly. So things like
> nesting, or the Try class, or async apis, or even just a complex set of
> classes representing arguments or return values kind of all stack
> conceptual burdens on the user to figure out correct usage. So like, for
> example, the Try class is very elegant and represents a whole generalized
> class of possibly completed actions, but the flip side is maybe I'm just a
> working guy who needs to list his kafka topics but doesn't know Rust, take
> pity on me! :-)
>
> Nit picking aside, super excited to see us progress on this.
>
> -Jay
>
> On Tue, Feb 7, 2017 at 3:46 PM Ismael Juma <ismael@juma.me.uk> wrote:
>
> > Hi Jay,
> >
> > Thanks for the feedback. Comments inline.
> >
> > On Tue, Feb 7, 2017 at 8:18 PM, Jay Kreps <jay@confluent.io> wrote:
> > >
> > >    - I think it would be good to not use "get" as the prefix for things
> > >    making remote calls. We've tried to avoid the java getter convention
> > >    entirely (see code style guide), but for remote calls in particular
> it
> > > kind
> > >    of blurs the line between field access and remote RPC in a way that
> > > leads
> > >    people to trouble. What about, e.g., fetchAllGroups() vs
> > getAllGroups().
> > >
> >
> > Agreed that we should avoid the `get` prefix for remote calls. There are
> a
> > few possible prefixes for the read operations: list, fetch, describe.
> >
> >
> > >    - I think futures and callbacks are a bit of a pain to use. I'd
> second
> > >    Becket's comment: let's ensure there a common use case motivating
> > these
> > >    that wouldn't be just as easily satisfied with batch operations
> (which
> > > we
> > >    seem to have at least for some things). In terms of flexibility
> > > Callbacks >
> > >    Futures > Batch Ops but I think in terms of usability it is the
> exact
> > >    opposite so let's make sure we have worked out how the API will be
> > used
> > >    before deciding. In particular I think java Futures are often an
> > >    uncomfortable half-way point since calling get() and blocking the
> > > thread is
> > >    often not what you want for chaining sequences of operations in a
> > truly
> > >    async way, so 99% of people just use the future as a way to batch
> > calls.
> > >
> >
> > We should definitely figure out how the APIs are going to be used before
> > deciding. I agree that callbacks are definitely painful and there's
> little
> > reason to expose them in a modern API unless it's meant to be very low
> > level. When it comes to Futures, I think it's important to distinguish
> what
> > is available in Java 7 and below versus what is available from Java 8
> > onwards. CompletableFuture makes it much easier to compose/chain
> operations
> > (in a similar vein to java.util.Stream, our own Streams API, etc.) and it
> > gives you the ability to register callbacks if you really want to
> (avoiding
> > the somewhat odd situation in the Producer where we return a Future _and_
> > allow you to pass a callback).
> >
> >
> > >    - Personally I don't think splitting the admin methods up actually
> > makes
> > >    things more usable. It just makes you have to dig through our
> > > hierarchy. I
> > >    think a flat class with a bunch of operations (like the consumer
> api)
> > is
> > >    probably the easiest for people to grok and find things on. I am
> kind
> > > of a
> > >    dumb PHP programmer at heart, though.
> > >
> >
> > I am not sure it's fair to compare the AdminClient with the Consumer. The
> > former has APIs for a bunch of unrelated APIs (topics, ACLs, configs,
> > consumer groups, delegation tokens, preferred leader election, partition
> > reassignment, etc.) where the latter is pretty specialised. For each of
> the
> > resources, you may have 3-4 operations, it will get confusing fast. Also,
> > do you really think an API that has one level of grouping will mean that
> > users have to "dig through our hierarchy"? Or are you concerned that once
> > we go in that direction, there is a danger of making the hierarchy more
> > complicated?
> >
> > Finally, I am not sure I would use the consumer as an example of
> something
> > that is easy to grok. :) The fact that methods behave pretty differently
> > (some are blocking while others only have an effect after poll) with no
> > indication from the type signature or naming convention makes it harder,
> > not easier, to understand.
> >
> > Ismael
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message