kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paolo Patierno <ppatie...@live.com>
Subject Re: Command tools : from Scala to Java, from Zookeeper utils to Admin Client API
Date Wed, 16 Aug 2017 15:38:03 GMT
Hi Ismael,


after your first review on my PR and the conversation here in the mailing list ... I have
a doubt ...

We are going to remove the --zookeeper option but adding the --broker-list but according to
this discussion we agree on your solution having the choice at shell script level.

Does the Java re-implementation of the TopicCommand tool need a KIP ?


Thanks,

Paolo


Paolo Patierno
Senior Software Engineer (IoT) @ Red Hat
Microsoft MVP on Windows Embedded & IoT
Microsoft Azure Advisor

Twitter : @ppatierno<http://twitter.com/ppatierno>
Linkedin : paolopatierno<http://it.linkedin.com/in/paolopatierno>
Blog : DevExperience<http://paolopatierno.wordpress.com/>


________________________________
From: ismaelj@gmail.com <ismaelj@gmail.com> on behalf of Ismael Juma <ismael@juma.me.uk>
Sent: Wednesday, August 2, 2017 8:51 AM
To: dev@kafka.apache.org
Subject: Re: Command tools : from Scala to Java, from Zookeeper utils to Admin Client API

Hi Arseniy,

At some point (before I joined), it was decided that clients would be
written in Java for a few reasons:

1. It's easier to maintain binary compatibility
2. Calling Java from Scala is easier than calling Scala from Java (without
extra effort)
3. A single artifact instead of one artifact per Scala version supported
4. There are a larger number of Java users than Scala users

Since the plan is for most tools to be a thin wrapper over the Java
clients, it seems sensible for such tools not to bring unnecessary
dependencies.

Ismael


On Wed, Aug 2, 2017 at 9:28 AM, Arseniy Tashoyan <tashoyan@gmail.com> wrote:

> Hi Paolo,
>
> I doubt that rewriting in Java makes value by itself. What is the reason of
> redoing the things that are already done? The switching to new API can be
> done without the switching to another language. Less new code - less new
> bugs.
> In some cases, Scala may be more convenient than Java. For example, one can
> find Scala collections more handy than Java collections. Just select a
> language that gives necessary tools.
>
> Arseniy
>
> 2017-08-01 17:30 GMT+03:00 Paolo Patierno <ppatierno@live.com>:
>
> > Hi Tom,
> >
> >
> > thanks for your reply. I think that you are right and what you have
> > proposed should be the way to go.
> >
> >
> > Until today I have been working on refactoring the TopicCommand tool in
> > Java using the AdminClient getting rid of the Zookeeper usage in only
> "one
> > step" and maybe it's wrong.
> >
> > I'd like to have some input from committers as well to be sure that the
> > way is good about how handling such use cases.
> >
> >
> > Thanks,
> >
> >
> > Paolo Patierno
> > Senior Software Engineer (IoT) @ Red Hat
> > Microsoft MVP on Windows Embedded & IoT
> > Microsoft Azure Advisor
> >
> > Twitter : @ppatierno<http://twitter.com/ppatierno>
> > Linkedin : paolopatierno<http://it.linkedin.com/in/paolopatierno>
> > Blog : DevExperience<http://paolopatierno.wordpress.com/>
> >
> >
> > ________________________________
> > From: Tom Bentley <t.j.bentley@gmail.com>
> > Sent: Friday, July 28, 2017 10:36 AM
> > To: dev@kafka.apache.org
> > Subject: Re: Command tools : from Scala to Java, from Zookeeper utils to
> > Admin Client API
> >
> > Hi Paolo,
> >
> > Replies in line...
> >
> > On 28 July 2017 at 11:14, Paolo Patierno <ppatierno@live.com> wrote:
> >
> > > Hi committers,
> > >
> > > in my understanding there is the common idea to move all tools from
> Scala
> > > to Java and then using the new Admin Client API instead of using the
> > > Zookeeper connection.
> > >
> > > Regarding this subject I started to work on refactoring the
> TopicCommand
> > > tool but with two steps at same time :
> > >
> > >
> > >   *   re-writing it in Java
> > >   *   removing --zookeeper option (so no Zookeeper connection) and
> adding
> > > --broker-list option (so using the Admin Client API)
> > >
> > > Of course, such option substitution is a breaking change for the tool
> > (and
> > > the users who are using it).
> > > In such a scenario, the two steps path should be needed : first
> > > deprecation, then removal (for the --zookeeper option)
> > >
> >
> > A change to tools (and their options) requires a KIP. There's no
> > fundamental reason why both couldn't be supported during a transition
> > period. So I doubt a KIP that didn't propose a transition period would
> get
> > passed.
> >
> >
> > It seems that in the "deprecation" phase we have two possible solutions :
> > >
> > >
> > >   1.  Adding Admin Client API to the Scala tools and so the
> --broker-list
> > > option and a warning on --zookeeper for deprecation
> > >   2.  Rewriting the tool in Java using the Admin Client API (so
> > > --broker-list) but at same time providing --zookeeper as well (so
> > Zookeeper
> > > connection)
> > >
> > > With the solution 2) we have the advantage to have the tool already in
> > > Java when the --zookeeper option will be removed in a next step. In any
> > > case we have to write the part related to use the Admin Client API so
> it
> > > make more sense to me option 2) porting the Zookeeper needed code from
> > > Scala to Java (then removing it in the next phase).
> > >
> > > Is my understanding right on how we have to handle deprecation and
> > removal
> > > for something that is a breaking change ?
> > > Or this case is something "special" and we can live with a new Java
> based
> > > tool without zookeeper but with Admin Client API at same time ?
> > >
> > > Of course having both Admin Client API and Zookeeper utils working at
> the
> > > same time in the tools means more complexity in the code but it's
> > something
> > > that could be factorized.
> > >
> >
> > I think the right thing to do would be:
> >
> > 1. deprecate the old option, adding support for the replacement option
> > (using the AdminClient). Keep the code in scala. All this is in one KIP.
> > 2. Remove the old option (needs a KIP)
> > 3. Rewrite the tool in Java.
> >
> > Parts 2 and 3 could be done at the same time. I don't believe part 3
> needs
> > a KIP if it were a drop-in replacement.
> >
> > The reason I think this is the right way to proceed is:
> >
> > * It gives users a transition period to learn about the new option, and
> > replace any tooling of their own.
> > * By keeping the tool in scala you get to release your new AdminClient
> API
> > and get to iron out all the creases while users still have the
> --zookeeper
> > option as a fallback.
> > * Then when you know the AdminClient API works in the field you have a
> > straight porting job to do, and you have less code to port because you
> > don't have to port the code to support --zookeeper.
> >
> > But I'm fairly new here, so maybe a committer will chime in an correct me
> > where I'm wrong!
> >
> > Cheers,
> >
> > Tom
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message