Mailing-List: contact dev-help@kafka.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@kafka.apache.org
From: Paolo Patierno <ppatierno@live.com>
To: "dev@kafka.apache.org" <dev@kafka.apache.org>
Subject: Re: Command tools : from Scala to Java, from Zookeeper utils to Admin
 Client API
Thread-Topic: Command tools : from Scala to Java, from Zookeeper utils to
 Admin Client API
Thread-Index: AQHTB4hSOBa/stpXXU6eH/aAG+Q5AqJpC7aAgAaJ+VSAAS4FAIAAA3OL
Date: Wed, 2 Aug 2017 08:49:04 +0000
Message-ID: <DB6PR0202MB290128F4F3B47A073BC3E5BAC9B00@DB6PR0202MB2901.eurprd02.prod.outlook.com>
References: <DB6PR0202MB290194298A6A0FD0D02F3409C9BF0@DB6PR0202MB2901.eurprd02.prod.outlook.com>
 <CAMd5YswhAAfOYa1SZpVAd2XCjqkx9P4J1fDW=-3Fkw-gDF9akA@mail.gmail.com>
 <HE1PR0202MB290683A5E5A026148454FE9BC9B30@HE1PR0202MB2906.eurprd02.prod.outlook.com>,<CALWig4LDaoT5EBYxM4tBuRrASVnQEbzbuZ7XqMn=7qX1cX=KtQ@mail.gmail.com>
In-Reply-To: <CALWig4LDaoT5EBYxM4tBuRrASVnQEbzbuZ7XqMn=7qX1cX=KtQ@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-messagesentrepresentingtype: 1
x-tmn: [5xcwtRJnnSFtHxYV+Y1vYKdE4WhjRbK1]
x-ms-publictraffictype: Email
x-microsoft-exchange-diagnostics: 1;AM5EUR03HT196;7:N7fywUoaEjmMfMPi1SFbXztlblu5yLn2K9hCDDORKXmUBvH4+ZOQ3CKX05Z9ME3sPJvd5WWa9cB3UtmF+wSwr1+g/B6eFYOHOfDUWjPX8XRXknHBih+t3/HmCMjLqcc7NLfrqOXTXuioZzKtQNGkQeYbnWeffuqPWrX3DaCSBU7kx4AJ9xAZEwaHF+WHDSUe0BG6pE30n0Aj3iEjFUo9z9c5jzDC4UlQhy87LiQ/Zj7fnBdPsFykMF+rtH1vOXuVMkbBX0p+on4FGExnd34o9Brkyrj8MQ2fkybk+KrBZSXLVLj4UnEeiE63jK9JWlRTpAq8o0SN6S5LkY9qV9ArC1YIhlnXbwwK7tlPY7LgTLGTYrfrYJuNlNOXDq+/ow1OcgbFNTnDee3darKnNbl8aRfeSgZphw26uon4rvZLhqf958ncED6j63nKNCjATUGILWHL8AOWsUTzTalURZmJYHzaDPXoxN52x3KW93Bq5Q2rl58dwx8s2BlAbPFqmxOBxCjjkOcXx9DbpqoOCiaAy5bUozEw7fu6xQ6eVTpA3TlfHDUXaHSCm3X3wZtBtCjjZUaVb/NMYpZy5YHZhr5KPBwPFGZvkLGrru2bXOp7FF5hTUA91/7P6EPp+O4qFkH3bvIVag24kPmonTEaRlaqs29AEcbYP8CvGhuKsnxPvIvO2mJJFeZkjlj/eYe4ArhKDCjSV+P+ILbPtCch3oBMuKNWaaqyfusmvAbxecly0xYwXqwynvF4zpVdoQZUonpcGXC/AW/y1Nlw96NuTVbHNw==
x-incomingheadercount: 45
x-eopattributedmessage: 0
x-forefront-antispam-report: EFV:NLI;SFV:NSPM;SFS:(7070007)(98901004);DIR:OUT;SFP:1901;SCL:1;SRVR:AM5EUR03HT196;H:DB6PR0202MB2901.eurprd02.prod.outlook.com;FPR:;SPF:None;LANG:en;
x-ms-office365-filtering-correlation-id: 0d8a09b8-0e83-46e4-2957-08d4d983548a
x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:(300000500095)(300135000095)(300000501095)(300135300095)(22001)(300000502095)(300135100095)(300000503095)(300135400095)(201702061074)(5061506573)(5061507331)(1603103135)(2017031320274)(2017031324274)(2017031323274)(2017031322377)(1603101448)(1601125374)(1701031045)(300000504095)(300135200095)(300000505095)(300135600095)(300000506095)(300135500095);SRVR:AM5EUR03HT196;
x-ms-traffictypediagnostic: AM5EUR03HT196:
x-exchange-antispam-report-test: UriScan:(134217032509453)(148322886591682)(31418570063057)(47647156867600)(37052965297144)(116415991822766)(788757137089)(81160342030619)(105182351903845)(5213294742642);
x-exchange-antispam-report-cfa-test: BCL:0;PCL:0;RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(444000031);SRVR:AM5EUR03HT196;BCL:0;PCL:0;RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095);SRVR:AM5EUR03HT196;
x-forefront-prvs: 0387D64A71
spamdiagnosticoutput: 1:99
spamdiagnosticmetadata: NSPM
Content-Type: multipart/alternative;
	boundary="_000_DB6PR0202MB290128F4F3B47A073BC3E5BAC9B00DB6PR0202MB2901_"
MIME-Version: 1.0
X-OriginatorOrg: live.com
X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Aug 2017 08:49:04.7741
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Internet
X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM5EUR03HT196
archived-at: Wed, 02 Aug 2017 08:49:26 -0000

--_000_DB6PR0202MB290128F4F3B47A073BC3E5BAC9B00DB6PR0202MB2901_
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

Hi Arseniy,


I opened this JIRA about one month ago (https://issues.apache.org/jira/brow=
se/KAFKA-5536) asking why we have some tools in Java and other in Scala.

As Ismael pointed out the end state should be having tools written in Java =
(as already happened to clients from Scala to Java) living in the external =
"tools" module (where there are already some tools) so outside of the "core=
" module in Kafka. Another key point is to use the Admin Client API provide=
d by the "clients" module in all the tools where we don't need Zookeeper co=
nnection anymore.

My guess is that, even if Kafka is developed in Scala, the committers have =
the idea to have all the external modules written in Java (so clients and t=
ools for now).

As you know it's the same as for the other two bigger projects around Kafka=
 (core) like Kafka Connect and Kafka Streams ... entirely developed in Java=
.


Thanks,


Paolo Patierno
Senior Software Engineer (IoT) @ Red Hat
Microsoft MVP on Windows Embedded & IoT
Microsoft Azure Advisor

Twitter : @ppatierno<http://twitter.com/ppatierno>
Linkedin : paolopatierno<http://it.linkedin.com/in/paolopatierno>
Blog : DevExperience<http://paolopatierno.wordpress.com/>


________________________________
From: Arseniy Tashoyan <tashoyan@gmail.com>
Sent: Wednesday, August 2, 2017 8:28 AM
To: dev@kafka.apache.org
Subject: Re: Command tools : from Scala to Java, from Zookeeper utils to Ad=
min Client API

Hi Paolo,

I doubt that rewriting in Java makes value by itself. What is the reason of
redoing the things that are already done? The switching to new API can be
done without the switching to another language. Less new code - less new
bugs.
In some cases, Scala may be more convenient than Java. For example, one can
find Scala collections more handy than Java collections. Just select a
language that gives necessary tools.

Arseniy

2017-08-01 17:30 GMT+03:00 Paolo Patierno <ppatierno@live.com>:

> Hi Tom,
>
>
> thanks for your reply. I think that you are right and what you have
> proposed should be the way to go.
>
>
> Until today I have been working on refactoring the TopicCommand tool in
> Java using the AdminClient getting rid of the Zookeeper usage in only "on=
e
> step" and maybe it's wrong.
>
> I'd like to have some input from committers as well to be sure that the
> way is good about how handling such use cases.
>
>
> Thanks,
>
>
> Paolo Patierno
> Senior Software Engineer (IoT) @ Red Hat
> Microsoft MVP on Windows Embedded & IoT
> Microsoft Azure Advisor
>
> Twitter : @ppatierno<http://twitter.com/ppatierno>
> Linkedin : paolopatierno<http://it.linkedin.com/in/paolopatierno>
> Blog : DevExperience<http://paolopatierno.wordpress.com/>
>
>
> ________________________________
> From: Tom Bentley <t.j.bentley@gmail.com>
> Sent: Friday, July 28, 2017 10:36 AM
> To: dev@kafka.apache.org
> Subject: Re: Command tools : from Scala to Java, from Zookeeper utils to
> Admin Client API
>
> Hi Paolo,
>
> Replies in line...
>
> On 28 July 2017 at 11:14, Paolo Patierno <ppatierno@live.com> wrote:
>
> > Hi committers,
> >
> > in my understanding there is the common idea to move all tools from Sca=
la
> > to Java and then using the new Admin Client API instead of using the
> > Zookeeper connection.
> >
> > Regarding this subject I started to work on refactoring the TopicComman=
d
> > tool but with two steps at same time :
> >
> >
> >   *   re-writing it in Java
> >   *   removing --zookeeper option (so no Zookeeper connection) and addi=
ng
> > --broker-list option (so using the Admin Client API)
> >
> > Of course, such option substitution is a breaking change for the tool
> (and
> > the users who are using it).
> > In such a scenario, the two steps path should be needed : first
> > deprecation, then removal (for the --zookeeper option)
> >
>
> A change to tools (and their options) requires a KIP. There's no
> fundamental reason why both couldn't be supported during a transition
> period. So I doubt a KIP that didn't propose a transition period would ge=
t
> passed.
>
>
> It seems that in the "deprecation" phase we have two possible solutions :
> >
> >
> >   1.  Adding Admin Client API to the Scala tools and so the --broker-li=
st
> > option and a warning on --zookeeper for deprecation
> >   2.  Rewriting the tool in Java using the Admin Client API (so
> > --broker-list) but at same time providing --zookeeper as well (so
> Zookeeper
> > connection)
> >
> > With the solution 2) we have the advantage to have the tool already in
> > Java when the --zookeeper option will be removed in a next step. In any
> > case we have to write the part related to use the Admin Client API so i=
t
> > make more sense to me option 2) porting the Zookeeper needed code from
> > Scala to Java (then removing it in the next phase).
> >
> > Is my understanding right on how we have to handle deprecation and
> removal
> > for something that is a breaking change ?
> > Or this case is something "special" and we can live with a new Java bas=
ed
> > tool without zookeeper but with Admin Client API at same time ?
> >
> > Of course having both Admin Client API and Zookeeper utils working at t=
he
> > same time in the tools means more complexity in the code but it's
> something
> > that could be factorized.
> >
>
> I think the right thing to do would be:
>
> 1. deprecate the old option, adding support for the replacement option
> (using the AdminClient). Keep the code in scala. All this is in one KIP.
> 2. Remove the old option (needs a KIP)
> 3. Rewrite the tool in Java.
>
> Parts 2 and 3 could be done at the same time. I don't believe part 3 need=
s
> a KIP if it were a drop-in replacement.
>
> The reason I think this is the right way to proceed is:
>
> * It gives users a transition period to learn about the new option, and
> replace any tooling of their own.
> * By keeping the tool in scala you get to release your new AdminClient AP=
I
> and get to iron out all the creases while users still have the --zookeepe=
r
> option as a fallback.
> * Then when you know the AdminClient API works in the field you have a
> straight porting job to do, and you have less code to port because you
> don't have to port the code to support --zookeeper.
>
> But I'm fairly new here, so maybe a committer will chime in an correct me
> where I'm wrong!
>
> Cheers,
>
> Tom
>

--_000_DB6PR0202MB290128F4F3B47A073BC3E5BAC9B00DB6PR0202MB2901_--