kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Manikumar <manikumar.re...@gmail.com>
Subject Re: [DISCUSS] KIP-278: Add version option to Kafka's commands
Date Wed, 02 May 2018 19:11:41 GMT
Hi Colin,

Thanks for explanation. It's definitely useful to have  --version flag.

kafka-broker-api-versions.sh gives the API versions, not Kafka release
version.
Is not easy to figure out release version from API versions. Currently
release version is available via metric/JMX.
If required, we can implement this in future.


Thanks,

On Wed, May 2, 2018 at 10:58 PM, Colin McCabe <cmccabe@apache.org> wrote:

> Hi Manikumar,
>
> We already have a tool for getting the Kafka broker API versions,
> "./bin/kafka-broker-api-versions.sh".  It was added as part of KIP-97.
>
> What Saski is proposing here is having a way of getting the version of
> locally installed Kafka software, which may be different from the server
> version.  Many pieces of software offer a --version flag, and it's always
> understood to refer to the local version of the software, not a version
> running somewhere else.  The user has no way to get this information now,
> other than perhaps trying to look at he names of jar files.
>
> cheers,
> Colin
>
> On Tue, May 1, 2018, at 08:20, Manikumar wrote:
> > I assume the intent of the KIP to find out the Kafka broker version.  In
> > this case, maybe we should expose
> > version using a Kafka request. This will help the remote scripts/tools to
> > query the Kafka version.
> > scripts (kafka-topics.sh, kafka-configs.sh, etc..)  may run from remote
> > machines  and may use
> > older Kafka versions. In this case, current proposal prints on the older
> > version.
> >
> > On Tue, May 1, 2018 at 7:47 PM, Colin McCabe <cmccabe@apache.org> wrote:
> >
> > > Thanks, Sasaki.
> > >
> > > Colin
> > >
> > > On Sat, Apr 28, 2018, at 00:55, Sasaki Toru wrote:
> > > > Hi Colin, Jason,
> > > >
> > > > Thank you for your beneficial comment.
> > > > I have updated my Pull Request to show git commit hash in version
> > > > information.> In my current Pull Request, we cat get the result such
> > > below:
> > > >
> > > >    $ bin/kafka-topics.sh --version
> > > >    (snip)
> > > >    2.0.0-SNAPSHOT (Commit:f3876cd9617faf7e)
> > > >
> > > >
> > > > I have also updated to accept double-dash for this option (--
> > > > version) only.>
> > > >
> > > > Many thanks,
> > > > Sasaki
> > > >
> > > > > From: Jason Gustafson <jason@confluent.io>
> > > > > Date: 2018-04-25 9:42 GMT+09:00
> > > > > Subject: Re: [DISCUSS] KIP-278: Add version option to Kafka's
> > > > > commands> > To: dev <dev@kafka.apache.org>
> > > > >
> > > > >
> > > > > +1 on adding the git commit id to the output. We often encounter
> > > > > environments which are testing off of trunk or have modifications
> on
> > > > > top of> > an existing release.
> > > > >
> > > > > -Jason
> > > > >
> > > > > On Tue, Apr 24, 2018 at 10:06 AM, Colin McCabe <cmccabe@apache.org
> >
> > > > > wrote:> >
> > > > >> On Tue, Apr 24, 2018, at 05:36, Sasaki Toru wrote:
> > > > >>> Hi Jason, Colin,
> > > > >>>
> > > > >>> Thank you for your comment, and I'm sorry for late reply.
> > > > >>>
> > > > >>>   > we refactored all of the tools so that we could use
a common
> > > > >>>   > set of> >>> base options,
> > > > >>>   > it might be a little annoying to have to continue
supporting
> > > > >>>   > both> >>> variations.
> > > > >>>
> > > > >>> I feel KIP-14 is good idea (I'm sorry, I didn't know about
> > > > >>> it), and> >>> I think it's no longer necessary
both single-dash
> and
> > > double-
> > > > >>> dash are> >>> supported when this KIP will be
accepted, as you
> said.
> > > > >>> I revise my Pull Request to support single-dash option only.
> > > > >>>
> > > > >>> Some my code in kafka-run-class.sh will become unnecessary
when
> > > > >>> KIP-14> >>> is accepted.
> > > > >>> But I will keep this as temporary, because I seem that it's
> useful>
> > > >>> before KIP-14 accepted.
> > > > >>>
> > > > >>>
> > > > >>>   > Can you give a little more detail about what would
be
> > > > >>>   > displayed when> >>> the version command
was used?
> > > > >>>
> > > > >>> As Ismael said, the version string is got from
> > > > >>> AppInfoParser#getVersion.> >>>
> > > > >>> In my Pull Request, we can get the result such as below::
> > > > >>>
> > > > >>>     $ bin/kafka-topics.sh --version
> > > > >>>     (snip)
> > > > >>>     Kafka 1.2.0-SNAPSHOT
> > > > >> Hi Sasaki,
> > > > >>
> > > > >> Thanks for the info.  Can you add this to the KIP?
> > > > >>
> > > > >> Also, I really think we should include the git hash.
> > > > >>
> > > > >> best,
> > > > >> Colin
> > > > >>
> > > > >>
> > > > >>>
> > > > >>> Many thanks,
> > > > >>> Sasaki
> > > > >>>
> > > > >>>
> > > > >>>> From: Ismael Juma <ismael@juma.me.uk>
> > > > >>>> Date: 2018-04-24 3:44 GMT+09:00
> > > > >>>> Subject: Re: [DISCUSS] KIP-278: Add version option to
Kafka's
> > > > >>>> commands> >>>> To: dev <dev@kafka.apache.org>
> > > > >>>>
> > > > >>>>
> > > > >>>> FYI, the injection via the build process that is mentioned
here
> > > > > already
> > > > >>>> happens. See AppInfoParser.
> > > > >>>>
> > > > >>>> Ismael
> > > > >>>>
> > > > >>>> On Mon, Apr 23, 2018 at 9:39 AM, Colin McCabe
> > > > >>>> <cmccabe@apache.org>> >> wrote:
> > > > >>>>> Hi Sasaki,
> > > > >>>>>
> > > > >>>>> Thanks for the KIP.  I think a version flag is a
good idea.
> > > > >>>>>
> > > > >>>>> Can you give a little more detail about what would
be displayed
> > > > >>>>> when> >> the
> > > > >>>>> version command was used?
> > > > >>>>>
> > > > >>>>> We clearly want the version number, but we probably
also want
> to
> > > > >>>>> know> >> if
> > > > >>>>> this is an official release, or a random SNAPSHOT
from a
> branch.
> > > > >>>>> If> >> this
> > > > >>>>> is a release candidate, we probably want the RC number
as well,
> > > > >>>>> like> >>>>> "1.1-rc3"  We also
want a git hash.  This can be
> > > injected by the> > build
> > > > >>>>> process.  In the case of an official release, where
the source
> > > > >>>>> code> >> is not
> > > > >>>>> under git, we can pull it from a file.
> > > > >>>>>
> > > > >>>>> For example, hadoop's version output looks like this:
> > > > >>>>>
> > > > >>>>>    > cmccabe@aurora:~/Downloads/hadoop-2.8.3>
./bin/hadoop
> > > > >>>>>    > version> >>>>>    >
Hadoop 2.8.3
> > > > >>>>>    > Subversion
> > > > >>>>>    > https://git-wip-us.apache.org/repos/asf/hadoop.git
-r>
> >>>>>
> > > b3fe56402d908019d99af1f1f4fc65cb1d1436a2
> > > > >>>>>    > Compiled by jdu on 2017-12-05T03:43Z
> > > > >>>>>    > Compiled with protoc 2.5.0
> > > > >>>>>    > From source with checksum 9ff4856d824e983fa510d3f843e3f1
> 9d>
> > > >>>>>    > This command was run using /home/cmccabe/Downloads/
> > > > >>>>> hadoop-2.8.3/share/hadoop/common/hadoop-common-2.8.3.jar
> > > > >>>>>
> > > > >>>>> (The "subversion" line here is a little weird --
it now refers
> > > > >>>>> to> >> git, not
> > > > >>>>> svn)
> > > > >>>>>
> > > > >>>>> On Wed, Apr 11, 2018, at 13:58, Jason Gustafson wrote:
> > > > >>>>>> Hey Sasaki,
> > > > >>>>>>
> > > > >>>>>> Yeah, I don't feel too strongly about only supporting
--
> > > > >>>>>> version. I> >> agree
> > > > >>>>> it
> > > > >>>>>> may help discoverability given the current approach.
On the
> > > > >>>>>> other> >> hand,
> > > > >>>>> if
> > > > >>>>>> we refactored all of the tools so that we could
use a common
> > > > >>>>>> set of> >> base
> > > > >>>>>> options, it might be a little annoying to have
to continue
> > > > > supporting
> > > > >>>>> both
> > > > >>>>>> variations. For example, tool standardization
was proposed in
> > > > >>>>>> KIP-14> >> and
> > > > >>>>>> I'm still holding out hope that someone will
have time to pick
> > > > >>>>>> this> >> work
> > > > >>>>>> back up. It's always easier to add an option
than remove one,
> > > > >>>>>> so I'm> >>>>>> slightly
inclined to have only --version for
> now.
> > > What do you
> > > > >>>>>> think?> >>>>> The double dash
version is more consistent with
> how
> > > our other
> > > > >>>>> flags> >> work.
> > > > >>>>> In general, I feel that if --version is supported,
--help
> should
> > > > >>>>> say> >> so.
> > > > >>>>> best,
> > > > >>>>> Colin
> > > > >>>>>
> > > > >>>>>
> > > > >>>>>> Thanks,
> > > > >>>>>> Jason
> > > > >>>>>>
> > > > >>>>>> On Tue, Apr 10, 2018 at 12:00 AM, Sasaki Toru
<
> > > > >> sasakitoa@oss.nttdata.com
> > > > >>>>>> wrote:
> > > > >>>>>>
> > > > >>>>>>> Hi Jason
> > > > >>>>>>>
> > > > >>>>>>> Thank you for helpful comments. I updated
wiki based on your
> > > > > advice.
> > > > >>>>>>> I thought this option was relatively common
and making
> > > > >>>>>>> maintenance> >>>> easy
> > > > >>>>>>> was also important.
> > > > >>>>>>> However, as you said, it is not good that
version option
> won't
> > > > >>>>>>> be> >>>>> shown up
> > > > >>>>>>> in help description.
> > > > >>>>>>>
> > > > >>>>>>> I thought accepting both single-dash and
double-dash will
> help
> > > > >>>>>>> to> >> find
> > > > >>>>>>> this option.
> > > > >>>>>>> In my approach this option won't be showed,
but most of
> > > > >>>>>>> software> >> which
> > > > >>>>> has
> > > > >>>>>>> this option accepts either single-dash or
double-dash.
> > > > >>>>>>> I guess it doesn't need to support both if
we take another
> > > > >>>>>>> way.> >>>>>>>
> > > > >>>>>>>
> > > > >>>>>>> Thanks
> > > > >>>>>>>
> > > > >>>>>>> @Ted Yeah, you're right. Sorry about the
confusion.
> > > > >>>>>>>> Since we're here, I think this KIP is
a nice improvement.
> > > > >>>>>>>> It's> >>>>> definitely
> > > > >>>>>>>> nice to have an easy way to check the
version. That said, do
> > > > >>>>>>>> we> >>>> really
> > > > >>>>>>>> need
> > > > >>>>>>>> to support both `-version` and `--version`?
The latter is
> > > > >> consistent
> > > > >>>>> with
> > > > >>>>>>>> our current tools.
> > > > >>>>>>>>
> > > > >>>>>>>> Also, I think the approach we've taken
is basically to build
> > > > >>>>>>>> the> >>>>> --version
> > > > >>>>>>>> functionality into the bash script. This
is nice because it
> > > > >>>>>>>> saves> > a
> > > > >>>>> lot of
> > > > >>>>>>>> work to update the commands individually
and we don't need
> to
> > > > >>>>>>>> do> >>>>> anything
> > > > >>>>>>>> when we add new tools. The downside is
that `--version`
> won't
> > > > >>>>>>>> show> >> up
> > > > >>>>> as
> > > > >>>>>>>> an
> > > > >>>>>>>> option in any of the --help output. Not
sure if that is too
> > > > >>>>>>>> big of> >> a
> > > > >>>>>>>> problem, but maybe worth mentioning this
in the rejected
> > > > >> alternatives
> > > > >>>>>>>> section.
> > > > >>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>> -Jason
> > > > >>>>>>>>
> > > > >>>>>>>> On Wed, Apr 4, 2018 at 9:42 AM, Ted Yu
<yuzhihong@gmail.com
> >>
> > > >> wrote:
> > > > >>>>>>>> Jason:
> > > > >>>>>>>>> Maybe your reply was intended for
another KIP ?
> > > > >>>>>>>>>
> > > > >>>>>>>>> KIP-278 is about adding version option,
not timeout.
> > > > >>>>>>>>>
> > > > >>>>>>>>> Cheers
> > > > >>>>>>>>>
> > > > >>>>>>>>> On Wed, Apr 4, 2018 at 9:36 AM, Jason
Gustafson <
> > > > >> jason@confluent.io>
> > > > >>>>>>>>> wrote:
> > > > >>>>>>>>>
> > > > >>>>>>>>> Hi Sasaki,
> > > > >>>>>>>>>> Thanks for the KIP. I think the
timeout controls the
> > > > >>>>>>>>>> maximum> >>>>
allowed
> > > > >>>>>>>>> time
> > > > >>>>>>>>> that the consumer will block for
the next record. Maybe
> the>
> > > >> meaning
> > > > >>>>>>>>> would
> > > > >>>>>>>>> be clearer with the more concise
name `--timeout`? That
> also
> > > > >>>>>>>>> fits> >>>>> with
> > > > >>>>>>>>> the
> > > > >>>>>>>>>
> > > > >>>>>>>>>> old consumer which overrides
the `consumer.timeout.ms`
> > > > >>>>>>>>>> property.> >>>>>>>>>>
> > > > >>>>>>>>>> By the way, it seems like the
default value was
> > > > >>>>>>>>>> intentionally> > set
> > > > >>>>> low
> > > > >>>>>>>>> for
> > > > >>>>>>>>> both the old and new consumers, but
I'm not sure of the
> > > > >>>>>>>>> reason.> > We
> > > > >>>>> could
> > > > >>>>>>>>>> leave the default as it is if
we want to be safe, but
> > > > >>>>>>>>>> increasing> >> it
> > > > >>>>>>>>> seems
> > > > >>>>>>>>> ok to me. Perhaps we could start
a little lower, though,
> say
> > > > >>>>>>>>> 10> >>>>> seconds?
> > > > >>>>>>>>> In
> > > > >>>>>>>>>
> > > > >>>>>>>>>> any case, we should make it clear
to the user that the
> > > > >>>>>>>>>> timeout> >> was
> > > > >>>>>>>>> reached.
> > > > >>>>>>>>>
> > > > >>>>>>>>>> It's surprising to see only the
incomplete reported
> results>
> > > >>>>> following a
> > > > >>>>>>>>>> timeout.
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> Thanks,
> > > > >>>>>>>>>> Jason
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> On Wed, Apr 4, 2018 at 4:37 AM,
Sasaki Toru <
> > > > >>>>> sasakitoa@oss.nttdata.com>
> > > > >>>>>>>>>> wrote:
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> Hello everyone,
> > > > >>>>>>>>>>> I would like to start a discussion
for KIP 278. Cloud
> you> >
> > > please
> > > > >>>>> give
> > > > >>>>>>>>>>> comments and advice ?
> > > > >>>>>>>>>>> <https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > > >>>>>>>>>>> 278+-> >>>>>>>>>>>
+Add+version+option+to+Kafka%2
> > > 7s+commands>
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> JIRA ticket and Pull Request
are bellow:
> > > > >>>>>>>>>>> <https://issues.apache.org/jira/browse/KAFKA-2061>
> > > > >>>>>>>>>>> <https://github.com/apache/kafka/pull/639>
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> Many thanks,
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> Sasaki
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> --
> > > > >>>>>>>>>>> Sasaki Toru(sasakitoa@oss.nttdata.com)
NTT DATA
> > > > >>>>>>>>>>> CORPORATION> >>>>>>>>>>>
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>>
> > > > >>>>>>> --
> > > > >>>>>>> Sasaki Toru(sasakitoa@oss.nttdata.com) NTT
DATA CORPORATION
> > > > >>>>>>>
> > > > >>>>>>>
> > > > >>> --
> > > > >>> Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA CORPORATION
> > > > >>>
> > > >
> > > > --
> > > > Sasaki Toru(sasakitoa@oss.nttdata.com) NTT DATA CORPORATION
> > > >
> > >
> > >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message