kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vahid S Hashemian" <vahidhashem...@us.ibm.com>
Subject Re: [DISCUSS] KIP-289: Improve the default group id behavior in KafkaConsumer
Date Thu, 02 Aug 2018 18:55:38 GMT
Hi Jason, Colin,

Thanks for your feedback.

If we are going to keep supporting the empty group id like before then it 
makes sense to do what you suggested.

Though, IMHO if using the empty group id for committing offsets is a bad 
practice we should stop supporting it at some point; and it would be 
better to do it sooner before potentially more users adopt that.
I understand that changing the default would reduce the chance of someone 
using it, but it doesn't stop them. Note that we did not receive any 
objections to dropping the support even right-away (a good sign).
We could keep supporting the old API version in case someone must use this 
group id like before. But at least we won't expose new users to this bad 
practice.

I, too, do now have a strong opinion about this. So if the current KIP 
should not move forward as is, I'm okay with just changing the default and 
deprecating the use on the client, as you suggested.

Thanks.
--Vahid



From:   Colin McCabe <cmccabe@apache.org>
To:     dev@kafka.apache.org
Date:   08/02/2018 11:23 AM
Subject:        Re: [DISCUSS] KIP-289: Improve the default group id 
behavior in KafkaConsumer



Thanks, Jason.  I don't have a very strong opinion on this.  But like you 
said, if we skip bumping the RPC versions, this would be a smaller change, 
which might be good.

best,
Colin


On Wed, Aug 1, 2018, at 17:43, Jason Gustafson wrote:
> Hey Vahid,
> 
> I talked with Colin offline. I think specifically he felt the version 
bump
> on the broker was overkill since the broker still has to support the 
empty
> group id for older versions. I had thought that eventually we would be 
able
> to remove those old versions, but it's true that this may not happen 
until
> indefinitely far in the future. I think the main improvement here is
> changing the default group.id to null instead of "". I could go either 
way
> on whether bumping the protocol is useful. I do think it is helpful 
though
> to signal clearly that it its use is deprecated and discouraged, 
especially
> in light of the ACL problem. I guess we could just deprecate the use on 
the
> client. What do you think?
> 
> Thanks,
> Jason
> 
> On Wed, Aug 1, 2018 at 3:19 PM, Vahid S Hashemian 
<vahidhashemian@us.ibm.com
> > wrote:
> 
> > Thanks Jason for responding to Colin's concerns.
> >
> > If there are no other comment / feedback / objection I'll start a vote
> > soon.
> >
> > Thanks.
> > --Vahid
> >
> >
> >
> > From:   Jason Gustafson <jason@confluent.io>
> > To:     dev <dev@kafka.apache.org>
> > Date:   07/27/2018 10:38 AM
> > Subject:        Re: [DISCUSS] KIP-289: Improve the default group id
> > behavior in KafkaConsumer
> >
> >
> >
> > Hey Colin,
> >
> > The problem is both that the empty group id is the default value and 
that
> > it is actually accepted by the broker for offset commits. Combine that
> > with
> > the fact that auto commit is enabled by default and you users get
> > surprising behavior. If you look at a random Kafka cluster, you'll
> > probably
> > find a bunch of inadvertent offset commits for the empty group id. I 
was
> > hoping we could distinguish between users who are using the empty 
group id
> > as an accident of the default configuration and those who use it
> > intentionally. By default, there will be no group id and the consumer 
will
> > not commit offsets. If a user has actually intentionally used the 
empty
> > group id, however, it will continue to work. I actually think there 
are
> > probably very few people doing this (maybe even no one), but I thought 
we
> > might err on the side of compatibility.
> >
> > The big incompatible change here is having brokers reject using
> > assign(...)
> > > with empty / null group.id.
> >
> >
> > This is not correct. In the proposal, the broker will only reject the
> > empty
> > group id for the new version of OffsetCommit. Older clients, which 
cannot
> > be changed, will continue to work because the old versions of the
> > OffsetCommit API still accept the empty group id. The null group id is
> > different from the empty group id: it is not allowed in any version of 
the
> > API. It is basically a way to indicate that the consumer has no 
dependence
> > on the coordinator at all, which we actually have a surprising number 
of
> > use cases for. Furthermore, if a user has an actual need for the empty
> > group id, it will still be allowed. We are just deprecating it.
> >
> > -Jason
> >
> > On Fri, Jul 27, 2018 at 9:56 AM, Colin McCabe <cmccabe@apache.org> 
wrote:
> >
> > > Sorry if this is a silly question, but what's the rationale for
> > switching
> > > to using null for the default group id, rather than the empty 
string?
> > > Continuing to use the empty string seems like less churn.  And after
> > all,
> > > we're not using the empty string group name for anything else.
> > >
> > > The big incompatible change here is having brokers reject using
> > > assign(...) with empty / null group.id.  If I understand correctly, 
the
> > > KIP proposes that this change be made on the brokers on the next
> > > incompatible Kafka release.  But that has nothing to do with client
> > > versions.  Why not just have a broker config which controls this? 
Maybe
> > "
> > > allow.assign.empty.group.id", or something like that.  At first, the
> > > default will be true, and then eventually we can flip it over to 
false.
> > >
> > > It seems like the main rationale for tying this behavior to the 
Kafka
> > > client version is to force people to stop using the empty group id 
so
> > that
> > > they can upgrade their clients.  But it's also possible that people 
will
> > > stop upgrading their Kafka clients instead.  That would be pretty
> > negative
> > > since  they'd miss out on any efficiency and feature improvements in 
the
> > > new clients and eventually have to do more protocol downgrading, 
etc.
> > >
> > > best,
> > > Colin
> > >
> > >
> > > On Thu, Jul 26, 2018, at 11:50, Vahid S Hashemian wrote:
> > > > Hi Jason,
> > > >
> > > > That makes sense.
> > > > I have updated the KIP based on the recent feedback.
> > > >
> > > > Thanks!
> > > > --Vahid
> > > >
> > > >
> > > >
> > > >
> > > > From:   Jason Gustafson <jason@confluent.io>
> > > > To:     dev <dev@kafka.apache.org>
> > > > Date:   07/25/2018 02:23 PM
> > > > Subject:        Re: [DISCUSS] KIP-289: Improve the default group 
id
> > > > behavior in KafkaConsumer
> > > >
> > > >
> > > >
> > > > Hi Vahid,
> > > >
> > > > I was thinking we'd only use the old API version if we had to. 
That
> > is,
> > > > only if the user has explicitly configured "" as the group.id.
> > > Otherwise,
> > > > we'd just use the new one. Another option is to just drop support 
in
> > the
> > > > client for the empty group id, but usually we allow a deprecation
> > period
> > > > for changes like this.
> > > >
> > > > -Jason
> > > >
> > > > On Wed, Jul 25, 2018 at 12:49 PM, Vahid S Hashemian <
> > > > vahidhashemian@us.ibm.com> wrote:
> > > >
> > > > > Hi Jason,
> > > > >
> > > > > Thanks for additional clarification.
> > > > >
> > > > > So the next version of the OffsetCommit API will return an
> > > > > INVALID_GROUP_ID error for empty group ids; but on the client 
side
> > we
> > > > call
> > > > > the older version of the client until the next major release.
> > > > > The table below should summarize this.
> > > > >
> > > > > +-----------------------------------------------------+
> > > > >                   |                 Client (group.id="") |
> > > > > +-----------------------------------------------------+
> > > > >                   | pre-2.1 |           2.1          |       3.0

|
> > > > >
> > > > +-----+-----------+---------+------------------------+------
> > > ------------+
> > > > > |     | V5 (cur.) | works   | works                  | works |
> > > > > + API
> > > > 
+-----------+---------+------------------------+------------------+
> > > > > |     | V6        | N/A     | N/A (calls V5/warning) |
> > > INVALID_GROUP_ID
> > > > |
> > > > >
> > > > +-----+-----------+---------+------------------------+------
> > > ------------+
> > > > >
> > > > > Assumptions:
> > > > > * 2.1: The target release version for this KIP
> > > > > * 3.0: The next major release
> > > > >
> > > > > Please advise if you see an issue; otherwise, I'll update the 
KIP
> > > > > accordingly.
> > > > >
> > > > > Thanks!
> > > > > --Vahid
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > From:   Jason Gustafson <jason@confluent.io>
> > > > > To:     dev <dev@kafka.apache.org>
> > > > > Date:   07/25/2018 12:08 AM
> > > > > Subject:        ***UNCHECKED*** Re: [DISCUSS] KIP-289: Improve 
the
> > > > default
> > > > > group id        behavior in KafkaConsumer
> > > > >
> > > > >
> > > > >
> > > > > Hey Vahid,
> > > > >
> > > > > Sorry for the confusion. I think we all agree that going 
forward, we
> > > > > shouldn't support the empty group id, so the question is just 
around
> > > > > compatibility. I think we have to bump the OffsetCommit API 
version
> > so
> > > > > that
> > > > > old clients which are unknowingly depending on the default empty
> > group
> > > > id
> > > > > will continue to work with new brokers. For new versions of the
> > > client,
> > > > we
> > > > > can either drop support for the empty group id immediately or we

can
> > > > give
> > > > > users a grace period. I was thinking we would do the latter. We 
can
> > > > change
> > > > > the default group.id, but in the case that a user has explicitly
> > > > > configured
> > > > > the empty group, then we can just use an old version of the
> > > OffsetCommit
> > > > > API which still supports it. In a future release, we can drop 
this
> > > > support
> > > > > and only use the latest OffsetCommit version. Does that make 
sense?
> > > > >
> > > > > Thanks,
> > > > > Jason
> > > > >
> > > > >
> > > > > On Tue, Jul 24, 2018 at 12:36 PM, Vahid S Hashemian <
> > > > > vahidhashemian@us.ibm.com> wrote:
> > > > >
> > > > > > Hi Jason,
> > > > > >
> > > > > > Thanks for clarifying.
> > > > > >
> > > > > > So if we are going to continue supporting the empty group id

as
> > > before
> > > > > > (with only an addition of a deprecation warning), and disable
> > > > > > enable.auto.commit for the new default (null) group id on the
> > client
> > > > > side,
> > > > > > do we really need to bump up the OffsetCommit version?
> > > > > >
> > > > > > You mentioned "If an explicit empty string is configured for

the
> > > group
> > > > > id,
> > > > > > then maybe we keep the current behavior for compatibility" 
which
> > > makes
> > > > > > sense to me, but I find it in conflict with your earlier
> > suggestion
> > > > "we
> > > > > > just need to bump the OffsetCommit request API and only accept

the
> > > > > offset
> > > > > > commit for older versions.". Maybe I'm missing something?
> > > > > >
> > > > > > Thanks!
> > > > > > --Vahid
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > From:   Jason Gustafson <jason@confluent.io>
> > > > > > To:     dev <dev@kafka.apache.org>
> > > > > > Date:   07/23/2018 10:52 PM
> > > > > > Subject:        Re: [DISCUSS] KIP-289: Improve the default 
group
> > id
> > > > > > behavior in KafkaConsumer
> > > > > >
> > > > > >
> > > > > >
> > > > > > Hey Vahid,
> > > > > >
> > > > > > Thanks for the updates. Just to clarify, I was suggesting that

we
> > > > > disable
> > > > > > enable.auto.commit only if no explicit group.id is configured.

If
> > an
> > > > > > explicit empty string is configured for the group id, then 
maybe
> > we
> > > > keep
> > > > > > the current behavior for compatibility. We can log a warning
> > > > mentioning
> > > > > > the
> > > > > > deprecation and we can use the old version of the OffsetCommit

API
> > > > that
> > > > > > allows the empty group id. In a later release, we can drop 
this
> > > > support
> > > > > in
> > > > > > the client. Does that seem reasonable?
> > > > > >
> > > > > > By the way, instead of using the new ILLEGAL_OFFSET_COMMIT 
error
> > > code,
> > > > > > couldn't we use INVALID_GROUP_ID?
> > > > > >
> > > > > > Thanks,
> > > > > > Jason
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Mon, Jul 23, 2018 at 5:14 PM, Stanislav Kozlovski
> > > > > > <stanislav@confluent.io
> > > > > > > wrote:
> > > > > >
> > > > > > > Hey Vahid,
> > > > > > >
> > > > > > > No I don't see an issue with it. I believe it to be the
best
> > > > approach.
> > > > > > >
> > > > > > > Best,
> > > > > > > Stanisav
> > > > > > >
> > > > > > > On Mon, Jul 23, 2018 at 12:41 PM Vahid S Hashemian <
> > > > > > > vahidhashemian@us.ibm.com> wrote:
> > > > > > >
> > > > > > > > Hi Stanislav,
> > > > > > > >
> > > > > > > > Thanks for the feedback.
> > > > > > > > Do you see an issue with using `null` as the default
group 
id
> > (as
> > > > > > > > addressed by Jason in his response)?
> > > > > > > > This default group id would not support offset commits
and
> > > > consumers
> > > > > > > would
> > > > > > > > use `auto.offset.reset` config when there is no current
> > offset.
> > > > > > > >
> > > > > > > > Thanks.
> > > > > > > > --Vahid
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > From:   Stanislav Kozlovski <stanislav@confluent.io>
> > > > > > > > To:     dev@kafka.apache.org
> > > > > > > > Date:   07/20/2018 11:09 AM
> > > > > > > > Subject:        Re: [DISCUSS] KIP-289: Improve the
default
> > group
> > > > id
> > > > > > > > behavior in KafkaConsumer
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > I agree with Jason's notion that
> > > > > > > > >  implicit use of the empty group.id to commit
offsets is
> > more
> > > > > likely
> > > > > > > to
> > > > > > > > be causing users unexpected problems than actually

providing a
> > > > > useful
> > > > > > > > capability.
> > > > > > > > I was initially confused that this is the behavior
when
> > > > > investigating
> > > > > > a
> > > > > > > > new-ish JIRA issue <
> > > > > > > >
> > > > > >
> > > > >
> > > >
> > 
https://issues.apache.org/jira/browse/KAFKA-6758

> >
> > > >
> > > > >
> > > > > >
> > > > > > > > > about
> > > > > > > > the same topic.
> > > > > > > > So, +1 to deprecating "" as a group.id
> > > > > > > >
> > > > > > > > The question after that becomes what the *default*
value
> > should
> > > be
> > > > -
> > > > > > > > should
> > > > > > > > we:
> > > > > > > > a) treat an unconfigured group.id consumer as a sort
of
> > > > intermittent
> > > > > > > > consumer where you don't store offsets at all (thereby

making
> > the
> > > > > user
> > > > > > > > explicitly sign up for them)
> > > > > > > > b) have a default value which makes use of them? I
sort of
> > like
> > > > the
> > > > > > > > former.
> > > > > > > >
> > > > > > > > @Dhruvil, thinking about it at a high-level - yes.
I can't
> > think
> > > > of
> > > > > a
> > > > > > > > situation where it makes sense to name something an
empty
> > string
> > > > as
> > > > > > far
> > > > > > > as
> > > > > > > > I'm aware - to me it seems like potential for confusion
> > > > > > > >
> > > > > > > >
> > > > > > > > On Fri, Jul 20, 2018 at 10:22 AM Rajini Sivaram
> > > > > > <rajinisivaram@gmail.com
> > > > > > > >
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > +1 to deprecate use of "" as group.id since it
is odd to
> > have
> > > a
> > > > > > > resource
> > > > > > > > > name that you cannot set ACLs for. Agree, we
have to 
support
> > > > older
> > > > > > > > clients
> > > > > > > > > though.
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > >
> > > > > > > > > Rajini
> > > > > > > > >
> > > > > > > > > On Fri, Jul 20, 2018 at 5:25 PM, Jason Gustafson
> > > > > > <jason@confluent.io>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Hi Vahid,
> > > > > > > > > >
> > > > > > > > > > Sorry for getting to this so late. I think
there are 
two
> > > > things
> > > > > > here:
> > > > > > > > > >
> > > > > > > > > > 1. The use of "" as a groupId has always
been a 
dubious
> > > > practice
> > > > > > at
> > > > > > > > best.
> > > > > > > > > > We definitely ought to deprecate its use
in the 
client.
> > > > Perhaps
> > > > > in
> > > > > > > the
> > > > > > > > > next
> > > > > > > > > > major release, we can remove support completely.

However,
> > > > since
> > > > > > older
> > > > > > > > > > clients depend on it, we may have to continue
letting 
the
> > > > broker
> > > > > > > > support
> > > > > > > > > it
> > > > > > > > > > to some extent. Perhaps we just need to
bump the
> > OffsetCommit
> > > > > > request
> > > > > > > > API
> > > > > > > > > > and only accept the offset commit for older
versions. 
You
> > > > > probably
> > > > > > > > have
> > > > > > > > > to
> > > > > > > > > > do this anyway if you want to introduce
the new error 
code
> > > > since
> > > > > > old
> > > > > > > > > > clients will not expect it.
> > > > > > > > > >
> > > > > > > > > > 2. There should be a way for the consumer
to indicate 
that
> > it
> > > > > has
> > > > > > no
> > > > > > > > > group
> > > > > > > > > > id and will not commit offsets. This is
an explicit
> > > > instruction
> > > > > > that
> > > > > > > > the
> > > > > > > > > > consumer should not bother with coordinator
lookup and
> > such.
> > > > We
> > > > > > > > currently
> > > > > > > > > > have some brittle logic in place to let
users avoid 
the
> > > > > > coordinator
> > > > > > > > > lookup,
> > > > > > > > > > but it is a bit error-prone. I was hoping
that we 
could
> > > change
> > > > > the
> > > > > > > > > default
> > > > > > > > > > value of group.id to be null so that the
user had to 
take
> > an
> > > > > > > explicit
> > > > > > > > > > action to opt into coordinator management
(groups or
> > > offsets).
> > > > > > > > However,
> > > > > > > > > it
> > > > > > > > > > is true that some users may be unknowingly
depending 
on
> > > offset
> > > > > > > storage
> > > > > > > > if
> > > > > > > > > > they are using both the default group.id
and the 
default
> > > > > > > > > > enable.auto.commit. Perhaps one option is
to disable
> > > > > > > > enable.auto.commit
> > > > > > > > > > automatically if no group.id is specified?
I am not 
sure
> > if
> > > > > there
> > > > > > > are
> > > > > > > > > any
> > > > > > > > > > drawbacks, but my feeling is that implicit
use of the
> > empty
> > > > > > group.id
> > > > > > > > to
> > > > > > > > > > commit offsets is more likely to be causing
users
> > unexpected
> > > > > > problems
> > > > > > > > > than
> > > > > > > > > > actually providing a useful capability.
> > > > > > > > > >
> > > > > > > > > > Thoughts?
> > > > > > > > > >
> > > > > > > > > > Thanks,
> > > > > > > > > > Jason
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > On Mon, May 28, 2018 at 9:50 AM, Vahid S
Hashemian <
> > > > > > > > > > vahidhashemian@us.ibm.com> wrote:
> > > > > > > > > >
> > > > > > > > > > > Hi Viktor,
> > > > > > > > > > >
> > > > > > > > > > > Thanks for sharing your opinion.
> > > > > > > > > > > So you're in favor of disallowing the
empty ("") 
group
> > id
> > > > > > > altogether
> > > > > > > > > > (even
> > > > > > > > > > > for fetching).
> > > > > > > > > > > Given that ideally no one should be
using the empty
> > group
> > > id
> > > > > (at
> > > > > > > > least
> > > > > > > > > in
> > > > > > > > > > > a production setting) I think the impact
would be
> > minimal
> > > in
> > > > > > either
> > > > > > > > > case.
> > > > > > > > > > >
> > > > > > > > > > > But as you said, let's hear what others
think and 
I'd be
> > > > happy
> > > > > > to
> > > > > > > > > modify
> > > > > > > > > > > the KIP if needed.
> > > > > > > > > > >
> > > > > > > > > > > Regards.
> > > > > > > > > > > --Vahid
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > From:   Viktor Somogyi <viktorsomogyi@gmail.com>
> > > > > > > > > > > To:     dev <dev@kafka.apache.org>
> > > > > > > > > > > Date:   05/28/2018 05:18 AM
> > > > > > > > > > > Subject:        Re: [DISCUSS] KIP-289:
Improve the
> > default
> > > > > group
> > > > > > id
> > > > > > > > > > > behavior in KafkaConsumer
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > Hi Vahid,
> > > > > > > > > > >
> > > > > > > > > > > (with the argument that using the default
group id 
for
> > > > offset
> > > > > > > commit
> > > > > > > > > > > should not be the user's intention
in practice).
> > > > > > > > > > >
> > > > > > > > > > > Yea, so in my opinion too this use
case doesn't seem 
too
> > > > > > practical.
> > > > > > > > > Also
> > > > > > > > > > I
> > > > > > > > > > > think breaking the offset commit is
not smaller from
> > this
> > > > > > > > perspective
> > > > > > > > > > than
> > > > > > > > > > > breaking fetch and offset fetch. If
we suppose that
> > someone
> > > > > uses
> > > > > > > the
> > > > > > > > > > > default group id and we break the offset
commit then
> > that
> > > > > might
> > > > > > be
> > > > > > > > > harder
> > > > > > > > > > > to detect than breaking the whole thing
altogether. 
(If
> > we
> > > > > think
> > > > > > > > about
> > > > > > > > > an
> > > > > > > > > > > upgrade situation.)
> > > > > > > > > > > So since we think it is not a practical
use case, I
> > think
> > > it
> > > > > > would
> > > > > > > > be
> > > > > > > > > > > better to break altogether but ofc
that's just my 2
> > cents
> > > > :).
> > > > > > Let's
> > > > > > > > > > gather
> > > > > > > > > > > other's input as well.
> > > > > > > > > > >
> > > > > > > > > > > Cheers,
> > > > > > > > > > > Viktor
> > > > > > > > > > >
> > > > > > > > > > > On Fri, May 25, 2018 at 5:43 PM, Vahid
S Hashemian <
> > > > > > > > > > > vahidhashemian@us.ibm.com> wrote:
> > > > > > > > > > >
> > > > > > > > > > > > Hi Victor,
> > > > > > > > > > > >
> > > > > > > > > > > > Thanks for reviewing the KIP.
> > > > > > > > > > > >
> > > > > > > > > > > > Yes, to minimize the backward
compatibility 
impact,
> > there
> > > > > > would
> > > > > > > be
> > > > > > > > no
> > > > > > > > > > > harm
> > > > > > > > > > > > in letting a stand-alone consumer
consume messages
> > under
> > > a
> > > > > ""
> > > > > > > > group
> > > > > > > > > id
> > > > > > > > > > > (as
> > > > > > > > > > > > long as there is no offset commit).
> > > > > > > > > > > > It would have to knowingly seek
to an offset or 
rely
> > on
> > > > the
> > > > > > > > > > > > auto.offset.reset config for the
starting offset.
> > > > > > > > > > > > This way the existing functionality
would be 
preserved
> > > for
> > > > > the
> > > > > > > > most
> > > > > > > > > > part
> > > > > > > > > > > > (with the argument that using
the default group id 
for
> > > > > offset
> > > > > > > > commit
> > > > > > > > > > > > should not be the user's intention
in practice).
> > > > > > > > > > > >
> > > > > > > > > > > > Does it seem reasonable?
> > > > > > > > > > > >
> > > > > > > > > > > > Thanks.
> > > > > > > > > > > > --Vahid
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > From:   Viktor Somogyi <viktorsomogyi@gmail.com>
> > > > > > > > > > > > To:     dev <dev@kafka.apache.org>
> > > > > > > > > > > > Date:   05/25/2018 04:56 AM
> > > > > > > > > > > > Subject:        Re: [DISCUSS]
KIP-289: Improve the
> > > default
> > > > > > group
> > > > > > > > id
> > > > > > > > > > > > behavior in KafkaConsumer
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > Hi Vahid,
> > > > > > > > > > > >
> > > > > > > > > > > > When reading your KIP I coldn't
fully understand 
why
> > did
> > > > you
> > > > > > > > decide
> > > > > > > > > at
> > > > > > > > > > > > failing with "offset_commit" in
case #2? Can't we 
fail
> > > > with
> > > > > an
> > > > > > > > empty
> > > > > > > > > > > group
> > > > > > > > > > > > id even in "fetch" or "fetch_offset"?
What was the
> > reason
> > > > > for
> > > > > > > > > deciding
> > > > > > > > > > > to
> > > > > > > > > > > > fail at "offset_commit"? Was it
because of upgrade
> > > > > > compatibility
> > > > > > > > > > > reasons?
> > > > > > > > > > > >
> > > > > > > > > > > > Thanks,
> > > > > > > > > > > > Viktor
> > > > > > > > > > > >
> > > > > > > > > > > > On Thu, May 24, 2018 at 12:06
AM, Ted Yu
> > > > > <yuzhihong@gmail.com>
> > > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > > Looks good to me.
> > > > > > > > > > > > > -------- Original message
--------From: Vahid S
> > > > Hashemian
> > > > > <
> > > > > > > > > > > > > vahidhashemian@us.ibm.com>
Date: 5/23/18  11:19 
AM
> > > > > > > (GMT-08:00)
> > > > > > > > > To:
> > > > > > > > > > > > > dev@kafka.apache.org Subject:
Re: [DISCUSS] 
KIP-289:
> > > > > Improve
> > > > > > > the
> > > > > > > > > > > default
> > > > > > > > > > > > > group id behavior in KafkaConsumer
> > > > > > > > > > > > > Hi Ted,
> > > > > > > > > > > > >
> > > > > > > > > > > > > Thanks for reviewing the
KIP. I updated the KIP 
and
> > > > > > introduced
> > > > > > > > an
> > > > > > > > > > > error
> > > > > > > > > > > > > code for the scenario described.
> > > > > > > > > > > > >
> > > > > > > > > > > > > --Vahid
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > From:   Ted Yu <yuzhihong@gmail.com>
> > > > > > > > > > > > > To:     dev@kafka.apache.org
> > > > > > > > > > > > > Date:   04/27/2018 04:31
PM
> > > > > > > > > > > > > Subject:        Re: [DISCUSS]
KIP-289: Improve 
the
> > > > default
> > > > > > > group
> > > > > > > > id
> > > > > > > > > > > > > behavior in KafkaConsumer
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > bq. If they attempt an offset
commit they will
> > receive
> > > > an
> > > > > > > error.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Can you outline what specific
error would be
> > > encountered
> > > > ?
> > > > > > > > > > > > >
> > > > > > > > > > > > > Thanks
> > > > > > > > > > > > >
> > > > > > > > > > > > > On Fri, Apr 27, 2018 at 2:17
PM, Vahid S 
Hashemian <
> > > > > > > > > > > > > vahidhashemian@us.ibm.com>
wrote:
> > > > > > > > > > > > >
> > > > > > > > > > > > > > Hi all,
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > I have drafted a proposal
for improving the
> > behavior
> > > > of
> > > > > > > > > > > KafkaConsumer
> > > > > > > > > > > > > when
> > > > > > > > > > > > > > using the default group
id:
> > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > >
> > > > > >
> > > > >
> > > >
> > 
https://cwiki.apache.org/confluence/display/KAFKA/KIP-

> >
> > > >
> > > > >
> > > > > >
> > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > > 
289%3A+Improve+the+default+group+id+behavior+in+
> > > > > > > KafkaConsumer
> > > > > > > > > > > > > > The proposal based on
the issue and suggestion
> > > > reported
> > > > > in
> > > > > > > > > > > KAFKA-6774.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Your feedback is welcome!
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Thanks.
> > > > > > > > > > > > > > --Vahid
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > Best,
> > > > > > > > Stanislav
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Best,
> > > > > > > Stanislav
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > >
> > > >
> > >
> >
> >
> >
> >
> >






Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message