kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin McCabe" <cmcc...@apache.org>
Subject Re: [DISCUSS] KIP-631: The Quorum-based Kafka Controller
Date Fri, 23 Oct 2020 20:19:35 GMT
On Wed, Oct 21, 2020, at 05:51, Tom Bentley wrote:
> Hi Colin,
> 
> On Mon, Oct 19, 2020, at 08:59, Ron Dagostino wrote:
> > > Hi Colin.  Thanks for the hard work on this KIP.
> > >
> > > I have some questions about what happens to a broker when it becomes
> > > fenced (e.g. because it can't send a heartbeat request to keep its
> > > lease).  The KIP says "When a broker is fenced, it cannot process any
> > > client requests.  This prevents brokers which are not receiving
> > > metadata updates or that are not receiving and processing them fast
> > > enough from causing issues to clients." And in the description of the
> > > FENCED(4) state it likewise says "While in this state, the broker does
> > > not respond to client requests."  It makes sense that a fenced broker
> > > should not accept producer requests -- I assume any such requests
> > > would result in NotLeaderOrFollowerException.  But what about KIP-392
> > > (fetch from follower) consumer requests?  It is conceivable that these
> > > could continue.  Related to that, would a fenced broker continue to
> > > fetch data for partitions where it thinks it is a follower?  Even if
> > > it rejects consumer requests it might still continue to fetch as a
> > > follower.  Might it be helpful to clarify both decisions here?
> >
> > Hi Ron,
> >
> > Good question.  I think a fenced broker should continue to fetch on
> > partitions it was already fetching before it was fenced, unless it hits a
> > problem.  At that point it won't be able to continue, since it doesn't have
> > the new metadata.  For example, it won't know about leadership changes in
> > the partitions it's fetching.  The rationale for continuing to fetch is to
> > try to avoid disruptions as much as possible.
> >
> > I don't think fenced brokers should accept client requests.  The issue is
> > that the fenced broker may or may not have any data it is supposed to
> > have.  It may or may not have applied any configuration changes, etc. that
> > it is supposed to have applied.  So it could get pretty confusing, and also
> > potentially waste the client's time.
> >
> >
> When fenced, how would the broker reply to a client which did make a
> request?
> 

Hi Tom,

The broker will respond with a retryable error in that case.  Once the client has re-fetched
its metadata, it will no longer see the fenced broker as part of the cluster.  I added a note
to the KIP.

best,
Colin

>
> Thanks,
> 
> Tom
>

Mime
View raw message