kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tom Bentley <t.j.bent...@gmail.com>
Subject Re: [DISCUSS] KIP-178: Change ReassignPartitionsCommand to use AdminClient
Date Wed, 19 Jul 2017 14:05:01 GMT
OK, I will work on adding support for this to the KIP, with the intention
of a two part implementation.

On 19 July 2017 at 14:59, Ismael Juma <ismael@juma.me.uk> wrote:

> Hi Tom,
>
> It's fine for the tool not to have this functionality from the start.
> However, since we're adding new Kafka protocol APIs, we need to consider
> some of these details to ensure we're building towards the end state, if
> that makes sense. Protocol APIs are used by multiple clients, so there is
> value in thinking ahead a bit when it comes to the design. The
> implementation can often be done in stages.
>
> Does that make sense?
>
> Ismael
>
> On Wed, Jul 19, 2017 at 6:23 AM, Tom Bentley <t.j.bentley@gmail.com>
> wrote:
>
> > Hi Ismael,
> >
> > Answers in-line:
> >
> > 1. Have you considered how progress would be reported? Partition
> > > reassignment can take a long time and it would be good to have a
> > mechanism
> > > for progress reporting.
> > >
> >
> > The ReassignPartitionsCommand doesn't currently have a mechanism to track
> > progress. All you can do at the moment is initiate a reassignment (with
> > --execute), and later check whether the assignment is in the state you
> > asked for (with --verify). I agree it would be nice to be able to track
> > progress.
> >
> > This will be the first 'big' bit of work I've done on Kafka, so I would
> > prefer to limit the scope of this KIP where possible. That said, I
> suppose
> > it could be done by having receiving controllers publish their progress
> to
> > ZooKeeper, and adding Protocol and AdminClient API for getting this
> > information. If you're keen on this I can certainly modify the KIP to add
> > this.
> >
> > Alternatively I could write a second KIP to add this ability. What other
> > long running tasks are there for which we'd like the ability to report
> > progress? If there are others it might be possible to come up with a
> common
> > mechanism.
> >
> >
> > > 2. Removals can only happen in major releases. In your example, the
> > removal
> > > could only happen in 2.0.0.
> > >
> >
> > OK, I'll update the KIP.
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message