kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tom Bentley <t.j.bent...@gmail.com>
Subject Re: [DISCUSS] KIP-113: Support replicas movement between log directories
Date Fri, 11 Aug 2017 12:37:48 GMT
Hi Jun and Dong,

Thanks for your replies...

On 10 August 2017 at 20:43, Dong Lin <lindong28@gmail.com> wrote:

> This is a very good idea. I have updated the KIP-113 so that
> DescribeDirResponse returns lag instead of LEO.


Excellent!

On Thu, Aug 10, 2017 at 10:21 AM, Jun Rao <jun@confluent.io> wrote:
>
> > 2. Tom, note that currently, the LeaderAndIsrRequest doesn't specify the
> > log dir. So, I am not sure in your new proposal, how the log dir info is
> > communicated to all brokers. Is the broker receiving the
> > ReassignPartitionsRequest
> > going to forward that to all brokers?
>

My understanding of KIP-113 is that each broker has its own set of log dirs
(even though in practice they might all have the same names, and might all
be distributed across the brokers disks in the same way, and all those
disks might be identical), so it doesn't make sense for one broker to be
told about the log dirs of another broker.

Furthermore, it is the AlterReplicaDirRequest that is sent to the receiving
broker which associates the partition with the log dir on that broker. To
quote from KIP-113 (specifically, the notes in this section
<https://cwiki.apache.org/confluence/display/KAFKA/KIP-113%3A+Support+replicas+movement+between+log+directories#KIP-113:Supportreplicasmovementbetweenlogdirectories-1%29Howtomovereplicabetweenlogdirectoriesonthesamebroker>
):

- If broker doesn't not have already replica created for the specified
> topicParition when it receives AlterReplicaDirRequest, it will reply
> ReplicaNotAvailableException AND remember (replica, destination log
> directory) pair in memory to create the replica in the specified log
> directory when it receives LeaderAndIsrRequest later.
>

I've not proposed anything to change that, really. All I've done is change
who creates the znode which causes the LeaderAndIsrRequest. Because KIP-113
has been accepted, I've tried to avoid attempting to change it too much.

Cheers,

Tom

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message