Mailing-List: contact dev-help@kafka.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@kafka.apache.org
MIME-Version: 1.0
Sender: ismaelj@gmail.com
In-Reply-To: <CAAaarBaztRO0A-d9b+uMf39cBOCHbYW7=dWjDkSJ5d3pYuS3tw@mail.gmail.com>
References: <CAAaarBYC5A7v8sWCDdEbacNTzOf03LC5jVVDEOBJ__AjZXu1kA@mail.gmail.com>
 <2418351484309552@web28j.yandex.ru> <CAAaarBbJiGhvR0xZ94DHRm9otKKXuP0cEpDQvj-aXCjTUkR2pw@mail.gmail.com>
 <161231485182708@web30o.yandex.ru> <CAAaarBbszz1AiyMGCT8QC5Jv1djz3Z6=vD6W70Re08+GmRc+aw@mail.gmail.com>
 <2868901485257412@web17m.yandex.ru> <CAAaarBYXeh1Fs_MP17pZzrNHnzHd3-i22WPaPXK6EVQcENOPSg@mail.gmail.com>
 <392271485944180@web29o.yandex.ru> <CAAaarBYVhaP-U6VUBCj7-02ERO1+NXSbUP7HNofqwS9hg67FHA@mail.gmail.com>
 <CAFc58G8X7Bz3bDdyfY8MJ2-xh3DVhmuUXZ9AVukjd99yi3uJPg@mail.gmail.com>
 <CAAaarBaPsDKudCp6jAdL_kA+nUL0A+JN8UVvS6HbOOmOTaa=BQ@mail.gmail.com>
 <CAFc58G_nLvNgt5=o9zwqsvLqX8FNVvShX=fmM-aitJq0Lv4b0g@mail.gmail.com>
 <CAAaarBbUcKJbfNnbsi8tcxeyhEN=XXXTdLf-1S+Ld4+CNsdUxw@mail.gmail.com>
 <CAFc58G849O6hSmqviZiN9+Df=iemD26iO8tn7mpoagDwv9hGpA@mail.gmail.com>
 <CAAaarBZ-HpNNp4y0DhG5sKeVgcNqPGaJbYkAR9d=e01cA4nvGw@mail.gmail.com>
 <CAAaarBY-D7C5xHp2j_d-es2zpWsRS2TcJMS6KEHO7KOUgbEBRQ@mail.gmail.com>
 <CAFc58G_cwQrQti1i1nKL4mdtYwK_bTyVCYqu1=U63k9xQyekAQ@mail.gmail.com>
 <CAAaarBYabR=8oNpfKsJ5gkUkD0eQMYgbQADVrJUtp5bq+a0X4A@mail.gmail.com>
 <CAFc58G_BadEk_ZP8ZuirsAHzT9JgF2oGes5BL1VpdqX5_HdNZQ@mail.gmail.com>
 <CAAaarBYK2k09gfuGJNDa1bVeA4x_n12kv4X7SNRhd-hLb9FTKw@mail.gmail.com>
 <CAFc58G8tbUwmsiBvA12vGs1ScYd16oKd9px9jBFE_6sDJo0zGg@mail.gmail.com>
 <CAAaarBapRxrS8bCd8C_mowYC4H9jw2VnVWom5rVyUJiwBvCKbw@mail.gmail.com>
 <CAFc58G-xmjykf61RqDewDMD+nveKHpDmobby2SSxR4tmii=BBw@mail.gmail.com>
 <CAAaarBaqN6znKS6QvHeiLZMxEf4eStDzFkShhjR6gwCoppaKqQ@mail.gmail.com>
 <CAFc58G-WKMmLto-Y6z2ZyV7yD4NZjwZtzwoN8xMUwJhOeejRdg@mail.gmail.com>
 <CAAaarBbyzrGMZbfzZUA2AJW44n9P-=EBTShskfTHVOa+wNZc+Q@mail.gmail.com>
 <CAFc58G8LnxGpxDekyK8cBainSQ+7u9EJnLcynFyDM-1dCGqbhA@mail.gmail.com>
 <CAAaarBZhxkxxrqA44XF_1zvZg5yiMoy9UYvtmunNi4kApjYVTw@mail.gmail.com>
 <CAFc58G_ZVUNhMTqNT=UEdCN-3dSUFFZcdS1NHnVONDPxdCtuZw@mail.gmail.com>
 <CAAaarBbdTm1xVNHmHnVo2jxpEZamLEoHbh59CoZSXttU5BHuLg@mail.gmail.com>
 <CAFc58G8NYkWbUFCAanxDBRk_3WLq8kAL9FHhgmctdK+1A4ipxQ@mail.gmail.com>
 <CAAaarBbonJk_Fs8n04d91VFnTrpSaoq=nBVvPYBYkgX2FSKTqQ@mail.gmail.com>
 <CAG+EsUfb0zo2tBJz8arZSgFo2U5qnv9H+_M3-+-ErEK4q5vhUw@mail.gmail.com>
 <CAAaarBY2HVyJ1=b7d=3m0-d8EF1FQW3ogsDZKgm6mCAtQLFqSw@mail.gmail.com>
 <CAOeJiJgjY8rW=LBie4UwTq9qa4SeYH8FbtP04AY-yaCBceqmzg@mail.gmail.com>
 <CAAaarBZKjHH9M+T4moczXFDmNDkW00E+U6Z5U7o-3Kkt34HMyw@mail.gmail.com>
 <CAG+EsUdMaiJNEGc0dGgpGG174B3mSANgjzrUZ0SFbQMRfVgixg@mail.gmail.com>
 <CAAaarBZrSUQt7jdwF=nQsS1aLBX7FjjigM2q9xue3+83Vim+cg@mail.gmail.com>
 <CAAaarBbdpj-rxgk--RRmjf_0OMRLe4uJiaPHvXU=eOO+DijYGQ@mail.gmail.com>
 <CAFc58G-NcV0OdN2Vy3CbrbJ8GpdknuG87h2=gxJZACceQ-hjWw@mail.gmail.com>
 <1497288987.733762.1006895272.4C41145A@webmail.messagingengine.com>
 <CAAaarBYmGfFZ5hJUKTG4jis3LsDvQijQnRJ3a4jC+yJt0DUyTw@mail.gmail.com>
 <CAAaarBaBBmmM9FyQCS2w1FM26AUug-cxwx-n=pDV4KZ8rtVKFw@mail.gmail.com>
 <CAAaarBZJkdC3opz2We0YUKPSdoYLVewiATw6AomD10KeUSQbnw@mail.gmail.com>
 <CAAaarBaOVW78RtR8kHiUFbn4St9YJuVbBRdNThWs-rBgdHjPBg@mail.gmail.com>
 <CAD5tkZZrypqmxvAa4C_sT9y8AONQcPDMjFUcTOha3OaU=R53JQ@mail.gmail.com> <CAAaarBaztRO0A-d9b+uMf39cBOCHbYW7=dWjDkSJ5d3pYuS3tw@mail.gmail.com>
From: Ismael Juma <ismael@juma.me.uk>
Date: Mon, 7 Aug 2017 13:28:48 +0100
Message-ID: <CAD5tkZbof7LELnMx=nsY4O4S3oU+PrY=2LbA=WSyKOHO7ZdcJQ@mail.gmail.com>
Subject: Re: [DISCUSS] KIP-113: Support replicas movement between log directories
To: dev@kafka.apache.org
Content-Type: multipart/alternative; boundary="94eb2c05c52a8c2afb055628ff1f"
archived-at: Mon, 07 Aug 2017 12:29:46 -0000

--94eb2c05c52a8c2afb055628ff1f
Content-Type: text/plain; charset="UTF-8"

Hi Dong,

Thanks for the explanation. Comments inline.

On Fri, Aug 4, 2017 at 6:47 PM, Dong Lin <lindong28@gmail.com> wrote:

> 1. Yes it has been considered. Here are the reasons why we don't do it
> through controller.
>
> - There can be use-cases where we only want to rebalance the load of log
> directories on a given broker. It seems unnecessary to go through
> controller in this case.
>

Even though this is true, not sure how common it will be.

 - If controller is responsible for sending ChangeReplicaDirRequest, and if
> the user-specified log directory is either invalid or offline, then
> controller probably needs a way to tell user that the partition
> reassignment has failed. We currently don't have a way to do this since
> kafka-reassign-partition.sh simply creates the reassignment znode without
> waiting for response. I am not sure that is a good solution to this.
>

Since the JSON is provided by the user, we would ideally validate its
contents before storing it. Why are the log directories different than the
other information in the JSON?

- If controller is responsible for sending ChangeReplicaDirRequest, the
> controller logic would be more complicated because controller needs to
> first send ChangeReplicaRequest so that the broker memorize the partition
> -> log directory mapping, send LeaderAndIsrRequest, and keep sending
> ChangeReplicaDirRequest (just in case broker restarted) until replica is
> created. Note that the last step needs repeat and timeout as the proposed
> in the KIP-113.
>
> Overall I think this adds quite a bit complexity to controller and we
> probably want to do this only if there is strong clear of doing so.
> Currently in KIP-113 the kafka-reassign-partitions.sh is responsible for
> sending ChangeReplicaDirRequest with repeat and provides error to user if
> it either fails or timeout. It seems to be much simpler and user shouldn't
> care whether it is done through controller.
>

If I understand correctly, the logic is the same in both cases, it's just a
question of where it lives. The advantage of placing it in the Controller
is that the whole reassignment logic is in one place (instead of split
between the tool and the Controller). That seems easier to reason about.

Also, how do you think things would work in the context of KIP-179? Would
the tool still invoke these requests or would it be done by the broker
receiving the alterTopics/reassignPartitions protocol call?

And thanks for the suggestion. I will add this to the Rejected Alternative
> Section in the KIP-113.
>
> 2) I think user needs to be able to specify different log directories for
> the replicas of the same partition in order to rebalance load across log
> directories of all brokers. I am not sure I understand the question. Can
> you explain a bit more why "that the log directory has to be the same for
> all replicas of a given partition"?


I think I misunderstood the schema. The KIP has the following example:

"partitions" : [
    {
      "topic" : str,
      "partition" : int,
      "replicas" : [int],
      "log_dirs" : [str]    <-- NEW. A log directory can be either "any",
or a valid absolute path that begins with '/'. This is an optional filed.
It is treated as an array of "any" if this field is not explicitly
specified in the json file.
    },
    ...
  ]

Is it right that `log_dirs` is an array in the same order as `replicas`?
That's a bit obscure and we should document it more clearly. Did we discard
the option of a more readable schema (i.e. a JSON object mapping a replica
id to a log dir) due to efficiency concerns? It would be good to include
that in the KIP.

Thanks,
Ismael

--94eb2c05c52a8c2afb055628ff1f--