kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Manikumar (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (KAFKA-2796) add support for reassignment partition to specified logdir
Date Wed, 21 Feb 2018 18:36:00 GMT

     [ https://issues.apache.org/jira/browse/KAFKA-2796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Manikumar resolved KAFKA-2796.
------------------------------
    Resolution: Fixed

Fixed in KIP-113 (KAFKA-5163/KAFKA-5694)

> add support for reassignment partition to specified logdir
> ----------------------------------------------------------
>
>                 Key: KAFKA-2796
>                 URL: https://issues.apache.org/jira/browse/KAFKA-2796
>             Project: Kafka
>          Issue Type: Improvement
>          Components: clients, controller, core, log
>            Reporter: Yonghui Yang
>            Assignee: Yonghui Yang
>            Priority: Major
>              Labels: features
>
> Currently when creating a log, the directory is chosen by calculating the number of partitions
> in each directory and then choosing the data directory with the fewest partitions.
> However, the sizes of different TopicParitions are very different, which lead to usage
vary greatly between different logDirs. And usually each logDir corresponds to a disk, so
the disk usage between different disks is very imbalance .
> The possible solution is to reassign partitions in high-usage logDirs to low-usage logDirs.
I change the format of /admin/reassign_partitions,add replicaDirs field. At reassigning
Partitions, when broker’s LogManager.createLog() is invoked , if replicaDir is specified
, the specified logDir will be chosen, otherwise the logDir with the fewest partitions will
be chosen.
> the old /admin/reassign_partitions:
>   {"version":1,
>    "partitions": 
>    [
>      {
>        "topic" : "Foo",
>        "partition": 1,
>        "replicas": [1, 2, 3]
>      }
>    ]
>   }
> the new /admin/reassign_partitions:
>   {"version":1,
>    "partitions": 
>    [
>      {
>        "topic" : "Foo",
>        "partition": 1,
>        "replicas": [1, 2, 3],
>        "replicaDirs": {"1":"/data1/kafka_data",  "3":"/data10/kakfa_data" }
>      }
>    ]
>   }
> This feature has been developed.
> PR: https://github.com/apache/kafka/pull/484



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message