kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ChenLin (JIRA)" <j...@apache.org>
Subject [jira] [Created] (KAFKA-8465) Copying strategy for the topic dimension
Date Sun, 02 Jun 2019 12:41:00 GMT
ChenLin created KAFKA-8465:

             Summary: Copying strategy for the topic dimension
                 Key: KAFKA-8465
                 URL: https://issues.apache.org/jira/browse/KAFKA-8465
             Project: Kafka
          Issue Type: Improvement
          Components: core
    Affects Versions: 2.2.1, 2.2.0, 2.0.1
            Reporter: ChenLin

When some partiton's replication is assigned to a broker, which disks should these copies
be placed on the broker? The original strategy is to allocate according to the number of partiitons,
but this will cause a partiton with too many topics to be stored on a disk, which may cause
disk hotspot problems.
In order to solve this problem, we propose an improved strategy: first ensure that the number
of partitions of each disk in the topic dimension is even. If the number of partitions of
a topic on two disks is equal, then sort according to the total number of partitions on the
disk. Select a disk with the least number of partitions to store the current replication.

This message was sent by Atlassian JIRA

View raw message