kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "huxihx (JIRA)" <j...@apache.org>
Subject [jira] [Created] (KAFKA-5560) LogManager should be able to create new logs based on free disk space
Date Thu, 06 Jul 2017 07:55:00 GMT
huxihx created KAFKA-5560:
-----------------------------

             Summary: LogManager should be able to create new logs based on free disk space
                 Key: KAFKA-5560
                 URL: https://issues.apache.org/jira/browse/KAFKA-5560
             Project: Kafka
          Issue Type: Improvement
          Components: log
    Affects Versions: 0.11.0.0
            Reporter: huxihx


Currently, log manager chooses a directory configured in `log.dirs` by calculating the number
partitions in each directory and then choosing the one with the fewest partitions. But in
some real production scenarios where data volumes of partitions are not even, some disks nearly
become full whereas the others have a lot of spaces which lead to a poor data distribution.

We should offer a new strategy to users to have log manager honor the real disk free spaces
and choose the directory with the most disk space. Maybe a new broker configuration parameter
is needed, `log.directory.strategy` for instance. Perhaps this needs a new KIP also.

Does it make sense?



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message