hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Leon Gao (Jira)" <j...@apache.org>
Subject [jira] [Created] (HDFS-14894) Add balancer parameter to balance top N used nodes
Date Sat, 05 Oct 2019 22:21:00 GMT
Leon Gao created HDFS-14894:
-------------------------------

             Summary: Add balancer parameter to balance top N used nodes
                 Key: HDFS-14894
                 URL: https://issues.apache.org/jira/browse/HDFS-14894
             Project: Hadoop HDFS
          Issue Type: Improvement
          Components: balancer &amp; mover
            Reporter: Leon Gao
            Assignee: Leon Gao


We sometimes see a few of our datanodes reach very high usage (due to various reasons) and
we need to reduce their usage in an urgent situation.

We see two ways to achieve it currently,

-Calculate and reset balancing threshold.

-Pick nodes manually according to usage stats and put them in a file and use `-resource` flag.

However, both of them are not very intuitive or too much manual work in an urgent close-to-outage
situation. Add a small feature to automatically pick top N used hosts will be a straightforward
option, for example `-top 10` to only target top 10 used datanodes.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message