hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Plamen Jeliazkov (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HDFS-8921) Add an option to Balancer for specifying the k-most over-utilized DNs or all over-utilized DNs as sources.
Date Wed, 18 Oct 2017 18:41:01 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16209798#comment-16209798
] 

Plamen Jeliazkov edited comment on HDFS-8921 at 10/18/17 6:40 PM:
------------------------------------------------------------------

Would it be acceptable to say something "k-most over-utilized DNs per iteration"? This would
be very trivial to implement but would not be equivalent to the include file example as the
set would change per iteration.

Example: [dn1: 75%, dn2: 60%, dn3: 25%], if max-sources was 1, then after on first iteration
dn1 would get chosen and we might see: [dn1:55%, dn2:60%, dn3:45%], and now the next iteration
would pick dn2.


was (Author: zero45):
Would it be acceptable to say something "k-most over-utilized DNs per iteration"? This would
be very trivial to implement but would not be equivalent to the include file example as the
set would change per iteration.

Example: {dn1: 75%, dn2: 60%, dn3: 25%}, if max-sources was 1, then after on first iteration
dn1 would get chosen and we might see: {dn1:55%, dn2:60%, dn3:45%}, and now the next iteration
would pick dn2.

> Add an option to Balancer for specifying the k-most over-utilized DNs or all over-utilized
DNs as sources.
> ----------------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-8921
>                 URL: https://issues.apache.org/jira/browse/HDFS-8921
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: balancer & mover
>            Reporter: Tsz Wo Nicholas Sze
>            Assignee: Tsz Wo Nicholas Sze
>
> Arpit suggested to add a separate option to source from the most over-utilized DataNodes
first so the administrator does not have to pass the source DNs manually; see [this comment|https://issues.apache.org/jira/browse/HDFS-8826?focusedCommentId=14700576&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14700576].
 The new option could allow specifying the k-most over-utilized DNs or all over-utilized DNs
as sources.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message