hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexey Kovyrin (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-5958) One very large node in a cluster prevents balancer from balancing data
Date Wed, 19 Feb 2014 00:51:29 GMT

    [ https://issues.apache.org/jira/browse/HDFS-5958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13904904#comment-13904904

Alexey Kovyrin commented on HDFS-5958:

I understand perfectly well why it is happening. I've reported the issue to make sure it will
be fixed and other users wouldn't need to spend hours pulling their hair out trying to figure
out what is going on with their balance processes hanging forever, promising to move data
around and not doing it. 

> One very large node in a cluster prevents balancer from balancing data
> ----------------------------------------------------------------------
>                 Key: HDFS-5958
>                 URL: https://issues.apache.org/jira/browse/HDFS-5958
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: balancer
>    Affects Versions: 2.2.0
>         Environment: Hadoop cluster with 4 nodes: 3 with 500Gb drives and one with 4Tb
>            Reporter: Alexey Kovyrin
> In a cluster with a set of small nodes and one much larger node balancer always selects
the large node as the target even though it already has a copy of each block in the cluster.
> This causes the balancer to enter an infinite loop and stop balancing other nodes because
each balancing iteration selects the same target and then could not find a single block to

This message was sent by Atlassian JIRA

View raw message