hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinayakumar B (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6247) Avoid timeouts for replaceBlock() call by sending intermediate responses to Balancer
Date Wed, 23 Jul 2014 09:35:38 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14071555#comment-14071555

Vinayakumar B commented on HDFS-6247:

Failure, even though related to Balancing, Its not caused by this patch.

In fact, its failed due to selection of a block belongs to "/system/balancer.id" for the movement
which is having default replication(3) and after movement it will not be detected as excess.
All other blocks in test having 1 replication.
So the calculation in TestBalancer#waitForBalancer(..) does not meet and test timesout. I
think this can be fixed in a separate jira if observed again.

Anyway, triggering the QA again.

> Avoid timeouts for replaceBlock() call by sending intermediate responses to Balancer
> ------------------------------------------------------------------------------------
>                 Key: HDFS-6247
>                 URL: https://issues.apache.org/jira/browse/HDFS-6247
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: balancer, datanode
>    Affects Versions: 2.4.0
>            Reporter: Vinayakumar B
>            Assignee: Vinayakumar B
>         Attachments: HDFS-6247.patch, HDFS-6247.patch, HDFS-6247.patch, HDFS-6247.patch
> Currently there is no response sent from target Datanode to Balancer for the replaceBlock()
> Since the Block movement for balancing is throttled, complete block movement will take
time and this could result in timeout at Balancer, which will be trying to read the status
> To Avoid this during replaceBlock() call in in progress Datanode  can send IN_PROGRESS
status messages to Balancer to avoid timeouts and treat BlockMovement as  failed.

This message was sent by Atlassian JIRA

View raw message